JP6573798B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method.

  When an infrared cut filter is applied to the imaging apparatus, color shading may occur in the captured image. In Patent Document 1, as an image processing apparatus that corrects such color shading, a plurality of shading correction coefficients are applied to a captured image, and a shading correction coefficient that can appropriately correct color shading is selected. An image processing apparatus that corrects and outputs color shading of the captured image by applying the selected shading correction coefficient is disclosed.

  Patent documents 2 and 3 are known as other related technologies.

JP 2013-198041 A JP 2006080752 A Japanese Patent Laying-Open No. 2015-099962

  In the conventional image processing apparatus, the captured image is divided into a plurality of blocks, the statistical value of the pixel value for each block is obtained, the statistical value is multiplied by the coefficient value for each block of the shading correction coefficient, and the multiplication result A dispersion value in the captured image is obtained. This dispersion value is obtained for each shading correction coefficient, and the shading correction coefficient having the smallest dispersion value is selected as a shading correction coefficient that can appropriately correct the color shading.

  However, the conventional technique has a problem that a shading correction coefficient that can appropriately correct color shading cannot be selected depending on an image.

  The present invention has been made to solve such problems, and an object thereof is to provide an image processing apparatus and an image processing method capable of selecting a shading correction coefficient suitable for an image.

  An image processing apparatus according to the present invention includes a block dividing unit that divides a captured image into a plurality of blocks, a statistical value calculating unit that calculates a block statistical value by taking pixel value statistics for each block, and a plurality of light sources According to at least one of an estimation coefficient group storage unit that stores a first shading estimation coefficient group including a plurality of shading estimation coefficients respectively corresponding to the image flatness, subject brightness, and subject color temperature of the captured image, A shading estimation unit that selects a second shading estimation coefficient group including at least one shading estimation coefficient appropriate for the captured image from the stored first shading estimation coefficient group; and the selected second shading estimation unit A shading correction coefficient corresponding to a shading estimation coefficient group is applied to the captured image, and shading of the captured image is performed. A shading correction unit that corrects, those comprising a.

  An image processing apparatus according to the present invention includes: a block dividing unit that divides a captured image into a plurality of blocks; and a statistical value calculation that calculates a first block statistical value by taking statistics of pixel values for each color for each block A block color evaluation value calculation unit that calculates a block color evaluation value for evaluating the calculated first block statistical value, a block coefficient evaluation value calculation unit that stores a plurality of shading estimation coefficients having different correction strengths, A block evaluation value calculation unit that selects an evaluation target block for shading estimation based on a histogram of the calculated block color evaluation value, and among the stored shading estimation coefficients based on the selected evaluation target block A shading estimation coefficient selection unit that selects an appropriate shading estimation coefficient for the captured image from the image, and the selected shading The shading correction coefficient corresponding to the estimated coefficient is applied to the captured image, a shading correction section that corrects the shading of the captured image, but with a.

  An image processing apparatus according to the present invention includes a block dividing unit that divides a captured image into a plurality of blocks, a statistical value calculating unit that calculates a block statistical value by taking statistics of pixel values for each color for each block, Based on an estimation coefficient storage unit that stores a plurality of shading estimation coefficients having different correction strengths, a group classification unit that classifies the blocks into a plurality of groups according to the distance from the center of the captured image, and the block statistics An effective block determination unit that determines whether the block is valid or invalid, and a group evaluation value calculation unit that calculates a group evaluation value based on the sum of each statistical value for each of the blocks determined to be valid in the group; An effective group determination unit that determines whether the group is valid or invalid based on the calculated group evaluation value; and a central portion of the captured image Among the groups heading toward the periphery, the straight-line approximation unit that linearly approximates the color ratio for the group that is determined to be effective, and the imaging from the stored shading estimation coefficients according to the slope of the approximated straight line A shading estimation coefficient selection unit that selects an appropriate shading estimation coefficient for the image, a shading correction unit that applies a shading correction coefficient corresponding to the selected shading estimation coefficient to the captured image, and corrects shading of the captured image; , Are provided.

  An image processing apparatus according to the present invention includes a group classification unit that classifies an image into a plurality of groups based on a distance from a center of the image, and a group for each group when a shading estimation coefficient is applied to the image. A group evaluation value calculating unit for calculating an evaluation value; an effective group determining unit for determining an effective group based on the group evaluation value; and determining the adjacent effective group as a group continuous region; and the group continuous region A shading estimation coefficient selecting unit that accumulates the variance values of the group evaluation values to obtain a table evaluation value of the shading estimation coefficient, selects a shading estimation coefficient having the smallest table evaluation value, and shading corresponding to the selected shading estimation coefficient A shading correction that corrects the image using a correction coefficient. In which and a part.

  An image processing method according to the present invention divides a captured image into a plurality of blocks, calculates a block statistical value by taking statistics of a pixel value for each block, and a plurality of shading estimation coefficients respectively corresponding to a plurality of light sources. And storing the first shading estimation coefficient group from the stored first shading estimation coefficient group according to at least one of image flatness, subject brightness, and subject color temperature of the captured image. Selecting a second shading estimation coefficient group including at least one shading estimation coefficient appropriate for the captured image, applying a shading correction coefficient corresponding to the selected second shading estimation coefficient group to the captured image, and This is to correct shading of a captured image.

  The image processing method according to the present invention divides a captured image into a plurality of blocks, calculates a first block statistical value by taking statistics of pixel values for each color for each of the blocks, and a plurality of different correction intensities. A shading estimation coefficient is stored, a block color evaluation value for evaluating the calculated first block statistical value is calculated, and an evaluation target block for shading estimation is calculated based on a histogram of the calculated block color evaluation value. Selecting a shading estimation coefficient appropriate for the captured image from the stored shading estimation coefficients based on the selected evaluation target block, and a shading correction coefficient corresponding to the selected shading estimation coefficient Is applied to the captured image to correct shading of the captured image.

  An image processing method according to the present invention divides a captured image into a plurality of blocks, calculates a block statistic value by taking a statistic of a pixel value for each color for each block, and a plurality of shading estimation coefficients having different correction strengths And classifying the blocks into a plurality of groups according to the distance from the center of the captured image, determining whether the block is valid or invalid based on the block statistics, and determining whether the block is valid or invalid. A group evaluation value is calculated based on the sum of each statistical value by color for the determined block, and the validity or invalidity of the group is determined based on the calculated group evaluation value. For the group judged to be effective among the groups heading to the section, the color ratio is linearly approximated, and the memory is stored according to the slope of the approximated straight line. Selecting an appropriate shading estimation coefficient for the captured image from among the estimated shading estimation coefficients, applying a shading correction coefficient corresponding to the selected shading estimation coefficient to the captured image, and correcting shading of the captured image It is.

  The image processing method according to the present invention classifies images into a plurality of groups based on distances from the center of the image, and calculates a group evaluation value for each group when a shading estimation coefficient is applied to the image. And determining an effective group based on the group evaluation value, determining the adjacent effective group as a group continuous region, integrating a variance value of the group evaluation value for each group continuous region, and calculating the shading estimation coefficient A shading estimation coefficient having the smallest table evaluation value is selected as a table evaluation value, and the image is corrected using a shading correction coefficient corresponding to the selected shading estimation coefficient.

  ADVANTAGE OF THE INVENTION According to this invention, the image processing apparatus and image processing method which can select the shading correction coefficient suitable for an image can be provided.

1 is a block diagram illustrating a configuration of a digital still camera according to Embodiment 1. FIG. 3 is a block diagram showing a configuration of a shading estimation unit according to Embodiment 1. FIG. 6 is a flowchart for explaining an operation of a shading estimation unit according to the first embodiment. 6 is a diagram illustrating an example of an image block for explaining an operation of a flat determination unit according to Embodiment 1. FIG. 6 is a diagram illustrating an example of an image value for explaining an operation of a flat determination unit according to Embodiment 1. FIG. 6 is a diagram illustrating an example of an image processed by a flat determination unit according to Embodiment 1. FIG. 5 is a flowchart for explaining an operation of a shading estimation coefficient selection unit according to the first embodiment. It is a figure which shows the example of the group which the shading estimation coefficient selection part which concerns on Embodiment 1 classifies. It is a figure which shows the example of another group which the shading estimation coefficient selection part which concerns on Embodiment 1 classifies. 10 is a diagram illustrating an example of an image for explaining an overview of a second embodiment. FIG. 6 is a block diagram showing a configuration of a shading estimation unit 124 according to Embodiment 2. FIG. 10 is a flowchart for explaining an operation of a block color evaluation value calculation unit 211 according to the second embodiment. 10 is a flowchart for explaining operations of a block color evaluation value calculation unit 211, a block weight coefficient calculation unit 212, and a block evaluation value calculation unit 213 according to the second embodiment. 10 is a flowchart for explaining operations of a block color evaluation value calculation unit 211, a block weight coefficient calculation unit 212, and a block evaluation value calculation unit 213 according to the second embodiment. It is a figure which shows the example of the image which the block weighting coefficient calculation part 212 which concerns on Embodiment 2 processes. It is a figure which shows the example of the histogram which the block weight coefficient calculation part 212 which concerns on Embodiment 2 produces | generates. It is a figure which shows the example of the image which the block weighting coefficient calculation part 212 which concerns on Embodiment 2 processes. FIG. 10 is a block diagram illustrating a configuration of a shading estimation unit according to Embodiment 3. 10 is a flowchart for explaining operations of a group classification unit and a group evaluation value calculation unit according to the third embodiment. It is a figure which shows the example of the group which the group classification part which concerns on Embodiment 3 classifies. 10 is a flowchart for explaining operations of an effective group determination unit and an approximate estimation unit according to Embodiment 3. 10 is a flowchart for explaining operations of an effective group determination unit and an approximate estimation unit according to Embodiment 3. It is a figure which shows the example of the group which the effective group determination part which concerns on Embodiment 3 processes. It is a figure which shows the example of the group which the effective group determination part which concerns on Embodiment 3 processes. It is a figure which shows the example of the group which the effective group determination part which concerns on Embodiment 3 processes. 10 is a diagram for explaining an interpolation method of an effective group determination unit according to Embodiment 3. FIG. 10 is a graph showing an approximate straight line image processed by the approximate estimation unit according to the third embodiment. 10 is a flowchart for explaining an operation of a variance estimation unit according to the third embodiment. FIG. 10 is a block diagram showing a configuration of a shading estimation unit 124 according to Embodiment 4. 10 is a flowchart for explaining an operation of a shading estimation unit 124 according to Embodiment 4. FIG. 10 is a diagram for explaining an effective distance group determination method according to Embodiment 4; FIG. 10 is a diagram for describing specific processing contents of an effective distance group determination method according to Embodiment 4;

(Embodiment 1)
The first embodiment will be described below with reference to the drawings.

<Outline of Embodiment 1>
As described above, in the conventional image processing apparatus described in Patent Document 1, the shading correction coefficient having the smallest variance value is selected as the shading correction coefficient that can appropriately correct the color shading. However, the present inventor has found that the shading correction coefficient with the smallest variance value does not necessarily become a shading correction coefficient that can appropriately correct color shading when the imaging scene is complicated in the prior art.

  In order to solve such a problem, the present inventor, as a study example of an image processing apparatus that corrects color shading, applies a shading estimation coefficient to a block statistical value and calculates a block characteristic value for each block. And a block weight unit that calculates a block weight based on the block characteristic similarity between the block of interest and the neighboring block, and an example of selecting an appropriate shading estimation coefficient by eliminating the influence of a complicated subject has been studied. In addition, an estimation unit that applies a plurality of shading estimation coefficients to an image, selects an appropriate shading estimation coefficient for the image, and a correction that corrects shading of the image by applying a shading correction coefficient corresponding to the appropriate shading estimation coefficient to the image. And an estimation unit classifies the images into a plurality of groups based on the distance from the central portion of the image, calculates an image characteristic value for each group when a shading estimation coefficient is applied to the image, and An example of calculating a statistical value of an image characteristic value and selecting an appropriate shading estimation coefficient for the image, and an example of improving the estimation accuracy by calculating an evaluation weight according to the complexity of the image were studied.

  As a result of the examination, shading of unknown intensity has occurred in the captured image, so the shading estimation coefficient applied to the block statistics value does not necessarily match the correction of the captured image, and it is corrected to the difference from the neighboring block. It has been found that the block weight may be low even for a flat subject due to the influence of shading that cannot be achieved. In other words, even with the above-described technique of the examination example, it is necessary to process an object having a signal difference (complexity) equivalent to the signal difference caused by shading between the central portion of the image and the peripheral portion of the image as flat. There is a risk of selecting an incorrect correction coefficient.

  The first embodiment further improves the above-described conventional techniques and examination examples, and makes it possible to suppress erroneous selection of a shading correction coefficient suitable for an image. That is, in the first embodiment, attention is paid to the feature that the image signal smoothly increases from the image edge toward the center and the image signal smoothly attenuates from the image center to the periphery, and is particularly noticeable when shading occurs. Provided is an image processing apparatus capable of selecting a shading correction coefficient more accurately than in the prior art by detecting a scene of a flat subject and selecting a correction coefficient.

<Configuration of Embodiment 1>
FIG. 1 is a block diagram illustrating a configuration of a digital still camera 100 including the image processing apparatus according to the first embodiment. As shown in FIG. 1, a digital still camera 100 that is an electronic imaging device includes a lens optical system 102, an imaging device 104, an AFE circuit 106, an image signal processing circuit 108, an image display unit 110, an image recording unit 112, and a driver 114. , A timing generator (TG) 116, an image block statistical circuit 118, a control unit 120, a coefficient storage unit 122, and the like.

  The lens optical system 102 includes a lens, a diaphragm, a shutter, and the like, and forms a subject image on the imaging surface of the imaging element 104. The image sensor 104 is an image sensor such as a CCD or a CMOS, and an infrared cut filter (not shown) is mounted on the lens optical system 102 side of the image sensor 104. The image sensor 104 photoelectrically converts the subject image to obtain an image signal (RGB color signal). The AFE circuit 106 A / D converts an image signal acquired by the image sensor 104 and subjected to signal processing by a CDS circuit (not shown) into a digital signal.

  The image signal processing circuit 108 performs demosaicing processing, edge enhancement processing, white balance (WB) correction processing, shading correction processing, gamma correction processing, and the like on the image signal output from the AFE circuit 106. The image display unit 110 is a liquid crystal display (LCD) or the like, and displays an image signal that has been subjected to various processes by the image signal processing circuit 108. The image recording unit 112 is a memory, and records the image signal that has been subjected to various processes by the image signal processing circuit 108.

  The driver 114 drives the lens, aperture, and shutter of the lens optical system 102. The timing generator 116 generates timing for driving the image sensor 104. The image block statistical circuit 118 divides the captured image area or a part of the captured image area of the image signal converted into a digital signal by the AFE circuit 106 into a plurality of blocks, and calculates a block statistical value for each block. calculate. It can be said that the image block statistical circuit 118 includes a block dividing unit and a statistical value calculating unit. The image block statistical circuit 118 calculates a value indicating the image characteristic of each block, such as a pixel integrated value for each RGB and a pixel average value for each RGB, as a block statistical value. The image block statistical circuit 118 may calculate a value indicating the image characteristic of each block other than the statistical value.

  The control unit 120 controls white balance correction processing of the image signal processing circuit 108 based on the block statistical value calculated by the image block statistical circuit 118. In addition, the control unit 120 includes a shading estimation unit 124 and a shading correction unit 126.

  The shading estimation unit 124 estimates shading based on the block statistical value calculated by the image block statistical circuit 118 and the shading estimation coefficient stored in the coefficient storage unit 122, and selects a shading estimation coefficient suitable for the image. The shading estimation coefficient is prepared for each light source, for each R value, G value, and B value, and has a coefficient value for each block. The configuration and operation of the shading estimation unit 124 will be described later. A set of a plurality of shading estimation coefficients set for each light source is referred to as a shading estimation coefficient group.

  The shading correction unit 126 selects a shading correction coefficient corresponding to the shading estimation coefficient selected by the shading estimation unit 124 from the coefficient storage unit 122, and corrects shading of the captured image in the image signal processing circuit 108. The shading correction coefficient is prepared for each light source and for each R value, G value, and B value, and has a coefficient value for each pixel or for each of a plurality of pixels.

  The coefficient storage unit (estimation coefficient storage unit and correction coefficient storage unit) 122 stores a combination of a shading estimation coefficient and a shading correction coefficient for each light source such as sunlight, a light bulb, and a fluorescent lamp. In many cases, the shading estimation coefficient and the shading estimation coefficient are in a one-to-one pair, but the present invention is not limited to this. The shading estimation coefficient and the shading correction coefficient can be calculated by an external device such as a personal computer. The coefficient storage unit 122 stores a shading estimation coefficient group including a plurality of shading estimation coefficients corresponding to each light source, and also stores a shading estimation coefficient group including a plurality of shading estimation coefficients having different correction strengths. It can also be said.

  The shading estimation coefficient is calculated using a white chart image taken under each light source. For the white chart, it is desirable to use a uniform diffuse reflection surface having a spectral reflectance such as a standard white reflector that is constant over the entire target wavelength range and 90% or more. For a light source such as a light bulb that contains a large amount of infrared light, a shading estimation coefficient that strongly corrects the R signal at the periphery of the image is calculated. For a light source such as a fluorescent lamp that does not contain much infrared light, a shading estimation coefficient for correcting the R signal at the periphery of the image weakly is calculated. In this specification, the shading estimation coefficient that corrects the R signal at the periphery of the image to the weakest level is set as an initial setting (default) shading estimation coefficient, and is particularly referred to as a shading estimation initial coefficient.

  The image processing apparatus according to the first embodiment includes the image signal processing circuit 108, the image block statistical circuit 118, the control unit 120, the coefficient storage unit 122, and the like, but is not limited to this configuration. Absent.

  Moreover, each component which the control part 120 implement | achieves is realizable by making a program run by control of the arithmetic unit (not shown) with which the control part 120 which is a computer is provided, for example. More specifically, the control unit 120 is realized by loading a program stored in a storage unit (not shown) into a main storage device (not shown) and executing the program under the control of the arithmetic unit. Each component is not limited to being realized by software by a program, and may be realized by any combination of hardware, firmware, and software.

  The above-described program can be stored using various types of non-transitory computer readable media and supplied to a computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROMs (Read Only Memory), CD-ROMs. R, CD-R / W, semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)).

  Further, the program may be supplied to the computer by various types of temporary computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

  FIG. 2 is a block diagram showing the configuration of the shading estimation unit 124 according to the first embodiment. The shading estimation unit 124 includes a flat determination unit 201, a shading estimation coefficient selection unit 202, and the like. Note that the present embodiment is not limited to this configuration because it is only necessary to realize the operation of the first embodiment described later. For example, a means for obtaining the brightness of the subject and a means for obtaining the color temperature of the subject may be provided.

  The flat determination unit 201 determines whether or not the photographed subject is flat. The shading estimation coefficient selection unit 202 selects a shading correction suitable for an image from the shading estimation coefficient group according to whether or not the subject is flat and according to imaging conditions such as subject brightness and light source color temperature. Select a coefficient.

<Operation of Embodiment 1>
FIG. 3 is a flowchart for explaining the operation of shading estimation section 124 according to the first embodiment. First, the flat determination unit 201 determines whether or not the photographed subject is flat (step S1). The flat determination unit 201 obtains the degree of signal change due to the subject in the local area of the image by excluding the signal change due to shading that has already occurred in the image, so that whether the captured subject is flat or not. Determine whether. As will be described later, the brightness and color temperature may be determined. Either step S2 or step S3 may be performed according to at least one of image flatness, subject brightness, and subject color temperature.

  When it is determined in step S1 that the subject is flat, the shading estimation coefficient selection unit 202 calculates from all the shading estimation coefficients prepared in advance (first shading estimation coefficient group) by Expression (13) described later. A shading estimation coefficient having the smallest evaluation value Ed is selected (step S2).

  On the other hand, when it is determined in step S1 that the subject is not flat (complex), the shading estimation coefficient selection unit 202 selects at least one shading estimation coefficient to be evaluated from the first shading estimation coefficient group. The second shading estimation coefficient group is selected to be selected (step S3).

  Subsequently, the shading estimation coefficient selection unit 202 selects a shading estimation coefficient having the smallest evaluation value Ed from the second shading estimation coefficient group (step S4). Further, when selecting the second shading estimation coefficient group, the shading estimation coefficient selection unit 202 selects a shading estimation coefficient to be evaluated according to imaging conditions such as subject brightness and light source color temperature.

  Next, the operation (step S1 in FIG. 3) of the flat determination unit 201 according to the first embodiment will be described. FIG. 4 schematically shows detection blocks (captured images) used by the flat determination unit 201. The flat determination unit 201 obtains the image flatness by integrating pixel value differences of blocks that are continuous from the peripheral part to the central part of the captured image.

  The flat determination unit 201 determines whether or not the subject is flat using the G pixel value of the block corresponding to the diagonal line of the image. As shown in FIG. 4, the image is divided into four lines so that the blocks on the diagonal of the image become the blocks to be determined. For example, from the upper left block 301 to the central block 302 on the L2C line extending from the upper left to the center of the image, from the central block 303 to the lower right block 304 on the C2R line extending from the center of the image to the lower right, from the upper right to the center of the image The blocks from the upper right block 305 to the center block 306 on the extending R2C line and the blocks from the center block 307 to the lower left block 308 on the C2L line extending from the center of the image to the lower left are set as the determination target blocks. Note that, here, a line extending from the corner of the image to the center and a line extending from the center to the diagonal are illustrated, but other lines for dividing the image may be used.

  FIG. 5 shows an example of the G pixel value on the diagonal line of the image. In the case of a flat subject, for example, the G pixel values from the upper left block 301 to the central block 302 and from the central block 303 to the lower right block 304 are as shown in FIG. That is, when the difference between adjacent blocks from the upper left block 301 to the central block 302 is added and the difference between adjacent blocks from the central block 303 to the lower right block 304 is subtracted (upG + dwG), the value becomes close to zero. The same applies to the upper right block 305 to the central block 306 and the central block 307 to the lower left block 308.

In this way, the flat determination process shown in the following equation is performed using the feature of the change in the pixel value of the adjacent block due to the occurrence of shading. First, the flat determination unit 201 includes an L2C line extending from the upper left to the center of the image, a C2R line extending from the center of the image to the lower right, an R2C line extending from the upper right to the center of the image, and a G of a C2L line extending from the center of the image to the lower left. The integration of the increment value (upG) and the G decrement value (dwG) is obtained by the following equation (1).

Next, the flat determination unit 201 obtains the line evaluation value El1 of the L2C line and the C2R line and the line evaluation value El2 of the R2C line and the C2L line by the following expression (2) using the result of the above expression 1.

Next, the flat determination unit 201 obtains the flat evaluation value Ef from the line evaluation values El1 and El2 of the above equation 2 using the following equation (3).

  When the flat evaluation value Ef is less than the threshold value as a result of the above expression (3), the flat determination unit 201 determines that the object is a flat object, and determines that the object is not flat when the flat evaluation value Ef is equal to or greater than the threshold value. . For example, in the case of the image of FIG. 6A, since the flat evaluation value Ef is less than the threshold value, it is determined that the subject is flat. In the case of the image of FIG. 6B, the flat evaluation value Ef is greater than or equal to the threshold value. Therefore, it is determined that the subject is not flat.

  Next, an operation (step S3 in FIG. 3) in which the shading estimation coefficient selection unit 202 according to the first embodiment selects the second shading estimation coefficient group from the first shading estimation coefficient group will be described. The shading estimation coefficient selection unit 202 selects a second shading estimation coefficient group to be used for shading estimation based on a combination of values of whether the subject is flat, subject brightness (Bv), and color temperature (R Color Gain).

In the example shown below, the second shading estimation coefficient group is obtained according to the light source color temperature and the subject brightness (brightness). For example, as shown in Table 1 below, a first shading estimation coefficient group including six types of shading estimation coefficients (shading estimation table) respectively corresponding to six types of light sources is stored in advance. Here, IR is the amount of infrared light contained in the light source. As the table number increases, the shading estimation coefficient having a stronger R correction strength corresponds to a light source with more infrared light.

In addition, a light bulb color threshold (R Gain Limit Threshold), an indoor threshold (Indoor Bv Threshold), and an outdoor threshold (Outdoor Bv Threshold) are set in advance. Select a shading estimation coefficient group. When the brightness is smaller than the indoor threshold, it can be determined that the image is an indoor image, and when the brightness is larger than the outdoor threshold, it can be determined that the image is an outdoor image. When the brightness is equal to or greater than the indoor threshold and equal to or less than the outdoor threshold, it can be determined that the image is from indoor to outdoor. When the color temperature is lower than the light bulb color threshold, it can be determined that the image includes a light source of the light bulb, and when the color temperature is equal to or higher than the light bulb color threshold, it can be determined that the image includes a light source other than the light bulb.

  That is, as shown in Table 2, when the subject is a flat object or when Bv <indoor threshold (condition 1) and R Color Gain <bulb color threshold (condition 2), the results are shown in Table 1. From the first shading estimation coefficient group, the shading estimation coefficients (shading estimation tables 0 to 5) of all the light sources are selected as the second shading estimation coefficient group. Further, in the case of a flat subject or when Bv <indoor threshold (condition 1) and R Color Gain ≧ light bulb color threshold (condition 2), a light source other than the bulb color light source is selected from Table 1. A shading estimation coefficient (for example, shading estimation tables 0 to 3) is selected. In the case of a non-flat subject (Condition 1) and when the indoor threshold ≦ Bv ≦ Outdoor threshold (Condition 2), the shading estimation coefficient (for example, shading estimation) of the outdoor light source from Table 1 is used. Select Tables 1-3). When the subject is not flat (condition 1) and Bv> outdoor threshold (condition 2), the shading estimation coefficient of the outdoor light source (for example, shading estimation tables 1 to 2) is selected from Table 1. ) Is selected. That is, when it is determined that the captured image is flat, the second shading estimation coefficient group is selected from the first shading estimation coefficient group based on the subject color temperature, and it is determined that the captured image is not flat The second shading estimation coefficient group is selected from the first shading estimation coefficient group based on the subject brightness. When it is determined that the captured image is an indoor image based on the subject brightness, a second shading estimation coefficient group is selected from the first shading estimation coefficient group based on the subject color temperature. Thereby, the shading estimation coefficient group suitable for the image can be selected.

  Next, as shown in FIG. 3, the shading estimation coefficient selection unit 202 selects a shading estimation coefficient from the first shading estimation coefficient group (step S2), or shading from the selected second shading estimation coefficient group. The operation (step S4) for selecting the estimation coefficient will be described. These are similar operations, and FIG. 7 shows an example of this operation. That is, according to the operation of FIG. 7, the shading estimation coefficient selection unit 202 selects a shading estimation coefficient from the first or second shading estimation coefficient group, but the present invention is not limited to this.

  First, the shading estimation coefficient selection unit 202 uses the block statistical value calculated from the image subjected to shading correction by the image block statistical circuit 118, and the average value Rav, R value, G value, and B value of all the blocks. Gav and Bav are calculated (step S010).

Next, normalization coefficients Rg, Bg, and Gg are calculated as shown in equation (4) (step S020). The normalization coefficients Rg, Bg, and Gg are coefficients for matching the RGB balance of the entire screen, and are gains that normalize the image.

Next, as shown in the equation (5), the shading estimation coefficient Cr [t] (x) for all the block statistical values R (x, y), G (x, y), and B (x, y) , y), Cg [t] (x, y), Cg [t] (x, y) and normalization coefficients Rg, Bg, Gg are applied and multiplied, and Rc [t] (x, y) is applied to each block. ), Gc [t] (x, y), Bc [t] (x, y) are calculated (step S030).

  Here, the aforementioned shading estimation initial coefficient is used as the shading estimation coefficient in order to calculate the R signal ratio for each group for selecting a reference group to be described later. In addition, the normalization coefficient is applied to determine the degree of color difference of the block from the RGB average value by regarding the RGB average value of the captured image as an achromatic color. Rc [t] (x, y), Gc [t] (x, y), Bc [t] (x, y) are equally close to 1.0 as the shading estimation coefficient is more suitable for an image. When [t] (x, y), Gc [t] (x, y), and Bc [t] (x, y) are imaged, the image becomes a single gray color.

Next, as shown in equation (6), an R signal ratio Rr [t] (x, y) that is a block characteristic value is obtained for each block (step S040, step S050).

  Note that the denominator of equation (6) may be the sum of Rc [t] (x, y), Gc [t] (x, y), and Bc [t] (x, y). Further, in addition to the R signal ratio for each block or instead of the R signal ratio for each block, the B signal ratio for each block may be calculated and used for the subsequent processing.

  In the equation (6), since the R signal ratio is obtained for the RGB image to which the normalization coefficient is applied, the chromatic subject, for example, the entire image becomes red as the shading estimation coefficient is suitable for the image. Even if the subject is a small subject, the R signal ratio is close to 1.0. Conversely, the R signal ratio becomes a value farther from 1.0 when the shading estimation coefficient is not suitable for an image.

Next, the shading estimation coefficient selection unit 202 uses the spatial filter m to change the R signal ratio difference Rd (x, y) between the block of interest and its neighboring blocks, as shown in equations (7) and (8). And the weight Wb (x, y) for each block is calculated as shown in the equation (9) (steps S060 and S070). Rdmax is the maximum value of the R signal ratio difference Rd (x, y). The complexity of the photographed subject and the complexity of the local area of the image are reflected in the block weight, and when the image change is small and smooth, that is, when the similarity between blocks is large, the block weight is increased. When the change is large, that is, when the similarity between blocks is small, the block weight is calculated to be small.

  In calculating the block weight Wb (x, y), the B signal ratio may be used in addition to the R signal ratio or instead of the R signal ratio.

  Next, the shading estimation coefficient selection unit 202 calculates an R signal ratio for each group that is a group characteristic value (steps S080 and S090). FIG. 8 is a diagram for explaining an example of a group according to the first embodiment. The shading estimation coefficient selection unit 202 classifies each block acquired by the image block statistical circuit 118 by dividing the image region into a grid pattern into a plurality of groups based on the distance from the center of the image. In the image 400 shown in FIG. 8, each block of the image is classified into six groups from group 0 to group 5 from the center of the image toward the periphery of the image.

Then, as shown in the equation (10), the block weight Wb (x, y) is used to average the R signal ratio Rr (i) of the blocks included in the group, and the R signal ratio GpRr [j for each group. ] Is calculated. By using the block weight Wb (x, y) based on the similarity between the target block and the neighboring block, the change in the image is small, and the characteristic value of the smooth block and the area formed by those blocks is more emphasized. Each characteristic value can be calculated. For example, in an image that shows green leaves, red flowers, sky, buildings, etc., the block or area characteristic values that are adjacent to green leaves and red flowers are more The characteristic value of the area can be emphasized.

Next, the shading estimation coefficient selection unit 202 selects a reference group (step S100). A search is performed sequentially from the group in the center of the image toward the outer group, and one group is selected as the reference group. For example, the search is started from the group 0 shown in FIG. 8, and the search is performed up to the group 3 that is considered not affected by the shading. At this time, as shown in the equation (11), threshold determination of the R signal ratio GpRr [j] of the group is performed, and a group satisfying the equation (11) is first selected as a reference group. The threshold values Th1 and Th2 are set to, for example, 0.9 and 1.1 so that 1.0 is included between them. Since the reference group is selected after applying the normalization coefficient, the reference group can be selected not only for an achromatic subject but also for a uniform chromatic subject.

Next, the presence / absence of a reference group is determined (step S110). When there is a reference group (Yes in step S110), a group evaluation value is calculated (step S120, step S130). The group evaluation value is calculated for each group outside the reference group selected in step S100. For example, when group 1 is selected as the reference group, a group evaluation value is calculated for each of the outer groups 2 to 5. As shown in the equation (12), the difference between the R signal ratio GpRr [j] of the reference group and the R signal ratio GpRr [j + k] of the outer group is obtained, and the weight Wg [j + k] of the outer group prepared in advance is obtained. Each group evaluation value Dg [j + k] is calculated by multiplication.

The group weight Wg is prepared for each group as shown in Table 3 below. The group at the periphery of the image has a larger weight Wg. As a result, it is possible to more accurately evaluate the suitability of the shading correction of the shading estimation coefficient at the periphery of the image.

Next, as shown in the equation (13), the sum of absolute values of the group evaluation values Dg [j + k] is taken to obtain the evaluation value Ed of the shading estimation coefficient applied in step S030 (step S140).

  This evaluation value Ed is used to extract the characteristic of the R signal ratio that attenuates from the center to the periphery of the image. Whether or not the shading estimation coefficient used in step S030 is appropriate for the image subjected to shading correction. It becomes an index when judging.

  Next, the shading estimation coefficient selection unit 202 evaluates all the shading estimation coefficients (the shading estimation coefficients included in the first shading estimation coefficient group or the second shading estimation coefficient group) for the image to be subjected to shading correction. It is determined whether the value Ed has been calculated (step S150). When the evaluation values Ed of all the shading estimation coefficients have not been calculated (No in step S150), the process returns to step S030, and the shading estimation coefficients for which the evaluation value Ed has not been calculated, that is, shading estimation other than the shading estimation initial coefficient, are returned. The processing from step S030 to step S150 is repeated for the coefficient.

  When calculating the evaluation value Ed of the shading estimation coefficient other than the shading estimation initial coefficient, the selection of the reference group in step S100 and the determination of the presence or absence of the reference group in step S110 are not performed. Then, the reference group selected when calculating the evaluation value Ed of the shading estimation initial coefficient, for example, the group 1 is selected as the reference group when calculating the evaluation value Ed of the shading estimation coefficient other than the shading estimation initial coefficient. To do. That is, the evaluation value Ed of the shading estimation coefficient other than the shading estimation initial coefficient is calculated based on the reference group selected when calculating the evaluation value Ed of the shading estimation initial coefficient.

  Finally, after the shading estimation coefficient selection unit 202 calculates the evaluation values Ed of all the shading estimation coefficients (Yes in step S150), among all the shading estimation coefficients including the shading estimation initial coefficient, the evaluation value Ed is The smallest one is selected as the shading estimation coefficient suitable for the image to be subjected to shading correction (step S160).

  Further, when there is no reference group (No in step S110), the shading estimation coefficient selection unit 202 selects a shading estimation initial coefficient as a shading estimation coefficient suitable for an image to be subjected to shading correction (step S160). ).

  In the first embodiment, as shown in FIG. 8, the shading estimation coefficient selection unit 202 performs the process from the reference group selection process to the evaluation value calculation process for the entire image to be subjected to shading correction (steps S100 to S100). Step S150) is performed, but the image to be subjected to the shading correction may be divided into a plurality of areas, and the process from the reference group selection process to the evaluation value calculation process may be performed in each area.

  FIG. 9 is a diagram for explaining another group according to the first embodiment. The shading estimation coefficient selection unit 202 divides the image 401 to be subjected to shading correction into four regions I to IV, and further blocks each block in the regions I to IV into a plurality of groups based on the distance from the center of the image. Classify. For example, in the image 401 shown in FIG. 9, in the region I, each block is classified into five groups from group I-1 to group I-5. Then, the shading estimation coefficient selection unit 202 performs from the reference group selection process to the evaluation value calculation process in each of the areas I to IV. Thereby, the evaluation value of each shading estimation coefficient can be calculated for each of the regions I to IV, and the shading estimation coefficient suitable for the subject can be selected for each of the regions I to IV.

  In the first embodiment, the image processing apparatus has been described using the digital still camera 100 as an example. However, the image processing apparatus is configured by an external apparatus such as a personal computer, and the apparatus performs shading estimation and shading correction of the image. You may make it do.

  In the first embodiment, the estimated coefficient selected as described in Patent Document 1 may be selected and used as a correction coefficient as it is. That is, the shading correction coefficient according to the first embodiment may be different from the shading estimation coefficient or may be the same as the shading estimation coefficient.

  In the first embodiment, the shading estimation coefficient selection unit 202 calculates the evaluation value Ed of the shading estimation coefficient by the sum of the absolute values of the group evaluation value Dg. You may obtain | require by dispersion | distribution etc. That is, the evaluation value of the shading estimation coefficient according to the first embodiment may be obtained from some statistical value of the group evaluation value.

  Further, in the first embodiment, the shading estimation coefficient selection unit 202 obtains each group evaluation value Dg by obtaining a difference between the R signal ratio GpRr of the reference group and the R signal ratio GpRr of the outer group. Although it is obtained by multiplying the group weight Wg, each group evaluation value may be obtained as a difference between the R signal ratio of the reference group and the R signal ratio of the outer group. That is, multiplying the weight of the outer group may be omitted.

  In Embodiment 1, the shading estimation coefficient is evaluated using the RGB color space. However, the shading estimation coefficient may be evaluated using another color space such as HSV. Specifically, when calculating a block weight that is a degree of similarity between blocks, the RGB values may be subjected to HSV conversion, and the block weight may be calculated according to a change in saturation or luminance between the blocks.

  As described above, the first embodiment includes a statistical value calculation unit that divides a captured image into a plurality of blocks and statistically calculates a pixel value for each block, and performs shading estimation generated in the captured image from the block statistical values. In the image processing apparatus to perform, the image flatness excluding the influence of shading that occurs in the image is obtained, and the evaluation target among the plurality of shading estimation coefficients (first shading estimation coefficient group) prepared in advance according to the image flatness A shading estimation coefficient selection unit that selects at least one estimation coefficient and sets it as a second shading estimation coefficient group, and estimates and corrects shading generated in the image using the second shading estimation coefficient group; Side effects caused by selecting an incorrect shading correction coefficient can be reduced. In the first embodiment, it is possible to detect a scene in which shading estimation is likely to be erroneous in accordance with subject brightness, subject complexity, and color temperature, and to suppress the occurrence of side effects due to the constraint conditions of the shading estimation table.

(Embodiment 2)
The second embodiment will be described below with reference to the drawings.

<Outline of Embodiment 2>
As described above, the conventional technique and the examination example are examined. In the first embodiment, even if the technique such as the examination example is used, the signal difference (complexity) is the same as the signal difference due to the shading between the central portion of the image and the peripheral portion of the image. To solve the problem of selecting an incorrect correction coefficient, it is determined whether the photographed subject is flat (simple) and shading estimation is performed. The estimation accuracy can be improved by limiting the shading estimation coefficients to be selected from the coefficient group.

  On the other hand, in the case of the image 410 including a flat subject in which two or more different hues are mixed as shown in FIG. 10, an erroneous estimation may occur even if any of the above methods is used. In the image 410 of FIG. 10, a blue flat subject is included on a gray background. For example, in a non-complex monotonous scene in which flat subjects of different hues such as blue and red are mixed, the block characteristic similarity of neighboring blocks is high, so that the subject is not complicated. In such a scene, flat portions having different hues have the same weight, so that the shading estimation result may be erroneously estimated due to the influence of the subject color.

  The second embodiment further improves the above examination example and the first embodiment, and reduces erroneous estimation of shading estimation in a scene in which a plurality of monotonous hues that are not complicated locally are mixed. That is, in the second embodiment, an image processing apparatus and an image processing method that can suppress erroneous estimation of a shading correction coefficient suitable for an image by extracting and evaluating a block of a hue that occupies a large area in the image. I will provide a.

<Configuration of Embodiment 2>
Since the configuration of the digital still camera 100 including the image processing apparatus according to the second embodiment is the same as that of FIG. 1 of the first embodiment, the description thereof is omitted.

  FIG. 11 is a block diagram showing a configuration of shading estimation section 124 according to the second embodiment. The shading estimation unit 124 includes a block color evaluation value calculation unit 211, a block weight coefficient calculation unit 212, a block evaluation value calculation unit 213, a shading estimation coefficient selection unit 214, and the like. Note that the present embodiment is not limited to this configuration because it is only necessary to realize the operation of the second embodiment described later. For example, the block color evaluation value calculation unit 211 and the block weight coefficient calculation unit 212 may be a single block.

  The block color evaluation value calculation unit 211 calculates a block color evaluation value Hb using the block statistical value and the normalization gain. The block weight coefficient calculation unit 212 calculates a histogram weight from the block color evaluation value Hb, and calculates a block weight Wb using the histogram weight and the G level weight. The block evaluation value calculation unit 213 calculates a block evaluation value Eb using the block statistical value and the block weight Wb. The shading estimation coefficient selection unit 214 calculates a group evaluation value using the block evaluation value Eb, and selects a shading estimation coefficient (shading estimation table) based on the change amount of the group evaluation value.

<Operation of Embodiment 2>
Next, a process for extracting a block of a hue that occupies a large area in an image according to the second embodiment will be described. For this purpose, the shading estimation unit 124 calculates the block weight Bw by using the block statistical value of the target image for shading correction multiplied by the shading estimation initial coefficient.
FIG. 12 is a flowchart for explaining the operation of the block color evaluation value calculation unit 211 according to the second embodiment. First, the block color evaluation value calculation unit 211 normalizes the R, G, and B values in the block to be evaluated. Therefore, by the operation in FIG. Multiplication by an estimation coefficient (shading estimation initial coefficient) reduces shading and calculates a normalized gain.

That is, the block color evaluation value calculation unit 211 multiplies the block detection data by the shading estimation initial coefficient using the following equation (14) (step S201). When the block statistical values are Ri, Gi, Bi and the coefficient values of the shading estimation initial coefficients are Nr, Ng, Nb, the block statistical values Rn (N, M), Gn (N, Nb) after multiplication of the shading estimation initial coefficients M) and Bn (N, M) are expressed by the equation (14).

  Next, block statistical values Rn (N, M), Gn (N, M), and Bn (N, M) multiplied by the shading estimation initial coefficient are added up for all blocks (steps S202 and S203), and the block statistical values are obtained. A normalization gain (Average Gain) is calculated from the ratio. Here, the normalization gain of the G pixel is 1.0, and the normalization gain of the R pixel and the B pixel is obtained.

Specifically, the block color evaluation value calculation unit 211 obtains a normalization gain using the following equation (15).

  That is, it is determined whether or not the integrated value of Rn (N, M) or Bn (N, M) is 0 (steps S204 and S207). If 0, the normalized gain (Average Gain R / R) of the R pixel or B pixel is determined. (Average Gain B) is set to 1.0 (steps S206 and S209). On the other hand, when the integrated value of Rn (N, M) or Bn (N, M) is not 0, normalized gain of R pixel = G integrated value / R integrated value or normalized gain of B pixel = G integrated value / The B integrated value is set (steps S205 and S208). In order to prevent overflow, the normalization gain of the R pixel or B pixel is clipped to the upper limit with the value of the normalization gain upper limit (Average Gain Limit) (R, B) (set as the upper limit).

  13A and 13B are flowcharts for explaining operations of the block color evaluation value calculation unit 211, the block weight coefficient calculation unit 212, and the block evaluation value calculation unit 213 according to the second embodiment. Following the operation of FIG. 12, the block weight coefficient calculation unit 212 calculates the G level weight for the purpose of excluding dark blocks and blocks near saturation from the evaluation measures as shown in FIG. 14 (step S210). The G level weight refers to the level of the G value of the block statistical value after being multiplied by the shading estimation coefficient, and the block within the range of the upper limit threshold and the lower limit threshold set to 1.0 is set to 0. 0. Further, a block in which any one of the R pixel value, the G pixel value, and the B pixel value is outside the preset range may be excluded from the evaluation target block.

Subsequently, hue blocks occupying a large area in the image are extracted in the subsequent processing. First, the block color evaluation value calculation unit 211 calculates block color evaluation values Hb for all blocks (step S211). Like the equation (16), the normalized gains (B, N, M) are multiplied by the block statistical values Gn (N, M) and Bn (N, M) obtained by multiplying the shading estimation initial coefficients obtained by the equations (14) and (15). (Average Gain) is multiplied to calculate a block color evaluation value Hb (N, M). It can be said that the block color evaluation value Hb is obtained based on the second block statistical value by multiplying the first block statistical value by the normalization gain to obtain the second block statistical value. The block color evaluation value may be obtained based on at least one of the R ratio, the G ratio, and the B ratio of the second block statistical value. The ratio of R may be three times the ratio of R to all RGB (3R / (R + G + B)) as shown in the equation (16), or may be the ratio of R to all RGB (R / (R + G + B)).

  Next, the block weight coefficient calculation unit 212 generates a histogram of the block color evaluation value Hb (step S212), and calculates histogram weights for the purpose of excluding blocks that are far from the mode value (steps S213 to S218). . FIG. 15A shows an example of a histogram to be generated and histogram weights, and is a setting example in which 0.65 to 1.35 are bins with 0.05 increments.

  Specifically, the mode value of the histogram is searched (step S213), and for each block, it is determined whether the block color evaluation value Hb is within the effective range from the histogram mode value (step S214), and FIG. ), The histogram weight = 1 (step S215) when the value is within the effective range, and the histogram weight = 0 when the value is outside the effective range (step S217). Then, the block having the histogram weight = 1 is selected as the evaluation target block. Further, following step S215, as shown in FIG. 16, the G level weight and the histogram weight are multiplied to generate a block weight Bw (step S216). These processes are repeated to calculate the block weight Bw for all blocks (step S218). Thereby, a block classified into bins in the vicinity of the mode value of the histogram can be selected as an evaluation target block. The shading estimation unit 124 performs processing up to the calculation of the block weight Bw (steps S201 to S218) by using the block statistical value of the target image for shading correction multiplied by the shading estimation initial coefficient.

Next, the shading estimation unit 124 calculates a shading estimation evaluation value for each shading estimation coefficient (shading estimation table), and selects a shading estimation coefficient for appropriately shading correcting the image. Specifically, first, the block evaluation value calculation unit 213 uses the equation (17) to multiply the block statistical value before shading correction by the coefficient value of one shading estimation table of the shading estimation candidate, and after the multiplication. Block statistical values Rc and Gc are obtained (steps S219 and S220). If the block statistical values before shading correction are Ri and Gi, and the coefficient values of the estimation candidate shading estimation table are Cr [t] and Cg [t], the block statistical value Rc (N , M) and Gc (N, M) are expressed by equation (17).

Next, the block evaluation value Eb (N, M) obtained by multiplying the block weight Bw (N, M) is calculated for each block (N, M) using the equation (18) (steps S221 and S222).

  It can be said that the block evaluation value calculation unit 213 selects the evaluation target block by calculating the block evaluation value Eb. After calculating the block evaluation value Eb of all blocks, the shading estimation coefficient selection unit 214 excludes the block evaluation value in the group that is 0 for each group in which the image area is divided according to the distance from the image center. Average to make the group evaluation value. Then, the amount of change in the group evaluation value from the central group to the peripheral group is used as the shading estimation evaluation value. After repeating the above processing to calculate the shading estimation evaluation value for each shading estimation coefficient, the shading estimation coefficient selection unit 214 calculates an appropriate shading estimation coefficient (shading correction coefficient) based on these shading estimation evaluation values. select. The processing after the block evaluation value calculation may be processed as in the third embodiment described later.

  As described above, the second embodiment includes a statistical value calculation unit that divides a captured image into a plurality of blocks and statistically calculates pixel values for each block, and performs shading estimation generated in the captured image from the block statistical values. In the image processing apparatus, the effective block used for the shading estimation is selected by the histogram of the R ratio calculated by the block statistical value “3R / (R + G + B)” or “R / (R + G + B)”. The estimation can be reduced and the shading estimation accuracy can be improved. In the second embodiment, since the blocks near the histogram mode value of the R ratio are evaluated, shading error estimation can be suppressed in a scene in which a plurality of hues are mixed.

(Embodiment 3)
The third embodiment will be described below with reference to the drawings.

<Outline of Embodiment 3>
In the image processing apparatus according to Patent Document 3, each block of the image is classified into a plurality of groups based on the distance from the center of the image, and the image characteristic value when the shading estimation table is applied to the image is determined for each group. The image characteristic value for each group is weighted according to the distance from the center of the image, the reference group is searched for sequentially toward the outer group from the group at the center of the image, and the R ratio of the reference group and the reference The absolute value of the difference from the R ratio of the group outside the group is obtained, and the sum of multiplications of group weights prepared in advance is used as the evaluation value of the shading estimation table. As a result, an effect of reducing erroneous determination when there is a subject with high saturation in the center of the image is obtained.

  However, in the prior art, an erroneous determination may be made when there is a chromatic subject in the periphery of the reference group.

  In the third embodiment, the block statistical value to which the shading estimation table for each light source prepared in advance is applied is determined whether the block is valid or invalid, the validity of the group composed of a set of blocks is determined, and the center of the image is determined. A determination unit that determines the continuity of the effective group from the group to the surrounding group is newly provided, and the invalid group is interpolated and enabled according to the determination result of the group continuity, and a group of subject areas of similar colors is extracted. By selecting a shading correction table appropriate for correcting the captured image according to the slope of the approximate straight line of the R ratio from the center to the periphery of the image of the effective group, erroneous determination by the subject is reduced.

<Configuration of Embodiment 3>
Since the configuration of the digital still camera 100 provided with the image processing apparatus according to the third embodiment is the same as that of FIG. 1 of the first embodiment, the description thereof is omitted.

  FIG. 17 is a block diagram showing a configuration of shading estimation section 124 according to the third embodiment. In addition to the configuration of the second embodiment, a group classification unit 221, a group evaluation value calculation unit 222, an effective group determination unit 223, an approximate estimation unit 224, and a variance estimation unit 225 are provided. In this example, the approximate estimation unit 224 and the variance estimation unit 225 are included in the shading estimation coefficient selection unit 214. Note that the present embodiment is not limited to this configuration as long as the operation of the third embodiment described later can be realized. For example, the shading estimation coefficient selection unit 214 may select a shading estimation coefficient according to the approximation of the approximation estimation unit 224.

  The group classification unit 221 classifies the image blocks into a plurality of groups according to the distance from the center of the image. The group evaluation value calculation unit 222 calculates a group evaluation value based on the block statistical value. The group evaluation value calculation unit 222 may include an effective block determination unit that determines whether the block is valid or invalid based on the block statistical value. The group evaluation value calculation unit 222 performs grouping based on the sum of the statistical values for each of the effective blocks in the group. An evaluation value is calculated. The effective group determination unit 223 determines the validity / invalidity of the group based on the group evaluation value or the number of effective blocks in the group. The approximate estimation unit (straight line approximation unit) 224 calculates an approximate expression that approximates correction by the shading estimation table based on the group evaluation value, and selects the shading estimation table based on the approximate expression. It can be said that the approximate estimation unit 224 linearly approximates the ratio of the color (R) for the effective group from the center to the periphery of the captured image. The variance estimation unit 225 calculates the sample variance of the block evaluation value and selects a shading estimation table that minimizes the variance value.

<Operation of Embodiment 3>
In the third embodiment, first, the same operation as the calculation up to the block evaluation value Eb in the second embodiment is performed. FIG. 18 is a flowchart for explaining operations of the group classification unit 221 and the group evaluation value calculation unit 222 according to the third embodiment.

  First, as shown in FIG. 19, the group classification unit 221 divides the detection block into four quadrants Q [0] to [3], and further groups the groups into G [0] to [g] according to the distance from the center. (Step S301).

  Next, the group evaluation value calculation unit 222 multiplies the block detection data by an estimation candidate shading estimation table (Small Table) (step S302), and calculates a block evaluation value Eb (step S303). As in the second embodiment, the block statistics value before shading correction is multiplied by the shading estimation table of the estimation candidate using the equation (17), and the block weight Bw is further multiplied by the equation (18). An evaluation value Eb is calculated.

Next, the group evaluation value calculation unit 222 calculates the group integrated value Eg [g] obtained by integrating the block evaluation value Eb (N, M) for each of the groups G [0] to [g] as shown in the equation (19). (Step S304), and the number of effective blocks Nb [g] obtained by integrating the block weights Bw (N, M) for each of the groups G [0] to [g] is calculated as shown in the equation (20) ( Step S305). The above processing (steps S301 to S306) is performed on the shading estimation tables of all estimation candidates. The group evaluation value calculation unit 222 sets the sum of the block statistical values of the effective blocks among the block statistical values to which the shading estimation coefficients included in the group are sequentially applied as the group integrated value Eg.

  Next, a method for evaluating a shading estimation table (shading estimation method) according to the third embodiment will be described. In the third embodiment, approximate line estimation is performed using the least square method based on the group integrated value Eg [g], and the table number t obtained from the slope value of the approximate line is used as the estimation result.

  FIG. 20A and unit 20B are flowcharts for explaining operations of effective group determination unit 223 and approximation estimation unit 224 according to Embodiment 3.

  First, the effective group determination unit 223 determines the effective block number Nb [g] for each of the groups G [0] to [g] (step S310). When the number of effective blocks Nb <the threshold number of effective blocks, the number of effective blocks Nb = 0.

Next, the effective group determination unit 223 calculates an average value Ag [g] of the block evaluation values Eb for each of the groups G [0] to [g] using the equation (21) (step S311). When the number of effective blocks Nb <the number of effective blocks in a group threshold, an average value Ag = 0, and a group in which the number Nb of effective blocks (block weight Bw = 1) is less than the number of effective blocks in a group (Enable Group Count Threshold) Exclude from estimation.

Next, using the equation (22), the effective group determination unit 223 averages the average value Ag [g] of the block evaluation values Eb for each of the distances D [0] to [d] from the center of the image, and performs group evaluation. A value Ad [d] is calculated (step S312). FIG. 21 shows an example of a group of distances D [0] to [6].

Next, the effective group determination unit 223 calculates the difference value Diff of the group evaluation value Ad with the adjacent distance group for each distance group (step S313). The above processing (steps S310 to S313) is also performed with the shading estimation tables of all estimation candidates.
Next, the effective group determination unit 223 uses the value of the group evaluation value Ad [d] for the default shading estimation table (Yes in step S314), and determines whether each group of distances D [0] to [d] is valid / Invalidity determination (Ed = 1: Valid, Ed = 0: Invalid) is performed (step S315). The group evaluation value Ad [d] = 0, or a group having a distance D having a large or small difference Diff of the group evaluation value Ad from the adjacent distance group is invalid. That is, when the difference value Diff is within MinTh (minimum threshold) to MaxTh (maximum threshold), it is valid (Ed = 1), and when the difference value Diff is outside MinTh to MaxTh, it is invalid (Ed = 0). 22A is an example in which D [0] to [1] are invalid, and D [2] to “6” are valid. In FIG. 22B, D [0] to [3] and [6] are valid. , D [4] to [5] are invalid examples.

Next, the effective group determination unit (invalid group enabling unit) 223 has two or more consecutive effective distance groups (Ed = 1) on both sides of a single invalid distance group (Ed = 0), and When the difference Diff of the group evaluation value Ad between the single invalid distance group and the adjacent effective distance groups is equal to or smaller than the invalid distance group interpolation threshold, Ed = 0 of the single invalid distance group is interpolated to 1, and the effective distance is interpolated. Grouping is performed (step S316).
The effective group determination unit 223 includes a group continuity determination unit that determines the continuity of the effective group among the groups from the center to the periphery of the captured image, and the group determined to be invalid according to the determination result of the group continuity. It may include an invalid group validation unit to validate. For example, the invalid group enabling unit selects an invalid group when there are two or more consecutive valid groups on both sides of a single invalid group and the difference between two valid groups adjacent to the invalid group is equal to or less than a threshold value. Activate. FIG. 23A is an example in which the difference between the group evaluation values Ad on both sides of D [2] is equal to or smaller than the threshold value, and FIG. 23B is an example in which both sides of D [1] are continuous. Therefore, FIG. 23C is an example in which D [2] to [3] are not independent because Ed = 0 is not independent.

  Next, the effective group determination unit (group continuity determination unit) determines an estimation method based on the number Ne of effective distances (Ed = 1) (step S317). When the number of Ne is small, there is a high possibility that the subject is a complicated subject that is difficult to estimate, so the process proceeds to variance estimation. That is, if Ne> effective distance group continuation number threshold value, approximate line estimation processing described later is performed, and if Ne ≦ effective distance group continuation number threshold value, variance estimation processing (step S318) is performed. The processing so far (steps S315 to S318) is performed using a default shading estimation table (shading estimation initial coefficient).

  Next, the approximate estimator (approximate straight line estimator) 224 calculates an approximate straight line of the group evaluation value Ad of the effective distance (Ed = 1) using the least square method for all the estimation candidate shading estimation tables ( In steps S320 and S321), an optimum table number t is obtained from the value of the inclination (step S322). At this time, the effective distance (Ed = 1) obtained in Steps S315 and S316 using the default shading estimation table is applied to the shading estimation tables of all estimation candidates. For example, if the effective distance of the default shading estimation table is D [2] to [4], the effective distance of all the shading estimation tables is set to D [2] to [4]. If the slope of the approximate straight line is negative, the table number closest to 0 among the negative slopes is used as the estimation result (step S323). If the slope is not negative (positive), the minimum value table is used. The number is used as an estimation result (step S324). The approximate estimation unit 224 (or the shading estimation coefficient selection unit) may select a shading estimation coefficient group such that the approximated straight line is close to horizontal or closest to a preset reference inclination. . When approximating the ratio of R, the ratio of R to all RGB (R / (R + G + B)) may be used, or three times the ratio of R to all RGB (3R / (R + G + B)) may be used. Good.

  Next, the difference between the table number of the estimation result and the previous estimation table number is obtained (step S325). When the difference is 1, the update of the estimation result is determined using hysteresis so that the estimation result is stabilized. That is, when the estimation result table number-previous estimation table number = 1, it is determined whether or not Slope (slope)> SlopeHys (hysteresis slope) *-1 is satisfied (step S326). When the number = -1, it is determined whether or not Slope (slope) <SlopeHys (hysteresis slope) is satisfied (step S327). When Slope> SlopeHys * -1 is satisfied (Yes in step S326) or when Slope <SlopeHys is satisfied (Yes in step S327), the previous estimation result is maintained (steps S328 and S330). In step S325, if the estimation result table number−the previous estimation table number is other than ± 1 (else in step S325), or if Slope> SlopeHys * −1 is not satisfied (No in step S326), or Slope <SlopeHys. If not satisfied (No in step S327), the estimation result is fixed to the current table number (step S329).

  FIG. 24 shows an approximate straight line image for each estimation candidate shading estimation table. As shown in FIG. 24, a table that is not overcorrected and the correction result is almost horizontal ((a) in FIG. 24) is used as the estimation result. If all the tables are overcorrected, the table with the weakest correction is used as the estimation result.

  FIG. 25 is a flowchart for explaining the operation of variance estimation section 225 according to Embodiment 3. In the case of a complex subject for which approximate line estimation is difficult, the variance estimation unit 225 obtains a sample variance from the block evaluation average value Ab that is an average value of the block evaluation values Eb of the entire image and each block evaluation value Eb (effective block). Only table), and the table number t with the smallest variance value is taken as the estimation result.

That is, the variance estimation unit 225 calculates the block evaluation average value Ab of the effective block using the equation (23) (step S331). The number Nb of effective blocks (block weight Bw = 1) and the block evaluation average value Ab are expressed by equation (23).

Next, the variance estimation unit 225 calculates and stores the sample variance Varp of the effective block using the equation (24) (steps S332 and S333), and uses the shading estimation table of all estimation candidates, The above process is repeated (step S334).

Next, the variance estimation unit 225 uses the equation (25) as the estimation result for the table number t that minimizes the variance value (step S335).

  As described above, Embodiment 3 includes a statistical value calculation unit that divides a captured image into a plurality of blocks and statistically calculates pixel values for each block, and estimates shading generated in the captured image based on the block statistical values. In the image processing apparatus that performs the above, a shading estimation table corresponding to a plurality of light sources is sequentially applied to the captured image. For the block image to which the shading estimation table is applied, the blocks are grouped according to the distance from the block at the center of the image, and it is determined whether the group is used for estimation. The R ratio of the image peripheral group to the image peripheral group is linearly approximated. By selecting a shading correction table corresponding to the shading estimation table on a one-to-one basis according to the inclination of the approximate line for the estimation table corresponding to a plurality of light sources, the shading can be corrected with higher accuracy than in the prior art. In the third embodiment, since the table evaluation value is calculated by evaluating the continuity of the R ratio change from the center area of the image to the peripheral area, the shading error estimation is suppressed by eliminating the change of the R ratio due to the subject color. can do.

(Embodiment 4)
The fourth embodiment will be described below with reference to the drawings.

<Outline of Embodiment 4>
In the image processing apparatus according to the third embodiment, the validity / invalidity of the block is determined, the validity / invalidity of the group is determined, the continuity of the effective group from the group in the center of the image toward the peripheral group is determined, and the invalid group is determined. Validate by interpolating, extract a group of subject areas of similar colors, and create a shading correction table suitable for correcting captured images according to the slope of the approximate straight line of the R ratio from the center of the effective group to the periphery of the image Selected.

  On the other hand, the image processing apparatus according to the fourth embodiment changes the effective group determination method to increase the number of effective groups, that is, the number of evaluation target groups, and the variance value for each continuous effective group. And by adding the variance value as an evaluation value of the shading estimation table, the occurrence of erroneous determination is suppressed and the accuracy of shading estimation is improved.

<Configuration of Embodiment 4>
The configuration of the digital still camera 100 provided with the image processing apparatus according to the fourth embodiment is also the same as that of FIG.

FIG. 26 is a block diagram showing a configuration of shading estimation section 124 according to the fourth embodiment. A group variance estimation unit 226 and a block variance estimation unit 227 are provided instead of the approximate estimation unit 224 and the variance estimation unit 225 of the shading estimation coefficient selection unit 214 according to Embodiment 3 shown in FIG.
Of course, it is only necessary to realize the operation of the fourth embodiment to be described later, and the present invention is not limited to this configuration.

The group variance estimation unit 226 obtains the variance value of the group evaluation value Ad for each continuous effective group (group continuous region), and uses the sum of the variance values as the evaluation value of the shading estimation coefficient, and has the smallest evaluation value. The shading estimation coefficient is determined as a shading estimation coefficient appropriate for the image.
The block variance estimation unit 227 performs an effective block variance estimation process instead of the above process of the group variance estimation unit 226 when the number of group continuous regions is smaller than a predetermined number.
That is, it can be said that the shading estimation unit 124 according to the fourth embodiment estimates shading by either group variance or block variance.

<Operation of Embodiment 4>
In the fourth embodiment, first, processing similar to that in steps S301 to S312 according to the third embodiment is performed, and the average value Ag [g] of the block evaluation values Eb is calculated from the distances D [0] to [[ The group evaluation value Ad [d] is calculated by averaging every d].

FIG. 27 is a flowchart for explaining the operation of shading estimation section 124 according to the fourth embodiment.
After performing the same processing as steps S301 to S306 according to the third embodiment, the effective group determination unit 223 determines the number of effective blocks Nb [g] for each of the distance groups G [0] to G [g] ( Step S401). At this time, if the effective block number Nb <the effective block number threshold, the effective block number Nb = 0.

  Next, the effective group determination unit 223 calculates the average value Ag [g] of the block evaluation values Eb for each of the groups G [0] to [g] using the above equation (21) (step S402). At this time, if the number of effective blocks Nb = 0, the average value Ag [g] = 0. That is, when the number of valid blocks in the group is small, the group is invalidated and excluded from the following estimation process.

Next, the effective group determination unit 223 averages the average value Ag [g] of the block evaluation values Eb for each of the distances D [0] to [d] from the center of the image using the above-described equation (22). An evaluation value Ad [d] is calculated (step S403).
Each process up to step S403 is performed for all shading estimation coefficients including a default shading estimation coefficient, that is, a shading estimation initial coefficient.

Next, the effective group determination unit 223 determines whether the applied shading image is a shading estimation initial coefficient or a shading estimation coefficient other than the shading estimation initial coefficient (step S404).
When the shading estimation initial coefficient is applied to the captured image (Yes in step S404), the effective group determination unit 223 calculates a difference value Diff of the group evaluation values Ad between adjacent distance groups (step S405). An effective distance group and a group continuous area are determined (step S406).

FIG. 28 is a diagram for explaining an effective distance group determination method according to the fourth embodiment. The horizontal direction in the figure shows each group when the image is divided into seven groups of distances D [0] to D [6]. The vertical direction in the figure indicates the group evaluation value Ad at each distance D [n].
In the fourth embodiment, when the difference value Diff of the group evaluation value Ad between the target group and the groups adjacent to the target group exceeds the adjacent group difference determination threshold value, the target group is determined as an invalid group.

  In the case of the scene 1 shown in FIG. 28A, the group evaluation values Ad of D [0] to D [2] and the group evaluation values Ad of D [3] to D [6] are substantially constant, Between these distances, Diff is almost zero. Also, Diff between D [2] and D [3] is larger than the adjacent group difference determination threshold. In this case, each group of D [0] to D [6] is determined as an effective distance group.

In the case of the scene 2 shown in FIG. 28B, the group evaluation values Ad of D [0] to D [2] and the group evaluation values Ad of D [4] to D [6] are substantially constant, Between these distances, Diff is almost zero. Also, Diff between D [2] and D [3] and between D [3] and D [4] are larger than the adjacent group difference determination threshold value. In this case, each group of D [0] to D [2] and D [4] to D [6] is determined as an effective distance group, and D [3] is determined as an invalid distance group.
Note that the attention group having only one adjacent group, such as D [0] or D [6], is effective / invalid by comparing the difference Diff of the group evaluation value Ad with the one adjacent group with a threshold value. Make a decision.

  FIG. 29 is a diagram for explaining specific processing contents of the effective distance group determination method according to the fourth embodiment. This is a processing content when images are divided into seven groups of distances D [0] to D [6]. From the top, distance groups D [0] to D [6] and their group evaluation values Ad values, Diff array values, and Ed array values are shown.

  First, a section where the difference Diff of the group evaluation value Ad with the adjacent group is equal to or greater than the threshold is stored in the Diff array as 0, and a section within the threshold is set as 1. At this time, (difference group number) +1 is prepared for the Diff array, and the beginning and end thereof are fixed to “0”. In this example, “0” is stored in four locations of the Diff array (* -D0), (D1-D2), (D2-D3), and (D7- *).

Next, the validity / invalidity of each group is determined. Specifically, the logical product of adjacent elements in the Diff array is obtained and stored in the Ed array. In this example, “0” is stored in one place of the Ed array (D2). That is, the distance group D [2] is determined as an invalid distance group, and the distance groups D [0], D [1], and D [3] to D [6] are determined as effective distance groups.
Then, a place where two or more “1” s continue in the Ed array, that is, a place where two or more effective distance groups continue is determined as a group continuous region (Series). In this example, D [0] to D [1] are determined as one group continuous area, and D [3] to D [6] as another group continuous area, that is, two group continuous areas are determined.

Next, the effective group determination unit 223 determines whether the number of group continuous areas is equal to or greater than a predetermined threshold (step S407). When the number of group continuous areas is equal to or greater than the predetermined threshold (Yes in step S407), the group Proceeding to the processing after step S408 by the variance estimating unit 226, and when the number of group continuous regions is less than the predetermined threshold (No in step S407), the processing proceeds to the effective block variance estimating process of step S412 by the block variance estimating unit 227. In this example, the number of group continuous regions Ns = 2, and the process proceeds to step S408 and subsequent steps.
The processing so far (steps S405 to S407) is performed using the characteristic values obtained by applying the shading estimation initial coefficient to the captured image.

  On the other hand, when a shading estimation coefficient other than the shading estimation initial coefficient is applied to the captured image (No in step S404), the process immediately proceeds to step S408 and subsequent steps by the group variance estimation unit 226.

Next, the group variance estimation unit 226 obtains a variance value SeriesVarp [n] in the group continuous region using the equations (26) and (27) (steps S408 and S409). Here, Ne is the number of groups in the group continuous area. Further, the group continuous region is a range of group continuous regions obtained in steps S405 and S406 using a captured image obtained by applying a shading estimation initial coefficient (in the above example, D [0] to D [1] and D [3] to D [6]) are commonly used when obtaining a variance value related to a shading estimation initial coefficient and when obtaining a variance value relating to a shading estimation coefficient other than the shading estimation initial coefficient. By obtaining the variance value SeriesVarp [n], the undulation of the group evaluation value Ad value for each group continuous region can be evaluated.

Next, the group variance estimation unit 226 calculates the table evaluation value Et [t] by integrating the variance values SeriesVarp [n] for each group continuous region using the equation (28) (step S410).
The processing so far (steps S408 to S410) is performed for all the shading estimation coefficients including the shading estimation initial coefficient.

Then, the group variance estimation unit 226 calculates a shading estimation coefficient that is most suitable for the captured image from among all the shading estimation coefficients, as shown in equation (29), that has the smallest table evaluation value Et [t]. Is selected (step S411).
The effective block variance estimation process (step S412) performed by the block variance estimation unit 227 is the same as the variance estimation process (step S318 or steps S331 to S335) performed by the variance estimation unit 225 according to the third embodiment. The description is omitted here.

  As described above, the image processing apparatus according to the fourth embodiment includes a group classification unit 221 that classifies an image into a plurality of groups based on the distance from the center of the image, and shading estimation on the image. A group evaluation value calculation unit 222 that calculates a group evaluation value for each group when the coefficient is applied, and an effective group determination unit that determines an effective group based on the group evaluation value and determines adjacent effective groups as a group continuous region 223 and a shading estimation coefficient selection unit 214 that selects a shading estimation coefficient having the smallest table evaluation value by accumulating the variance values of the group evaluation values for each group continuous region to obtain a table evaluation value of the shading estimation coefficient The image is compensated using the shading correction coefficient corresponding to the shading estimation coefficient. Those comprising a shading correction unit 108, and suppress the occurrence of erroneous determination, it is possible to improve the accuracy of the shading estimation.

  Note that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention. It is possible to appropriately combine the configurations according to the above embodiments.

  A part or all of each of the above embodiments can be described as in the following supplementary notes, but is not limited thereto.

(Appendix 1)
A block division unit that divides the captured image into a plurality of blocks;
A statistical value calculation unit that calculates block statistical values by statistically calculating pixel values for each block;
An image processing apparatus that estimates shading that occurs in the captured image by analyzing block statistic obtained by sequentially applying a shading correction table prepared for each light source in advance to the block statistic,
An image processing apparatus comprising: a shading estimation table selection unit that limits a table group used for estimation among at least one of image flatness, subject brightness, and subject color temperature among the table groups used for shading estimation. Or its image processing method.

(Appendix 2)
The image processing apparatus according to claim 1, wherein the image flatness is determined as a flat subject scene by integrating pixel value differences between blocks that are continuous from a block statistical value peripheral part to a central part. Or its image processing method.

(Appendix 3)
A block division unit that divides the captured image into a plurality of blocks;
A statistical value calculation unit for calculating a block statistical value in which pixel values for each RGB are statistically calculated for each block;
An image processing apparatus that estimates shading occurring in the captured image by analyzing block statistic obtained by sequentially applying a plurality of shading estimation tables having different correction strengths to the block statistic in advance,
The analysis process includes a block color evaluation value calculation unit that calculates a block color evaluation value from the block statistical value;
An image processing apparatus or an image processing method therefor, comprising: an evaluation block selecting unit that calculates a histogram of the block color evaluation value and selects an evaluation target block for shading estimation based on the histogram distribution.

(Appendix 4)
The block color evaluation value calculating means multiplies the block statistical value by a shading estimation coefficient having the weakest correction strength and a normalization gain to obtain a second block statistical value from the second block statistical value as a block color evaluation value. The image processing apparatus according to appendix 3, or an image processing method thereof.

(Appendix 5)
The normalization gain is obtained by integrating RGB of all blocks, wherein “integrated G value / integrated R value” is R gain, and “integrated G value / integrated B value” is B gain. Image processing apparatus or image processing method thereof.

(Appendix 6)
The block color evaluation value obtained by the block color evaluation value calculation means is obtained based on at least one of the R ratio, G ratio, and B ratio of the second block statistical value. Image processing apparatus or image processing method thereof.

(Appendix 7)
The image processing apparatus or the image processing method according to appendix 6, wherein the R ratio is calculated by “3R / (R + G + B)” or “R / (R + G + B)”.

(Appendix 8)
8. The image processing apparatus according to any one of appendices 3 to 7, wherein the evaluation block selecting means sets a block classified into bins in the vicinity of a histogram mode value as an evaluation target block, or image processing thereof Method.

(Appendix 9)
Any one of Supplementary notes 3 to 8, wherein the evaluation block selecting unit excludes the block from the evaluation target when any of the R value, G value, and B value of the block is outside a preset range. The image processing apparatus or image processing method according to one of the above.

(Appendix 10)
A block division unit that divides the captured image into a plurality of blocks;
A statistical value calculation unit for calculating a block statistical value in which pixel values for each RGB are statistically calculated for each block;
A block classification value for obtaining a block statistical value obtained by sequentially applying a plurality of shading estimation tables having different correction strengths to the block statistical value in advance, and grouping the blocks according to the distance from the center of the image;
A valid block judging means for judging whether the block statistical value is valid or invalid;
A group statistical value calculation means for obtaining a sum by RGB of the statistical values by RGB of the effective blocks in the group as a group statistical value;
An image processing apparatus comprising:
Effective group determination means for determining the validity / invalidity of the group according to the number of effective blocks in each group;
A group R ratio linear approximation means for linearly approximating the R ratio calculated by “3R / (R + G + B)” or “R / (R + G + B)” from the central part of the image to the peripheral part;
An image processing apparatus or an image processing method therefor, comprising: a shading correction table selecting means for selecting a shading correction table suitable for correcting a captured image in accordance with the inclination of the approximate straight line calculated by the group R ratio straight line approximating means. .

(Appendix 11)
The group statistical value calculation means uses, as a group statistical value, a sum of blocks determined to be valid by the block determination means among block statistical values to which the shading estimation table included in the group is sequentially applied. The image processing apparatus or the image processing method thereof.

(Appendix 12)
The effective group determination unit is a group continuity determination unit that determines the continuity of the effective group from the group in the center of the image toward the peripheral group.
The image processing apparatus or the image processing method according to claim 10 or 11, further comprising invalid group validation means for interpolating and validating invalid groups according to the group continuity determination result.

(Appendix 13)
The invalid group validating means validates an invalid group when there are two or more consecutive valid groups on both sides of a single invalid group and the difference between two valid groups adjacent to the invalid group is equal to or less than a threshold value. The image processing apparatus or the image processing method according to appendix 12, wherein

(Appendix 14)
The image according to any one of appendices 10 to 13, wherein the shading correction table selection means selects an approximate straight line that is close to horizontal or that that is closest to a preset reference inclination. A processing apparatus or an image processing method thereof.

(Appendix 15)
In an image processing apparatus that divides a captured image into a plurality of blocks, includes a statistical value calculation unit that statistically calculates a pixel value for each block, and estimates shading that occurs in the captured image from the block statistical values,
Apply a shading estimation table corresponding to each of a plurality of light sources to a captured image sequentially,
Classify blocks into groups according to the distance from the center of the image, determine whether the group is used for shading estimation,
A continuous group in which the difference in R ratio with an adjacent group is smaller than a threshold is defined as a group continuous region,
The table evaluation value is obtained by calculating the variance value for each group continuous area and adding the variance values of all the group continuous areas.
An image processing apparatus or an image processing method thereof, wherein a shading estimation table having the smallest table evaluation value is selected as a shading estimation table capable of appropriately correcting a captured image.

(Appendix 16)
A block division unit that divides the captured image into a plurality of blocks;
A statistical value calculation unit that calculates block statistical values by statistically calculating pixel values for each block;
A block statistic obtained by sequentially applying a shading correction table prepared in advance for each of a plurality of light sources to the block statistic, and grouping means for classifying the blocks into groups according to the distance from the center of the image;
Group statistical value calculating means for obtaining a group statistical value from block statistical values obtained by sequentially applying the shading correction table;
An image processing apparatus comprising group evaluation value calculation means for calculating a group evaluation value from the group statistical value,
Effective group determination means for determining the validity / invalidity of the group according to the number of effective blocks in each group
Group continuity determination means for determining the continuity of the effective group from the group in the center of the image to the surrounding group;
A table evaluation value calculating means for obtaining a dispersion value for each group continuous area determined as a continuous group based on the determination result of the group continuity and adding a dispersion value of all the group continuous areas;
A shading correction table selecting means for selecting a shading correction table having the smallest table evaluation value;
An image processing apparatus or an image processing method thereof capable of performing shading correction suitable for correction of a captured image.

DESCRIPTION OF SYMBOLS 100 Digital still camera 102 Lens optical system 104 Image pick-up element 106 AFE circuit 108 Image signal processing circuit 110 Image display part 112 Image recording part 114 Driver 116 Timing generator 118 Image block statistical circuit 120 Control part 122 Coefficient memory | storage part 124 Shading estimation part 126 Shading Correction unit 201 Flat decision unit 202 Shading estimation coefficient selection unit 211 Block color evaluation value calculation unit 212 Block weight coefficient calculation unit 213 Block evaluation value calculation unit 214 Shading estimation coefficient selection unit 221 Group classification unit 222 Group evaluation value calculation unit 223 Effective group Determination unit 224 Approximate estimation unit 225 Variance estimation unit 226 Group variance estimation unit 227 Block variance estimation unit

Claims (28)

A block division unit that divides the captured image into a plurality of blocks;
A statistical value calculation unit that calculates a block statistical value by taking statistics of a pixel value for each block;
An estimation coefficient group storage unit that stores a first shading estimation coefficient group including a plurality of shading estimation coefficients respectively corresponding to a plurality of light sources;
A shading estimation unit that estimates shading based on the block statistical value calculated by the statistical value calculation unit and the first shading estimation coefficient group stored by the estimation coefficient group storage unit;
With
The shading estimation unit may select at least one suitable for the captured image from the stored first shading estimation coefficient group according to at least one of image flatness, subject brightness, and subject color temperature of the captured image. One of the select second shading estimation coefficient group including a shading estimation coefficients,
An image processing apparatus further comprising a shading correction unit that applies a shading correction coefficient corresponding to the selected second shading estimation coefficient group to the captured image and corrects shading of the captured image. The shading estimation unit obtains the image flatness by accumulating pixel value differences of the blocks continuous from the peripheral part to the central part of the captured image.
The image processing apparatus according to claim 1. The shading estimation unit selects the second shading estimation coefficient group from the first shading estimation coefficient group when it is determined that the captured image is not flat based on the image flatness.
The image processing apparatus according to claim 1. The shading estimation unit
When it is determined that the captured image is flat based on the image flatness, the second shading estimation coefficient group is selected from the first shading estimation coefficient group based on the subject color temperature,
When it is determined that the captured image is not flat based on the image flatness, the second shading estimation coefficient group is selected from the first shading estimation coefficient group based on the subject brightness.
The image processing apparatus according to claim 1. The shading estimation unit determines the second shading from the first shading estimation coefficient group based on the subject color temperature when it is determined that the captured image is an indoor image based on the subject brightness. Select an estimation coefficient group,
The image processing apparatus according to claim 1. A block division unit that divides the captured image into a plurality of blocks;
A statistical value calculation unit that calculates the first block statistical value by taking the statistics of the pixel value for each color for each block;
An estimation coefficient storage unit that stores a plurality of shading estimation coefficients having different correction strengths;
A block color evaluation value calculation unit for calculating a block color evaluation value for evaluating the calculated first block statistical value;
A block evaluation value calculation unit that selects an evaluation target block for shading estimation based on the calculated block color evaluation value histogram;
A shading estimation coefficient selection unit that selects an appropriate shading estimation coefficient for the captured image from the stored shading estimation coefficients based on the selected evaluation target block;
Applying a shading correction coefficient corresponding to the selected shading estimation coefficient to the captured image, and correcting a shading of the captured image;
An image processing apparatus comprising: The block color evaluation value calculation unit multiplies the first block statistic value by a coefficient value of a shading estimation coefficient having the weakest correction strength and a normalization gain to obtain a second block statistic value. Calculating the block color evaluation value based on a block statistical value;
The image processing apparatus according to claim 6. The block color evaluation value calculation unit integrates RGB pixel values for all blocks, and sets a value obtained by dividing the integrated G pixel value by the integrated R pixel value as the normalized gain of R. A value obtained by dividing the obtained G pixel value by the accumulated B pixel value is used as the normalized gain of B.
The image processing apparatus according to claim 7. The block color evaluation value calculating unit obtains the block color evaluation value based on at least one of an R ratio, a G ratio, and a B ratio of the second block statistical value;
The image processing apparatus according to claim 7 or 8. The ratio of R is 3 times the ratio of R to all RGB or the ratio of R to all RGB.
The image processing apparatus according to claim 9. The block evaluation value calculation unit sets a block classified into bins in the vicinity of the mode value of the histogram as the evaluation target block.
The image processing apparatus according to claim 6. The block evaluation value calculation unit excludes a block whose R pixel value, G pixel value, or B pixel value is outside a preset range from the evaluation target block,
The image processing apparatus according to claim 6. A block division unit that divides the captured image into a plurality of blocks;
A statistical value calculation unit that calculates a block statistical value by taking statistics of pixel values for each color for each block;
An estimation coefficient storage unit that stores a plurality of shading estimation coefficients having different correction strengths;
A group classification unit that classifies the blocks into a plurality of groups according to the distance from the center of the captured image;
An effective block determination unit that determines whether the block is valid or invalid based on the block statistical value;
A group evaluation value calculation unit that calculates a group evaluation value based on the sum of each statistical value for each block in the group determined to be valid;
An effective group determination unit that determines the validity or invalidity of the group based on the calculated group evaluation value;
A linear approximation unit that linearly approximates a color ratio for the group determined to be effective among the groups from the center to the periphery of the captured image;
A shading estimation coefficient selection unit that selects an appropriate shading estimation coefficient for the captured image from the stored shading estimation coefficients according to the slope of the approximated straight line;
An image processing apparatus comprising: a shading correction unit that applies a shading correction coefficient corresponding to the selected shading estimation coefficient to the captured image and corrects shading of the captured image. The color ratio is 3 times the ratio of R to all RGB or the ratio of R to all RGB.
The image processing apparatus according to claim 13. The group evaluation value calculation unit calculates the group evaluation value based on a sum of blocks determined to be valid among the block statistical values to which the shading estimation coefficient included in the group is applied.
The image processing apparatus according to claim 13 or 14. The effective group determination unit is a group continuity determination unit that determines the continuity of the group determined to be effective among the groups from the central part to the peripheral part of the captured image,
In accordance with the determination result of the continuity of the group, an invalid group enabling unit that validates the group determined to be invalid,
The image processing apparatus according to claim 13, further comprising: The invalid group enabling unit includes two or more consecutive groups determined to be valid on both sides of the single group determined to be invalid, and 2 adjacent to the group determined to be invalid. Enabling the invalid group when the difference between the two groups determined to be valid is equal to or less than a threshold value,
The image processing apparatus according to claim 16. The shading estimation coefficient selection unit selects a shading estimation coefficient such that the approximate straight line is close to horizontal or closest to a preset reference inclination,
The image processing apparatus according to claim 13. A group classification unit that classifies the images into a plurality of groups based on the distance from the center of the image;
A group evaluation value calculation unit for calculating a group evaluation value for each group when a shading estimation coefficient is applied to the image;
An effective group determination unit that determines an effective group based on the group evaluation value and determines the adjacent effective group as a group continuous region;
A shading estimation coefficient selection unit that accumulates variance values of the group evaluation values for each group continuous region to obtain a table evaluation value of the shading estimation coefficient, and selects a shading estimation coefficient having the smallest table evaluation value;
An image processing apparatus comprising: a shading correction unit that corrects the image using a shading correction coefficient corresponding to the selected shading estimation coefficient. The effective group determination unit
The image processing device according to claim 19, wherein the attention group in which a difference in the group evaluation value between the attention group and a group adjacent to the attention group exceeds a predetermined threshold is determined as an invalid group. The effective group determination unit determines the effective group based on the group evaluation value when applying a shading estimation initial coefficient that corrects the image weakest to the image,
The shading estimation coefficient selection unit uses the range of the group continuous area when the shading estimation initial coefficient is applied to the image, and also when a shading estimation coefficient other than the shading estimation initial coefficient is applied to the image. Item 19. The image processing apparatus according to Item 19 or 20. A statistical value calculation unit that divides the image into a plurality of blocks and calculates a block statistical value by statistically calculating a pixel value for each block;
A block evaluation value calculating unit that calculates a block evaluation value for each block based on the block statistical value and the shading estimation coefficient;
The group classification unit classifies the blocks into the plurality of groups based on a distance from the center of the image.
The image processing device according to any one of claims 19 to 21, wherein the group evaluation value calculation unit calculates the group evaluation value based on the block evaluation value. The block evaluation value calculation unit further calculates the number of effective blocks,
The image processing device according to claim 22, wherein the effective group determination unit determines the effective group based on the number of the effective blocks. The shading estimation coefficient selection unit calculates a variance value of the block statistics value of the effective block when the number of the group continuous regions is less than a predetermined threshold, and the variance value of the block statistics value is the smallest The image processing apparatus according to claim 23, wherein a shading estimation coefficient is selected. Divide the captured image into multiple blocks,
Calculate block statistics by taking pixel value statistics for each block,
Storing a first shading estimation coefficient group including a plurality of shading estimation coefficients respectively corresponding to a plurality of light sources;
Estimating shading based on the block statistics and the first shading estimation coefficient group;
In the process of estimating the shading, an appropriate value for the captured image is selected from the stored first shading estimation coefficient group according to at least one of image flatness, subject brightness, and subject color temperature of the captured image. Selecting a second group of shading estimation coefficients including at least one shading estimation coefficient;
Applying a shading correction coefficient corresponding to the selected second shading estimation coefficient group to the captured image to correct shading of the captured image;
Image processing method. Divide the captured image into multiple blocks,
The first block statistic value is calculated by taking the statistics of the pixel value for each color for each block,
Stores multiple shading estimation coefficients with different correction strengths,
Calculating a block color evaluation value for evaluating the calculated first block statistical value;
Selecting an evaluation target block for shading estimation based on the calculated block color evaluation value histogram;
Based on the selected evaluation target block, a suitable shading estimation coefficient for the captured image is selected from the stored shading estimation coefficients,
Applying a shading correction coefficient corresponding to the selected shading estimation coefficient to the captured image to correct shading of the captured image;
Image processing method. Divide the captured image into multiple blocks,
By calculating the statistics of the pixel value for each color for each block, the block statistical value is calculated,
Stores multiple shading estimation coefficients with different correction strengths,
Classifying the blocks into a plurality of groups according to the distance from the center of the captured image;
Determining whether the block is valid or invalid based on the block statistics;
A group evaluation value is calculated based on the sum of each statistical value for each of the blocks determined to be valid in the group,
Determining whether the group is valid or invalid based on the calculated group evaluation value;
For the group determined to be effective among the groups from the center to the periphery of the captured image, the color ratio is linearly approximated,
According to the slope of the approximate line, select an appropriate shading estimation coefficient for the captured image from the stored shading estimation coefficients,
Applying a shading correction coefficient corresponding to the selected shading estimation coefficient to the captured image to correct shading of the captured image;
Image processing method. Classifying the images into a plurality of groups based on the distance from the center of the image;
Calculating a group evaluation value for each group when a shading estimation coefficient is applied to the image;
An effective group is determined based on the group evaluation value, the adjacent effective group is determined as a group continuous region,
The table evaluation value of the shading estimation coefficient is accumulated by integrating the variance value of the group evaluation value for each group continuous region, and the shading estimation coefficient having the smallest table evaluation value is selected.
An image processing method for correcting the image using a shading correction coefficient corresponding to the selected shading estimation coefficient.