JP6346431B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method.

  When an infrared cut filter is applied to the imaging apparatus, color shading may occur in the captured image. In Patent Document 1, as an image processing apparatus that corrects such color shading, a plurality of shading correction coefficients are applied to a captured image, and a shading correction coefficient that can appropriately correct color shading is selected. An image processing apparatus that corrects and outputs color shading of the captured image by applying the selected shading correction coefficient is disclosed.

JP 2013-198041 A

In the above image processing apparatus, the captured image is divided into a plurality of blocks, the statistical value of the pixel value for each block is obtained, the statistical value is multiplied by the coefficient value for each block of the shading correction coefficient, and the multiplication result is captured. Find the variance value in the image. The dispersion value is obtained for each shading correction coefficient, and the shading correction coefficient with the smallest dispersion value is selected as the shading correction coefficient that has been able to appropriately correct the color shading.
However, when the imaging scene is complicated, there is a problem that the shading correction coefficient with the smallest variance value is not necessarily a shading correction coefficient that can appropriately correct the color shading.

  The present invention has been made to solve such a problem, and an object thereof is to provide an image processing apparatus and an image processing method capable of selecting a shading correction coefficient suitable for an image.

  An image processing apparatus according to the present invention applies a plurality of shading estimation coefficients to an image, selects an appropriate shading estimation coefficient for the image, and a shading correction coefficient corresponding to the appropriate shading estimation coefficient to the image. And a correction unit that corrects shading of the image, and the estimation unit classifies the image into a plurality of groups based on a distance from a central part of the image, and the shading estimation coefficient is applied to the image. An image characteristic value when applied is calculated for each of the groups, a statistical value of the image characteristic value is calculated, and an appropriate shading estimation coefficient for the image is selected. With this configuration, a shading estimation coefficient suitable for an image can be selected, and a shading correction coefficient suitable for an image can be selected.

  An image processing method according to the present invention includes an estimation step of applying a plurality of shading estimation coefficients to an image, selecting an appropriate shading estimation coefficient for the image, and a shading correction coefficient corresponding to the appropriate shading estimation coefficient to the image. Applying and correcting the shading of the image, the estimating step classifying the image into a plurality of groups based on a distance from a central portion of the image, and the shading into the image Calculating an image characteristic value when the estimation coefficient is applied for each group, and calculating a statistical value of the image characteristic value and selecting an appropriate shading estimation coefficient for the image. . Accordingly, it is possible to select a shading estimation coefficient suitable for an image and select a shading correction coefficient suitable for the image.

  According to the present invention, it is possible to provide an image processing apparatus and an image processing method capable of selecting a shading correction coefficient suitable for an image.

1 is a block diagram illustrating a configuration of a digital still camera 100 including an image processing device according to an embodiment. It is a flowchart for demonstrating operation | movement of the shading estimation part 124 which concerns on embodiment. It is a figure for demonstrating the group which concerns on embodiment. It is a figure for demonstrating another group which concerns on embodiment.

Embodiments of the present invention will be described below with reference to the drawings.
First, the configuration of the image processing apparatus according to the present embodiment will be described including the configuration of the digital still camera 100 provided with the image processing apparatus.

FIG. 1 is a block diagram illustrating a configuration of a digital still camera 100 including an image processing apparatus according to the present embodiment.
A digital still camera 100 that is an electronic imaging apparatus includes a lens optical system 102, an imaging element 104, an AFE circuit 106, an image signal processing circuit 108, an image display unit 110, an image recording unit 112, a driver 114, and a timing generator (TG) 116. , An image block statistical circuit 118, a control unit 120, a coefficient storage unit 122, and the like.

  The lens optical system 102 includes a lens, a diaphragm, a shutter, and the like, and forms a subject image on the imaging surface of the imaging element 104. The image sensor 104 is an image sensor such as a CCD or a CMOS, and an infrared cut filter (not shown) is mounted on the lens optical system 102 side of the image sensor 104. The image sensor 104 photoelectrically converts the subject image to obtain an image signal (RGB color signal). The AFE circuit 106 A / D converts an image signal acquired by the image sensor 104 and subjected to signal processing by a CDS circuit (not shown) into a digital signal.

  The image signal processing circuit 108 performs demosaicing processing, edge enhancement processing, white balance (WB) correction processing, shading correction processing, gamma correction processing, and the like on the image signal output from the AFE circuit 106. The image display unit 110 is a liquid crystal display (LCD) or the like, and displays an image signal that has been subjected to various processes by the image signal processing circuit 108. The image recording unit 112 is a memory, and records the image signal that has been subjected to various processes by the image signal processing circuit 108.

The driver 114 drives the lens, aperture, and shutter of the lens optical system 102. The timing generator 116 generates timing for driving the image sensor 104.
The image block statistical circuit 118 divides the captured image area or a part of the captured image area of the image signal converted into a digital signal by the AFE circuit 106 into a plurality of blocks, and calculates a block statistical value for each block. calculate. The image block statistical circuit 118 calculates a pixel integrated value for each RGB or a pixel average value for each RGB in each block as a block statistical value.

  The control unit 120 controls white balance correction processing of the image signal processing circuit 108 based on the block statistical value calculated by the image block statistical circuit 118. In addition, the control unit 120 includes a shading estimation unit 124 and a shading correction unit 126.

  The shading estimation unit 124 estimates shading based on the block statistical value calculated by the image block statistical circuit 118 and the shading estimation coefficient stored in the coefficient storage unit 122, and selects a shading estimation coefficient suitable for the image. The shading estimation coefficient is set for each light source, for each R value, G value, and B value, and has a coefficient value for each block. The shading estimation process will be described later.

  The shading correction unit 126 selects a shading correction coefficient corresponding to the shading estimation coefficient selected by the shading estimation unit 124 from the coefficient storage unit 122, and corrects shading of the captured image in the image signal processing circuit 108. The shading correction coefficient is set for each light source, for each R value, G value, and B value, and has a coefficient value for each pixel or each of a plurality of pixels.

  The coefficient storage unit 122 stores a combination of a shading estimation coefficient and a shading correction coefficient for each light source such as sunlight, a light bulb, and a fluorescent lamp. In many cases, the shading estimation coefficient and the shading estimation coefficient are in a one-to-one pair, but the present invention is not limited to this. The shading estimation coefficient and the shading correction coefficient can be calculated by an external device such as a personal computer.

The shading estimation coefficient is calculated using a white chart image taken under each light source. For the white chart, it is desirable to use a uniform diffuse reflection surface having a spectral reflectance such as a standard white reflector that is constant over the entire target wavelength range and 90% or more. For a light source such as a light bulb that contains a large amount of infrared light, a shading estimation coefficient that strongly corrects the R signal at the periphery of the image is calculated. For a light source such as a fluorescent lamp that does not contain much infrared light, a shading estimation coefficient for correcting the R signal at the periphery of the image weakly is calculated. The shading estimation coefficient that corrects the R signal at the periphery of the image most weakly is set as an initial setting (default) shading estimation coefficient, and is particularly referred to as a shading estimation initial coefficient.
Note that the image processing apparatus according to this embodiment includes the image signal processing circuit 108, the image block statistical circuit 118, the control unit 120, the coefficient storage unit 122, and the like, but is not limited to this configuration.

  Moreover, each component which the control part 120 implement | achieves is realizable by making a program run by control of the arithmetic unit (not shown) with which the control part 120 which is a computer is provided, for example. More specifically, the control unit 120 is realized by loading a program stored in a storage unit (not shown) into a main storage device (not shown) and executing the program under the control of the arithmetic unit. Each component is not limited to being realized by software by a program, and may be realized by any combination of hardware, firmware, and software.

  The above-described program can be stored using various types of non-transitory computer readable media and supplied to a computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROMs (Read Only Memory), CD-ROMs. R, CD-R / W, semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)).

  Further, the program may be supplied to the computer by various types of temporary computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

Next, the operation of the shading estimation unit 124 of the present embodiment will be described.
FIG. 2 is a flowchart for explaining the operation of shading estimation section 124 according to the present embodiment.
First, the shading estimation unit 124 uses the block statistical values calculated from the image subjected to the shading correction by the image block statistical circuit 118, and average values Rav, Gav, Bav is calculated (step S010).

Next, normalization coefficients Rg, Bg, and Gg are calculated as shown in equation (1) (step S020). The normalization coefficients Rg, Bg, and Gg are coefficients for matching the RGB balance of the entire screen, and are gains that normalize the image.

Vav = max (Rav, Gav, Bav)
Rg = Vav / Rav
Gg = Vav / Gav
Bg = Vav / Bav (1)

Next, as shown in equation (2), the shading estimation coefficient Cr [t] (x) is calculated for all the block statistical values R (x, y), G (x, y), and B (x, y). , y), Cg [t] (x, y), Cg [t] (x, y) and normalization coefficients Rg, Bg, Gg are applied and multiplied, and Rc [t] (x, y) is applied to each block. ), Gc [t] (x, y), Bc [t] (x, y) are calculated (step S030).

Rc _{[t] (x, y)} = R _{(x, y)} · Cr _{[t] (x, y)} · Rg
Gc _{[t] (x, y)} = G _{(x, y)} · Cg _{[t] (x, y)} · Gg
Bc _{[t] (x, y)} = B _{(x, y)} · Cb _{[t] (x, y)} · Bg (2)

Here, the aforementioned shading estimation initial coefficient is used as the shading estimation coefficient in order to calculate the R signal ratio for each group for selecting a reference group to be described later.
In addition, the normalization coefficient is applied to determine the degree of color difference of the block from the RGB average value by regarding the RGB average value of the captured image as an achromatic color.
Rc [t] (x, y), Gc [t] (x, y), Bc [t] (x, y) are equally close to 1.0 as the shading estimation coefficient is more suitable for an image. When [t] (x, y), Gc [t] (x, y), and Bc [t] (x, y) are imaged, the image becomes a single gray color.

Next, as shown in equation (3), an R signal ratio Rr [t] (x, y) is obtained for each block (steps S040 and S050).

Rr _{[t] (x, y)} = Rc _{[t] (x, y)} / Gc _{[t] (x, y)} (3)

Note that the denominator of equation (3) may be the sum of Rc [t] (x, y), Gc [t] (x, y), and Bc [t] (x, y). Further, in addition to the R signal ratio for each block or instead of the R signal ratio for each block, the B signal ratio for each block may be calculated and used for the subsequent processing.

  In the equation (3), since the R signal ratio is obtained for the RGB image to which the normalization coefficient is applied, the chromatic object, for example, the entire image becomes red as the shading estimation coefficient is suitable for the image. Even if the subject is a small subject, the R signal ratio is close to 1.0. Conversely, the R signal ratio becomes a value farther from 1.0 when the shading estimation coefficient is not suitable for an image.

Next, the shading estimation unit 124 calculates the R signal ratio for each group (steps S060 and S070).
FIG. 3 is a diagram for explaining an example of a group according to the present embodiment.
The shading estimation unit 124 classifies each block acquired by the image block statistical circuit 118 by dividing the image region into a grid pattern into a plurality of groups based on the distance from the center of the image. In the image 300 shown in FIG. 3, each block of the image is classified into six groups from group 0 to group 5 from the center of the image toward the periphery of the image.
Then, as shown in equation (4), the R signal ratio Rr (i) of the blocks included in the group is averaged to calculate the R signal ratio GpRr [j] for each group.

Next, a reference group is selected (step S080). A search is performed sequentially from the group in the center of the image toward the outer group, and one group is selected as the reference group. For example, the search is started from the group 0 shown in FIG. 3, and the search is performed up to the group 3 that is considered not affected by the shading. At this time, as shown in the equation (5), threshold determination of the R signal ratio GpRr [j] of the group is performed, and a group satisfying the equation (5) is first selected as a reference group. The threshold values Th1 and Th2 are set to, for example, 0.9 and 1.1 so that 1.0 is included between them. Since the reference group is selected after applying the normalization coefficient, the reference group can be selected not only for an achromatic subject but also for a uniform chromatic subject.

Th1 <GpRr _[j] <Th2 (5)

Next, the presence / absence of a reference group is determined (step S090).
When there is a reference group (Yes in step S090), a group evaluation value is calculated (step S100, step S110).
The group evaluation value is calculated for each group outside the reference group selected in step S080. For example, when group 1 is selected as the reference group, a group evaluation value is calculated for each of the outer groups 2 to 5. As shown in the equation (6), the difference between the R signal ratio GpRr [j] of the reference group and the R signal ratio GpRr [j + k] of the outer group is obtained, and the weight Wg [j + k] of the outer group prepared in advance is obtained. Each group evaluation value Dg [j + k] is calculated by multiplication.

Dg _{[j + k]} = (GpRr _[j] −GpRr _{[j + k]} ) · Wg _{[j + k]} (6)

The group weight Wg is prepared for each group as follows. The group at the periphery of the image has a larger weight Wg. As a result, it is possible to more accurately evaluate the suitability of the shading correction of the shading estimation coefficient at the periphery of the image.

Group weight Wg
1 0.2
2 0.4
3 0.6
4 0.8
5 1.0

  Next, as shown in the equation (7), the sum of absolute values of the group evaluation values Dg [j + k] is taken to obtain the evaluation value Ed of the shading estimation coefficient applied in step S030 (step S120).

  This evaluation value Ed is used to extract the characteristic of the R signal ratio that attenuates from the center to the periphery of the image. Whether or not the shading estimation coefficient used in step S030 is appropriate for the image subjected to shading correction. It becomes an index to judge.

  Next, the shading estimation unit 124 determines whether the evaluation values of all the shading estimation coefficients have been calculated for the image to be subjected to shading correction (step S130). When evaluation values of all the shading estimation coefficients have not been calculated (No in step S130), the process returns to step S030, and shading estimation coefficients for which evaluation values have not been calculated, that is, shading estimation coefficients other than the shading estimation initial coefficient are calculated. The processes from step S030 to step S130 are repeated.

  When calculating an evaluation value of a shading estimation coefficient other than the shading estimation initial coefficient, the selection of the reference group in step S080 and the determination of the presence or absence of the reference group in step S090 are not performed. Then, the reference group selected when calculating the evaluation value of the shading estimation initial coefficient, for example, group 1 is selected as it is as the reference group when calculating the evaluation value of the shading estimation coefficient other than the shading estimation initial coefficient. That is, the evaluation value of the shading estimation coefficient other than the shading estimation initial coefficient is calculated based on the reference group selected when calculating the evaluation value of the shading estimation initial coefficient.

  Finally, after calculating the evaluation values of all the shading estimation coefficients (Yes in step S130), the shading estimation unit 124 has the smallest evaluation value Ed among all the shading estimation coefficients including the shading estimation initial coefficient. Is selected as a shading estimation coefficient suitable for an image to be subjected to shading correction (step S140).

  Further, when there is no reference group (No in step S090), the shading estimation unit 124 selects a shading estimation initial coefficient as a shading estimation coefficient suitable for an image to be subjected to shading correction (step S140).

  In this embodiment, as shown in FIG. 3, the entire process target for shading correction is performed from the reference group selection process to the evaluation value calculation process (steps S080 to S120). An image to be subjected to shading correction may be divided into a plurality of areas, and the process from the reference group selection process to the evaluation value calculation process may be performed in each area.

FIG. 4 is a diagram for explaining another group according to the present embodiment.
The shading estimation unit 124 divides the image 400 to be subjected to shading correction into four regions I to IV, and further classifies each block in the regions I to IV into a plurality of groups based on the distance from the image center. . For example, in the image 400 shown in FIG. 4, in the region I, each block is classified into five groups from group I-1 to group I-5. In region I, the process from the reference group selection process to the evaluation value calculation process is performed. The same applies to the regions II to IV. Thereby, the evaluation value of each shading estimation coefficient can be calculated for each of the regions I to IV, and the shading estimation coefficient suitable for the subject can be selected for each of the regions I to IV.

In the present embodiment, the image processing apparatus is incorporated in the digital still camera 100. However, the image processing apparatus is incorporated in an external apparatus such as a personal computer, and the apparatus performs shading estimation and shading correction on the apparatus. Anyway.
In the present embodiment, the estimated coefficient selected as described in Patent Document 1 may be selected and used as a correction coefficient as it is. That is, the shading correction coefficient according to the present embodiment may be different from the shading estimation coefficient or may be the same as the shading estimation coefficient.

In this embodiment, the evaluation value of the shading estimation coefficient is obtained by the sum of the absolute values of the group evaluation values. However, the evaluation value of the shading estimation coefficient may be obtained by variance of the group evaluation values. That is, the evaluation value of the shading estimation coefficient according to the present embodiment may be obtained from some statistical value of the group evaluation value.
Further, in the present embodiment, each group evaluation value is obtained by calculating the difference between the R signal ratio of the reference group and the R signal ratio of the outer group and multiplying this by the weight of the outer group. Each group evaluation value may be obtained as a difference between the R signal ratio of the reference group and the R signal ratio of the outer group. That is, multiplying the weight of the outer group may be omitted.

  As described above, the image processing apparatus according to this embodiment applies a plurality of shading estimation coefficients to an image, and corresponds to an estimation unit that selects an appropriate shading estimation coefficient for an image and an appropriate shading estimation coefficient. A correction unit that applies the shading correction coefficient to the image and corrects the shading of the image, and the estimation unit classifies the images into a plurality of groups based on the distance from the center of the image, and adds the shading estimation coefficient to the image. An image characteristic value when applied is calculated for each group, a statistical value of the image characteristic value is calculated, and an appropriate shading estimation coefficient for the image is selected. With this configuration, it is possible to select a shading correction coefficient suitable for an image.

  Further, in the image processing apparatus according to the present embodiment, the estimation unit determines the threshold value of the image characteristic value in order from the group at the center of the image and selects the reference group from the group, and the image characteristic value of the reference group, The absolute value of the difference from the image characteristic value of the group outside the reference group is calculated for each group outside, and the sum of the absolute values of the differences is calculated as the statistical value of the image characteristic value.

  Also, in the image processing apparatus according to the present embodiment, the estimation unit has a group weight that increases in accordance with the distance from the center of the image for each group, and the image characteristic value in order from the group at the center of the image. The threshold value is determined, a reference group is selected from the group, and the absolute value of the difference between the image characteristic value of the reference group and the image characteristic value of the group outside the reference group is multiplied by the group weight of the group outside. The multiplication result is calculated for each group outside, and the sum of the multiplication results is calculated as a statistical value of the image characteristic value.

In the image processing apparatus according to the present embodiment, the image characteristic value is at least one of the R signal ratio and the B signal ratio.
The image processing apparatus according to the present embodiment further includes a statistical unit that divides the image into a plurality of blocks and calculates a statistical value of the pixel value for each block, and the estimation unit removes the blocks from the central portion of the image. The image characteristic values are calculated based on the statistical values of the pixel values of the blocks belonging to the group.

In the image processing apparatus according to the present embodiment, the estimation unit applies a coefficient for matching the RGB balance of the entire screen when calculating the image characteristic value.
The image processing apparatus according to the present embodiment further includes a storage unit that stores a plurality of sets of shading estimation coefficients and shading correction coefficients, and the correction unit includes a shading correction coefficient paired with an appropriate shading estimation coefficient. Are selected as the corresponding shading correction coefficients.
In the image processing apparatus according to the present embodiment, a set is provided for each light source.

108 image signal processing circuit 118 image block statistical circuit 120 control unit 122 coefficient storage unit 124 shading estimation unit 126 shading correction unit

Claims (9)

Applying a plurality of shading estimation coefficients to the image, and selecting an appropriate shading estimation coefficient for the image;
A correction unit that applies a shading correction coefficient corresponding to the appropriate shading estimation coefficient to the image and corrects shading of the image, and an image processing apparatus comprising:
The estimation unit includes
Classifying the images into a plurality of groups based on the distance from the center of the image;
An image characteristic value when the shading estimation coefficient is applied to the image is calculated for each group,
An image processing apparatus that calculates a statistical value of the image characteristic value and selects an appropriate shading estimation coefficient for the image. The estimation unit includes
The threshold value of the image characteristic value is determined in order from the group at the center of the image, and a reference group is selected from the group,
Calculating the absolute value of the difference between the image characteristic value of the reference group and the image characteristic value of the group outside the reference group for each of the groups outside;
The image processing apparatus according to claim 1, wherein a sum of absolute values of the differences is calculated as a statistical value of the image characteristic value. The estimation unit has, for each group, a group weight that increases according to the distance from the center of the image,
The threshold value of the image characteristic value is determined in order from the group at the center of the image, and a reference group is selected from the group,
The multiplication result obtained by multiplying the absolute value of the difference between the image characteristic value of the reference group and the image characteristic value of the group outside the reference group by the group weight of the group outside is obtained for each group outside the group. To
The image processing apparatus according to claim 1, wherein the sum of the multiplication results is calculated as a statistical value of the image characteristic value. The image processing apparatus according to any one of claims 1 to 3, wherein the image characteristic value is at least one of an R signal ratio and a B signal ratio. Further comprising a statistical unit that divides the image into a plurality of blocks and calculates a statistical value of a pixel value for each block;
The estimation unit includes
Classifying the blocks into the plurality of groups based on a distance from a central portion of the image;
The image processing apparatus according to claim 1, wherein the image characteristic value is calculated based on a statistical value of the pixel values of the blocks belonging to the group. The estimation unit includes
The image processing apparatus according to claim 1, wherein a coefficient for matching the RGB balance of the entire screen is applied when calculating the image characteristic value. A storage unit for storing a plurality of sets of shading estimation coefficients and shading correction coefficients;
The image processing apparatus according to any one of claims 1 to 6, wherein the correction unit selects a shading correction coefficient paired with the appropriate shading estimation coefficient as the corresponding shading correction coefficient. The image processing apparatus according to claim 7, wherein the set is provided for each light source. An estimation step of applying a plurality of shading estimation coefficients to the image and selecting an appropriate shading estimation coefficient for the image;
A correction step of applying a shading correction coefficient corresponding to the appropriate shading estimation coefficient to the image to correct shading of the image, comprising:
The estimation step includes
Classifying the images into a plurality of groups based on a distance from a central portion of the images;
Calculating an image characteristic value for each group when the shading estimation coefficient is applied to the image;
Calculating a statistical value of the image characteristic value, and selecting an appropriate shading estimation coefficient for the image.

Images