JP2015109582A - Image processing device and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device and an image processing method which can perform shading compensation on an image with an appropriate intensity.SOLUTION: An image processing device comprises: a coefficient storage part 122 which stores a combination of a shading estimation coefficient E(x,y) and a shading compensation coefficient C(x,y) of each light source; a shading estimation part 120 which applies the shading estimation coefficient to an image which is an object of shading compensation and selects a suitable shading estimation coefficient for the image; and a shading compensation part 126 which applies the shading compensation coefficient which belongs to a combination the same with that of the selected shading estimation coefficient to the image and compensates for the image.

Description

  The present invention relates to an image processing apparatus and an image processing method.

  When an infrared cut filter is applied to the imaging apparatus, color shading may occur in the captured image. In Patent Document 1, in order to correct such color shading, a plurality of shading correction coefficients are respectively applied to a captured image, a shading correction coefficient that can appropriately correct color shading is selected, and the captured image is selected. An image processing apparatus that corrects and outputs color shading of the captured image by applying the shading correction coefficient is disclosed.

JP 2013-198041 A

  In the above image processing apparatus, a shading correction coefficient that can appropriately correct the color shading of the captured image is selected, and the selected shading correction coefficient is applied to the captured image as it is to correct the color shading of the captured image. It was. However, when the peripheral portion of the image is corrected so as to be equal to the luminance level of the central portion of the image, a high gain is applied to the peripheral portion, and noise near the peripheral portion may be noticeable. there were.

  The present invention has been made to solve such problems, and an object thereof is to provide an image processing apparatus and an image processing method capable of correcting shading of an image with an appropriate intensity.

  An image processing apparatus according to the present invention includes a storage unit that stores a plurality of sets of shading estimation coefficients and shading correction coefficients, and an estimation unit that applies the shading estimation coefficients to an image and selects an appropriate shading estimation coefficient for the image. And a correction unit that applies a shading correction coefficient belonging to the same set as the appropriate shading estimation coefficient to the image and corrects shading of the image. With this configuration, the image can be subjected to shading correction with an appropriate intensity.

  An image processing method according to the present invention includes a step of storing a plurality of sets of shading estimation coefficients and shading correction coefficients, a step of applying the shading estimation coefficients to an image, and selecting an appropriate shading estimation coefficient for the image, Applying a shading correction coefficient belonging to the same set as the appropriate shading estimation coefficient to the image to correct shading of the image. As a result, the image can be subjected to shading correction with an appropriate intensity.

  According to the present invention, it is possible to provide an image processing apparatus and an image processing method capable of correcting shading of an image with an appropriate intensity.

1 is a block diagram showing a configuration of a digital still camera 100 that is an image processing apparatus according to an embodiment. It is a figure for demonstrating each block 208 from which the image block statistics circuit 118 which concerns on embodiment calculates a block statistics value. It is a figure which shows the sampling point 300 of the shading correction coefficient which concerns on embodiment. It is a figure which shows the pixel value before and behind the shading correction which concerns on embodiment. It is a figure which shows a pixel value when each white balance correction process is applied to the pixel value before and after the shading correction which concerns on embodiment.

Embodiments of the present invention will be described below with reference to the drawings.
First, the configuration of the image processing apparatus according to the present embodiment will be described using the digital still camera 100 as an image processing apparatus as an example.

FIG. 1 is a block diagram showing a configuration of a digital still camera 100 that is an image processing apparatus according to the present embodiment.
A digital still camera 100 that is an electronic imaging apparatus includes a lens optical system 102, an imaging element 104, an AFE circuit 106, an image signal processing circuit 108, an image display unit 110, an image recording unit 112, a driver 114, and a timing generator (TG) 116. , An image block statistical circuit 118, a control unit 120, a coefficient storage unit 122, and the like.

  The lens optical system 102 includes a lens, a diaphragm, a shutter, and the like, and forms a subject image on the imaging surface of the imaging element 104. The image sensor 104 is an image sensor such as a CCD or a CMOS, and an infrared cut filter (not shown) is mounted on the lens optical system 102 side of the image sensor 104. The image sensor 104 photoelectrically converts the subject image to generate an image signal (RGB color signal). The AFE circuit 106 A / D converts an image signal generated by the image sensor 104 and subjected to signal processing by a CDS circuit (not shown) into a digital signal.

  The image signal processing circuit 108 performs demosaicing processing, edge enhancement processing, white balance (WB) correction processing, shading correction processing, gamma correction processing, and the like on the image signal output from the AFE circuit 106. The image display unit 110 is a liquid crystal display (LCD) or the like, and displays an image signal that has been subjected to various processes by the image signal processing circuit 108. The image recording unit 112 is a memory, and records the image signal that has been subjected to various processes by the image signal processing circuit 108.

  The driver 114 drives the lens, aperture, and shutter of the lens optical system 102. The timing generator 116 generates timing for driving the image sensor 104. The image block statistical circuit 118 calculates a block statistical value from the image signal converted into a digital signal by the AFE circuit 106. A method for calculating the block statistical value will be described later.

  The control unit 120 controls the white balance correction processing of the image signal processing circuit 108 based on the block statistical value calculated by the image block statistical circuit 118. In addition, the control unit 120 includes a shading estimation unit 124 and a shading correction unit 126.

  The shading estimation unit 124 estimates shading based on the block statistical value calculated by the image block statistical circuit 118 and the shading estimation coefficient stored in the coefficient storage unit 122. The shading correction unit 126 selects a shading correction coefficient stored in the coefficient storage unit 122 based on the shading estimation result of the shading estimation unit 124, and corrects shading of the captured image in the image signal processing circuit 108. The shading estimation process and the shading correction process will be described later.

The coefficient storage unit 122 stores a combination of a shading estimation coefficient and a shading correction coefficient for each light source such as sunlight, a light bulb, and a fluorescent lamp.
Note that the image processing apparatus according to this embodiment includes the image signal processing circuit 108, the image block statistical circuit 118, the control unit 120, the coefficient storage unit 122, and the like, but is not limited to this configuration.

  Moreover, each component which the control part 120 implement | achieves is realizable by making a program run by control of the arithmetic unit (not shown) with which the control part 120 which is a computer is provided, for example. More specifically, the control unit 120 is realized by loading a program stored in a storage unit (not shown) into a main storage device (not shown) and executing the program under the control of the arithmetic unit. Each component is not limited to being realized by software by a program, and may be realized by any combination of hardware, firmware, and software.

  The above-described program can be stored using various types of non-transitory computer readable media and supplied to a computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROMs (Read Only Memory), CD-ROMs. R, CD-R / W, semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)).

  Further, the program may be supplied to the computer by various types of temporary computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

Next, the operation of the image processing apparatus according to this embodiment will be described.
FIG. 2 is a diagram for explaining each block 208 for which the image block statistical circuit 118 according to the present embodiment calculates a block statistical value.

  The image block statistical circuit 118 is a captured image region (region inside the solid line in FIG. 2) 202 of the image signal converted into a digital signal by the AFE circuit 106 or an output image region (FIG. 2) output from the image signal processing circuit 108. 2) so that the block statistical area 206 (area where the dotted grid line in FIG. 2 exists) is the same area or a different area. Set to. The image block statistical circuit 118 may be set so that the block statistical area 206 is wider than the output image area 204.

  The image block statistical circuit 118 divides the block statistical area 206 into a plurality of blocks 208 and calculates a block statistical value for each block 208. The image block statistical circuit 118 calculates a pixel integrated value for each RGB or a pixel average value for each RGB in each block as a block statistical value. In the block statistics area 206, in many cases, the Pmax block (block near the center of the image in FIG. 2) 210 is a block with small color shading, and the Pmin block (block near the image end in FIG. 2) 212 is a block with large color shading. It becomes.

  The shading estimation unit 124 estimates the shading of the captured image based on the block statistical value calculated from the captured image to be subjected to the shading correction process by the image block statistical circuit 118 and the shading estimation coefficient stored in the coefficient storage unit 122. .

The shading estimation coefficient is stored in advance in the coefficient storage unit 122 by photographing a white plate for each light source. The shading estimation coefficient E (x, y) is calculated as shown in equations (1) and (2).

PBW _max = max (PBW ₍₀ , ₀₎ , PBW ₍₀ , ₁₎ ,..., PBW _{(x, y)} )
... (1)

_{E (x, y)} = PBW _max / PBW _{(x, y)} (2)

  In the expressions (1) and (2), PBW (x, y) is a block statistical value of R value, G value, or B value in a block (x, y) of an image obtained by photographing a white plate. Therefore, the shading estimation coefficient is set for each R value, G value, and B value for each light source, and has a coefficient value for each block.

Further, the shading estimation coefficient is set so as to correct the RGB signal that attenuates from the central portion of the image to the peripheral portion to the maximum signal level of each of RGB in the central portion of the image.
The calculation of the shading estimation coefficient and the shading correction coefficient described later can be performed by an external device such as a personal computer.

The shading estimation unit 124 sequentially acquires the shading estimation coefficient prepared for each light source from the coefficient storage unit 122 and applies it to the captured image to be subjected to the shading correction processing as shown in the equation (3) to apply PBE (x , Y) is calculated, and this PBE (x, y) is analyzed to select an appropriate shading estimation coefficient for the captured image.

PBE _{(x, y)} = PB _{(x, y)} · E _{(x, y)} (3)

In Equation (3), PB (x, y) is a block statistical value of R value, G value, or B value in the block (x, y) of the captured image that is the target of the shading correction process.

  The analysis method of PBE (x, y) is the same as the analysis method described in Patent Document 1. Although a detailed description is omitted, the shading estimation unit 124 calculates the deviation degree of the R signal ratio of the blocks continuous from the central part to the peripheral part of the captured image using the standard deviation or the variance, and the smallest R signal ratio. A shading estimation coefficient corresponding to the degree of shift is selected as a shading estimation coefficient appropriate for the captured image.

  The shading correction unit 126 selects a shading correction coefficient based on the shading estimation coefficient selected by the shading estimation unit 124, and corrects the shading of the captured image in the image signal processing circuit 108. Specifically, the coefficient storage unit 122 stores a set of shading estimation coefficients and shading estimation coefficients for each light source. In many cases, the shading estimation coefficient and the shading estimation coefficient are in a one-to-one pair, but the present invention is not limited to this.

  The shading correction unit 126 selects a shading correction coefficient paired with the shading estimation coefficient selected by the shading estimation unit 124 in the coefficient storage unit 122. The shading correction unit 126 applies the selected shading correction coefficient to the captured image and corrects the shading of the captured image. Note that the shading correction coefficient is not completely corrected for shading of the peripheral portion of the captured image, but is corrected with an arbitrary intensity so as to obtain a preferable image quality.

In this way, in the image processing apparatus, the shading estimation unit 124 accurately estimates the shading of the captured image, and the shading correction unit 126 corrects the shading of the captured image with an appropriate intensity.
The image signal processing circuit 108 performs white balance correction processing following the shading correction processing.

Next, a method for setting a shading correction coefficient according to the present embodiment will be described.
FIG. 3 is a diagram showing a sampling point 300 of the shading correction coefficient according to the present embodiment. Sampling points 300 are provided for each of a plurality of pixels, for example, every 5 pixels or every 10 pixels, in an area equivalent to or smaller than the output image area 204. Then, a shading correction coefficient value is set for each sampling point. That is, the shading correction coefficient is set for each R value, G value, and B value for each light source, and has a coefficient value for each sampling point.

  For this reason, when the shading correction unit 126 corrects the shading of the captured image using the shading correction coefficient, the shading correction unit 126 performs an interpolation process such as spline interpolation or linear interpolation on the coefficient value of the shading correction coefficient for each sampling point. The coefficient value of the shading correction coefficient corresponding to the pixel position is calculated.

  In addition, shading correction is a gain correction for a signal that attenuates toward the periphery of the image, but when a signal that attenuates from the center to the periphery of the image is corrected to be equivalent to the center of the image. , Noise in the periphery of an image with a poor S / N ratio is noticeable. Therefore, shading so as to eliminate color shading that occurs due to the difference in shading characteristics of the RGB signal between the center and the periphery of the image, and to correct the shading of the image brightness component with an intensity that makes the noise in the periphery less noticeable. Set the correction factor.

In general, since the image sensor has the highest sensitivity among R, G, and B, the shading correction coefficient is calculated based on the G signal. First, as shown in the equations (4) and (5), the maximum value PWgmax and the minimum value PWgmin at the sampling point of the G signal are obtained.

PWg _max = max (PWg _(0,0) , PWg _(0,1) ,..., PWg _{(x, y)} )
... (4)

PWg _min = min (PWg _(0,0) , PWg _(0,1) ,..., PWg _{(x, y)} )
... (5)

  In equations (4) and (5), PWg (x, y) is applied to each image of the smoothed image by applying a low-pass filter such as a Gaussian filter to the image obtained by photographing the white plate and removing noise. G value at sampling point (x, y). Normally, PWg at the sampling point near the center of the image is selected as PWgmax, and PWg at the sampling point near the corner of the image is selected as PWgmin.

Next, a target level (Tl) that determines which level to correct for PWgmin relative to PWgmax is set. For example, when the PWgmax after shading correction is set to 1.0 and the PWgmin after shading correction is to be set to 0.8, the target level is set to 0.8.
Next, as shown in equation (6), a shading correction strength coefficient K is calculated.

  Next, as shown in Expression (7), a shading correction coefficient Cg (x, y) at each sampling point of the G signal is calculated.

  Next, a method for calculating a shading correction coefficient at each sampling point of the R signal and the B signal will be described. The shading correction coefficients of the R signal and the B signal are obtained so as to coincide with the corrected signal level of the G signal. Thereby, color shading can be removed.

First, as shown in the equation (8), the G signal PWg (x, y) of the captured image of the white plate is corrected with the shading correction coefficient Cg (x, y), and the corrected signal level PWCg ( x, y) is calculated.

PWCg _{(x, y)} = PWg _{(x, y)} · Cg _{(x, y)} (8)

Next, as shown in equations (9) and (10), the maximum values Prmax and Pbmax of the sampling points of the R signal and the B signal are calculated.

PWr _max = max (PWr _(0,0) , PWr _(0,1) ,..., PWr _{(x, y)} )
... (9)

PWb _max = max (PWb ₍₀ , ₀₎ , PWb ₍₀ , ₁₎ ,..., PWb _{(x, y)} )
(10)

  In Equations (9) and (10), PWr (x, y) and PWb (x, y) are applied to a low pass filter such as a Gaussian filter applied to an image obtained by photographing a white plate to remove noise. The R value and the B value at each sampling point (x, y) of the smoothed image.

Next, as shown in the equations (11) and (12), the shading correction coefficient Cr (x, y) for the R signal and the B signal on the basis of the corrected signal level PWCg (x, y) of the G signal. ), Cb (x, y) is calculated. At this time, the reason why PWrmax and PWbmax are multiplied is that the white balance correction process is applied after the shading correction process, so that the maximum value at the center of the image is used as a reference.

Cr _{(x, y)} = PWr _max · (PWCg _{(x, y)} / PWr _{(x, y)} )
(11)
Cb _{(x, y)} = PWb _max · (PWCg _{(x, y)} / PWb _{(x, y)} )
(12)

As described above, the shading correction coefficients Cr (x, y) and Cb (x, y) for the R signal and the B signal can be calculated.
When the shading correction is actually performed on the captured image, the shading correction unit 126 selects the shading correction coefficient paired with the shading estimation coefficient selected by the shading estimation unit 124 from the coefficient storage unit 122 and performs image signal processing. The circuit 108 corrects shading.

For example, when the shading correction is performed on the G signal of the captured image, the shading correction coefficient Cg (x) of the G signal corresponding to the G signal value Pg (x, y) of the captured image, as shown in the equation (13). , y) is multiplied and corrected to obtain a G correction signal value PCg (x, y). Similarly, when shading is corrected for the R signal and B signal of the captured image, correction is performed as shown in the equation (14) or (15). These corrections are performed in units of pixels.

PCg _{(x, y)} = Pg _{(x, y)} · Cg _{(x, y)} (13)
PCr _{(x, y)} = Pr _{(x, y)} · Cr _{(x, y)} (14)
PCb _{(x, y)} = Pb _{(x, y)} · Cb _{(x, y)} (15)

  FIG. 4 is a diagram showing pixel values before and after shading correction according to the present embodiment. The graph shows changes in pixel values before and after correction from the center of the image to the periphery of the image when the target level (Tl) is set to 0.9 and the captured image is subjected to shading correction. A solid line indicates a pixel value before correction, and a dotted line indicates a pixel value after correction. As for each pixel value of G, R, and B, it can be seen that the value at the periphery of the image is about 0.9 times the value at the center of the image.

  FIG. 5 is a diagram illustrating pixel values when white balance correction processing is applied to pixel values before and after shading correction according to the present embodiment. The solid line is the pixel value when white balance correction processing is applied to the pixel value before shading correction, and the dotted line is the white balance correction to the pixel value after shading correction of the captured image with the target level (Tl) set to 0.9 The pixel value when processing is applied is shown.

  When the white balance correction process is applied to the captured image without performing the shading correction, it can be seen that coloring occurs and the RGB level is different from the image center to the image periphery.

  On the other hand, when the white balance correction process is applied after the shading correction according to the present embodiment is performed on the captured image, the G correction signal value PCg, the R correction signal value PCr, and the B correction signal value PCb overlap. It can be seen that the color deviation due to shading is removed over the entire image. Further, the luminance level of the peripheral portion of the image when the luminance level of the central portion of the image is 1.0 can be set to 0.9, and noise in the peripheral portion of the image does not stand out.

In this embodiment, the image processing apparatus has been described by taking the digital still camera 100 as an example. However, the image processing apparatus is configured by an external device such as a personal computer, and shading estimation and shading correction of an image are performed in the device. You may do it.
In the present embodiment, first, the shading correction coefficient of the G signal is obtained, and then the shading correction coefficient of the R signal and the B signal is obtained based on the shading correction coefficient of the G signal. It is not limited to.

  As described above, the image processing apparatus according to the present embodiment applies a storage unit that stores a plurality of combinations of a shading estimation coefficient and a shading correction coefficient, applies a shading estimation coefficient to an image, and applies an appropriate value to the image. An estimation unit that selects a shading estimation coefficient and a shading correction unit that applies a shading correction coefficient belonging to the same set as the appropriate shading estimation coefficient to the image and corrects shading of the image are provided. With this configuration, the image can be subjected to shading correction with an appropriate intensity.

  Further, in the image processing apparatus according to the present embodiment, the shading correction coefficient is obtained by using a reference color signal, which is one of the RGB color signals of the image, at the peripheral portion of the image of the reference color signal. The value is corrected so as to have an arbitrary value at the center of the image, and the color signals other than the reference color signal are corrected so as to have a constant level with respect to the corrected reference color signal.

In the image processing apparatus according to the present embodiment, the arbitrary size is less than 1.
The image processing apparatus according to the present embodiment further includes a statistical unit that divides an image into a plurality of blocks and calculates a statistical value of a pixel value for each block, and the shading estimation coefficient is a coefficient value for each block. The estimation unit multiplies the statistical value and the coefficient value for each block, and selects an appropriate shading estimation coefficient for the image based on the variation in the image of the multiplication result.

In the image processing apparatus according to the present embodiment, the above set is provided for each light source.
In the image processing apparatus according to the present embodiment, the shading correction coefficient has a coefficient value for each of a plurality of pixels of the image.

  Since the image processing apparatus according to the present embodiment estimates shading from a captured image and corrects shading, the shading can be corrected with an appropriate intensity by eliminating a color difference caused by shading characteristics that change depending on the light source.

108 Image signal processing circuit 118 Image block statistical circuit 120 Control unit 122 Coefficient storage unit 124 Shading estimation unit 126 Shading correction unit 202 Captured image area 208 Block 300 Sampling point

Claims (7)

A storage unit for storing a plurality of combinations of shading estimation coefficients and shading correction coefficients;
Applying the shading estimation coefficient to an image and selecting an appropriate shading estimation coefficient for the image;
An image processing apparatus comprising: a correction unit that applies a shading correction coefficient belonging to the same set as the appropriate shading estimation coefficient to the image and corrects shading of the image. The shading correction coefficient is a reference color signal that is any one of the RGB color signals of the image, and the value of the reference color signal at the periphery of the image is at the center of the image. The image processing apparatus according to claim 1, wherein the correction is performed so that the value becomes an arbitrary magnitude, and the color signals other than the reference color signal are corrected so as to have a constant level with respect to the corrected reference color signal. . The image processing apparatus according to claim 2, wherein the arbitrary size is less than one. Further comprising a statistic unit that divides the image into a plurality of blocks and calculates a statistical value of a pixel value for each block;
The shading estimation coefficient has a coefficient value for each block,
The estimation unit multiplies the statistical value and the coefficient value for each block, and selects a shading estimation coefficient appropriate for the image based on a variation in the image of a multiplication result. The image processing device according to any one of items 3 to 4. The image processing apparatus according to claim 1, wherein the set is provided for each light source. The image processing apparatus according to claim 1, wherein the shading correction coefficient has a coefficient value for each of a plurality of pixels of the image. Storing a plurality of combinations of shading estimation coefficients and shading correction coefficients;
Applying the shading estimation factor to an image and selecting an appropriate shading estimation factor for the image;
Applying a shading correction coefficient belonging to the same set as the appropriate shading estimation coefficient to the image, and correcting the shading of the image.