JP5681589B2 - Imaging apparatus and image processing method - Google Patents

Description

  The present invention relates to an imaging apparatus and an image processing method, and more particularly to a technique for performing color shading correction.

  Conventionally, as this type of color shading correction method, the color temperature of a photographed scene is estimated, and the shading correction amount of each primary color signal of red (R), green (G), and blue (B) is determined accordingly. In many cases, the intensity of each primary color signal is corrected.

  Patent Document 1 discloses a shading correction unit that performs shading correction on a captured subject image according to a shading coefficient, a color temperature estimation unit that estimates a color temperature of ambient light, and a shading coefficient based on the estimated color temperature. There has been proposed an image processing apparatus that includes a shading coefficient correction unit for correcting, and that can obtain a high-quality image by correcting a difference in color appearance due to the influence of color temperature. In addition, there is a description in which an ambient light sensor that detects ambient light is provided as part of the color temperature estimation unit.

  Patent Document 2 illuminates a scene based on a first color image sensor, a second color image sensor that detects a color distribution of the scene, and a color distribution detected by the second color image sensor. Determining means for determining the type of illumination light; and setting means for setting a correction amount for color shading correction to be applied to an image generated by the first color imaging device according to the type of illumination light determined by the determining means; There has been proposed a color imaging apparatus capable of enhancing the effect of color shading correction. Patent Document 2 describes changing the shading correction amount for each region.

  Japanese Patent Application Laid-Open No. 2004-228561 describes a method for making color shading inconspicuous by recognizing a scene where color shading is conspicuous and then suppressing the saturation of a portion where color shading occurs.

JP 2010-147800 A JP 2009-88800 A JP 2008-153834 A

  Although there is a description that an ambient light sensor is provided as part of the color temperature estimation unit described in Patent Document 1, the ambient light sensor is not a camera. That is, it is provided exclusively for detecting ambient light separately from the image sensor, which increases the number of parts and increases the cost.

  Since the subject light separated by the half mirror is incident on the second color image sensor described in Patent Document 2, the type of illumination light determined based on the color distribution detected by the second color image sensor is The determination is made from the same scene as the shooting scene. Therefore, the color temperature estimation accuracy is not high. Further, the color imaging device described in Patent Document 2 does not fit in the camera size of a mobile phone.

  Incidentally, color shading is a concern when there are objects with high flatness such as whiteboards and walls that can be easily compared in color between the center and the periphery. In addition, since color shading depends on ambient light, spectral reflection characteristics of an object, photographing lens, imaging device, and the like, color shading generated in the object region as described above is completely performed by the methods described in Patent Documents 1 and 2. It is difficult to correct.

  On the other hand, as described in Patent Document 3, a technique for reducing color saturation and making color shading inconspicuous with respect to a portion where color shading occurs is not a color shading correction, and a dull color is obtained. There is a problem that a part comes out.

  The present invention has been made in view of such circumstances, and it is possible to improve the estimation accuracy of the color temperature of the ambient light or the type of light source without the addition of new hardware, thereby performing color shading correction satisfactorily. In addition, an object of the present invention is to provide an imaging apparatus and an image processing method capable of reliably correcting color shading in a portion where color shading is easily noticeable.

  In order to achieve the above object, one aspect of the present invention is an imaging apparatus having a first imaging unit and a second imaging unit for self-portion provided on the opposite side of the first imaging unit. First estimation means for estimating a color temperature or a light source type of ambient light based on the color image acquired from the first imaging unit; and environmental light based on the color image acquired from the second imaging unit. Second estimation means for estimating a color temperature or a light source type; first shading correction amount determination means for determining a first color shading correction amount from the color temperature or the light source type estimated by the first estimation means; Second shading correction amount determining means for determining a second color shading correction amount from the color temperature or light source type estimated by the second estimating means; the determined first color shading correction amount and second color; Shading Is characterized by comprising a color shading correction means for performing color shading correction for each color pixel of the color image on the acquired color image from the first image-acquisition unit based on the correction amount.

  According to one aspect of the present invention, the first image pickup unit used during normal shooting and the second image pickup unit for self-photographing provided on the opposite side of the first image pickup unit are used in combination. Based on the color image acquired from the first imaging unit or the color image acquired from the second imaging unit, the color temperature or the light source type of ambient light can be estimated, respectively. Thereby, the estimation accuracy of the color temperature of the ambient light or the light source type can be increased. The second image pickup unit for self-portrait is provided in advance in a camera-equipped mobile phone having a videophone-compatible function. In the present invention, the second image pickup unit for self-portrait is replaced with the color temperature or light source type of ambient light. It is used as a means for acquiring a color image for estimating.

  By the way, scenes taken with a camera-equipped mobile phone are often taken indoors at a low angle (there is a high possibility that there is a light source on the side of the second imaging unit for selfies). For this reason, it is preferable to use the second imaging unit facing upward at a low angle as a color image acquisition unit for estimating the color temperature or the light source type of the ambient light.

  On the other hand, when shooting a subject that is a little far away, it is often taken so that it is not backlit. As a result, the light source is often above or behind the photographer. Therefore, it is desirable to use a second imaging unit that can capture the back side.

  In the imaging apparatus according to another aspect of the present invention, a planar object having a certain area or more is extracted from a color image that has been color-shaded corrected based on the first color-shading correction amount by the color-shading correction unit. A color shading amount analyzing unit for detecting a color shading amount in the object, and the color shading correction unit detects the second color shading correction when the color shading amount of a certain value or more is detected by the color shading amount analyzing unit; Color shading correction is performed based on the amount.

  The color shading is easily noticeable on a planar object having a certain area or more, and can be easily detected by comparing the color of the area on the center side of the screen with the area on the peripheral side. Therefore, normal color shading correction based on the color image acquired from the first imaging unit is performed, and a planar object having a certain area or more is extracted from the color image subjected to the color shading correction. When a color shading amount of a certain value or more is detected on the planar object, it is considered that the color shading correction based on the first color shading correction amount is not sufficient, and based on the color image acquired from the second imaging unit. Color shading correction based on the second color shading correction amount is performed.

  In the imaging apparatus according to still another aspect of the present invention, the color shading correction unit detects only the extracted planar object region when the color shading amount analysis unit detects a color shading amount of a certain value or more. Shading correction is performed based on the second color shading correction amount. As a result, it is possible to satisfactorily correct color shading in a planar object region where color shading is conspicuous.

In the imaging apparatus according to still another aspect of the present invention, a planar object having a certain area or more is extracted from a color image that has been color-shaded corrected based on the first color-shading correction amount by the color-shading correction unit, and color shading amount analyzing means for detecting the color shading amount in a planar object, the color shading of a certain value or more by the color shading amount analyzing means is detected, the third color shading correction Ru reduce the color shading amount A third shading correction amount determining means for determining an amount, wherein the color shading correction means detects the third color shading when a color shading amount of a certain value or more is detected by the color shading amount analyzing means. Color shading correction is performed based on the correction amount. As a result, color shading correction can be performed based on the third color shading correction amount for correcting color shading in the area of the planar object, and color shading in the area of the planar object can be favorably corrected.

  In the imaging apparatus according to still another aspect of the present invention, the color shading correction unit detects only the extracted planar object region when the color shading amount analysis unit detects a color shading amount of a certain value or more. Shading correction is performed based on the third color shading correction amount.

According to still another aspect of the present invention, there is provided an imaging apparatus having a first imaging unit and a second imaging unit for self-photographing provided on a side opposite to the first imaging unit. Second estimating means for estimating a color temperature or light source type of ambient light based on a color image acquired from the unit, a shading correction amount determining means for determining a color shading correction amount from the estimated color temperature or light source type, e Bei and a color shading correction means for performing color shading correction for each color pixel of the color image to the color image obtained from the first image pickup unit based on the color shading correction amount with the determined, the second estimation means, especially to estimate the color temperature or light source type of ambient light based on the most high luminance color image among the plurality of color image obtained from the second imaging unit during photographing preparation period It is set to.

  According to still another aspect of the present invention, the color temperature or light source type of ambient light is estimated based on the color image acquired from the second imaging unit for self-portrait, and the color temperature or light source type of ambient light is estimated. I try to increase the accuracy.

  In the imaging apparatus according to still another aspect of the present invention, the second estimation unit is based on a color image having the highest luminance among a plurality of color images acquired from the second imaging unit during a shooting preparation period. Thus, the color temperature of the ambient light or the light source type is estimated. There may be a case where the second image pickup unit for self-portrait faces upward during the shooting preparation period, and the possibility that the light source is reflected becomes high. Therefore, the color of the ambient light is estimated by estimating the color temperature of the ambient light or the type of the light source based on the color image having the highest luminance among the multiple color images (the color image in which the light source is highly likely to be reflected). The estimation accuracy of temperature or light source type can be increased.

  In the imaging apparatus according to still another aspect of the present invention, the second estimation unit extracts a face area from the color image acquired from the second imaging unit, and based on the image of the face area, the ambient light The color temperature or the light source type is estimated. The color image acquired from the second imaging unit for self-photographing is likely to have a large photographer's face, and the color of the subject (face) is also specified. By using these images, it is possible to accurately estimate the color temperature of the ambient light or the light source type.

  Still another aspect of the present invention is an image processing method in an imaging apparatus having a first imaging unit and a second imaging unit for self-taking provided on the opposite side of the first imaging unit. A first estimation step of estimating a color temperature or a light source type of ambient light based on the color image acquired from the first imaging unit; and an environment light based on the color image acquired from the second imaging unit. A second estimation step for estimating a color temperature or a light source type; a first shading correction amount determination step for determining a first color shading correction amount from the color temperature or the light source type estimated by the first estimation step; A second shading correction amount determining step for determining a second color shading correction amount from the color temperature or light source type estimated in the second estimating step; and the determined first color shading correction amount and second color Shading It is characterized in that it comprises a color shading correction step of performing color shading correction for each color pixel of the color image on the acquired color image from the first image-acquisition unit based on the correction amount.

  In the image processing method according to still another aspect of the present invention, the color shading correction step performs color shading correction of the color image based on the first color shading correction amount, and the color image subjected to the color shading correction. A color shading amount analyzing step of extracting a planar object having a certain area or more from the color object and detecting a color shading amount in the planar object, wherein the color shading correction step includes color shading of a certain value or more by the color shading amount analyzing step. Only when the amount is detected, the color shading correction is performed again based on the second color shading correction amount.

  In the image processing method according to still another aspect of the present invention, the color shading correction step may include only the extracted planar object region when a color shading amount of a certain value or more is detected by the color shading amount analysis step. Shading correction is performed based on the second color shading correction amount.

In the image processing method according to still another aspect of the present invention, the color shading correction step performs color shading correction of the color image based on the first color shading correction amount, and the color image subjected to the color shading correction. A plane object having a certain area or more is extracted from the color shading amount analyzing step of detecting a color shading amount in the plane object, and when a color shading amount of a certain value or more is detected by the color shading amount analysis step, and a third of the shading correction amount determination step of determining a third color shading correction amount Ru reduce color shading amount, a, the color shading correction process, color shading of a certain value or more by the color shading amount analyzing step Only when the amount is detected, the third color shading compensation. And to perform color shading correction on the basis of the amount.

  In the image processing method according to still another aspect of the present invention, the color shading correction step may include only the extracted planar object region when a color shading amount of a certain value or more is detected by the color shading amount analysis step. Shading correction is performed based on the third color shading correction amount.

Still another aspect of the present invention is an image processing method in an imaging apparatus having a first imaging unit and a second imaging unit for self-taking provided on the opposite side of the first imaging unit. A second estimation step of estimating a color temperature or light source type of ambient light based on a color image acquired from the second imaging unit, and shading for determining a color shading correction amount from the estimated color temperature or light source type A correction amount determining step, and a color shading correction step of performing color shading correction for each color pixel of the color image with respect to the color image acquired from the first imaging unit based on the determined color shading correction amount. unrealized, the second estimating step, said second plurality of most color temperature of high luminance on the basis of the color image the ambient light or the light source of the color image obtained from the imaging unit of during photographing preparation period It is characterized by estimating a.

  In the image processing method according to still another aspect of the present invention, the second estimation step is performed on the color image having the highest luminance among the plurality of color images acquired from the second imaging unit during the shooting preparation period. It is preferable to estimate the color temperature of the ambient light or the light source type based on it.

  In the image processing method according to still another aspect of the present invention, the second estimation step extracts a face area from the color image acquired from the second imaging unit, and the ambient light based on the image of the face area. The color temperature or the light source type is estimated.

  According to the present invention, based on a color image acquired in advance from a self-portrait imaging unit (second imaging unit) provided in the apparatus in addition to an imaging unit (first imaging unit) during normal shooting. Therefore, it is possible to improve the estimation accuracy of the color temperature or the light source type of the ambient light without adding new hardware, thereby performing the color shading correction well. be able to. In addition, since a planar object with a certain area or more where color shading is conspicuous is extracted and color shading correction is performed by detecting the amount of color shading in this planar object, color shading in areas where color shading is conspicuous is reliably corrected. can do.

1 is a side view of a camera-equipped mobile phone to which an imaging apparatus according to the present invention is applied. The block diagram which shows the whole structure of the imaging device mounted in the mobile phone with a camera Block diagram showing the internal configuration of the imaging unit The flowchart which shows the flow of a process of 1st Embodiment of the image processing method which concerns on this invention. The figure which shows an example of the image containing the plane object which has color shading The flowchart which shows the flow of a process of 2nd Embodiment of the image processing method which concerns on this invention.

  Embodiments of an imaging apparatus and an image processing method according to the present invention will be described below with reference to the accompanying drawings.

[Imaging device]
FIG. 1 is a side view of a camera-equipped mobile phone to which an imaging apparatus according to the present invention is applied.

  The camera-equipped mobile phone includes an imaging unit 1 (first imaging unit) for normal photography, and an imaging unit 2 (second imaging unit) for self-photographing provided on the opposite side of the imaging unit 1. Is provided. Normally, only one of these imaging units 1 and 2 is selected, and a captured image can be displayed on the display unit 30 provided on the same side as the imaging unit 2. In FIG. 1, A indicates an image captured by the imaging unit 1, and B indicates an image captured by the imaging unit 2.

  FIG. 2 is a block diagram showing the overall configuration of the imaging device 10 mounted on the camera-equipped mobile phone.

  As shown in the figure, the imaging apparatus 10 includes two imaging units 1 and 2. One of these imaging units 1 and 2 can be selected and used by an operation on the operation unit 14. An image taken by the imaging unit 1 can be displayed on the display unit 30 and can be recorded in an external recording unit 28 (or an internal recording unit not shown) such as a memory card. The imaging unit 2 is a self-portrait camera and can be used as a Web camera. In the present invention, when the imaging unit 1 is selectively used, the imaging unit 2 is also operated as will be described later.

  The central processing unit (CPU 12) is connected to the imaging units 1 and 2 via the control bus 16, and includes a main memory 20, an image processing unit 22, an integrating unit 24, a compression / decompression processing unit 26, an external recording unit 28, And the display unit 30 are connected, and the CPU 12 functions as a control unit that comprehensively controls the operation of the entire imaging apparatus 10 according to a predetermined control program based on an operation input from the operation unit 14.

  The imaging units 1 and 2, the main memory 20, the image processing unit 22, the integration unit 24, the compression / decompression processing unit 26, the external recording unit 28, and the display unit 30 are connected via a data bus 38.

  The imaging units 1 and 2 are each configured as an imaging module, and FIG. 3 shows an internal configuration of the imaging unit 1.

  As shown in FIG. 3, the imaging unit 1 mainly includes a photographing lens 40, a diaphragm 41, an IR cut filter 42, an optical low-pass filter 43, an imaging element (CMOS or CCD image sensor) 44, an A / D converter 45, and a lens driving unit. 46, an aperture drive unit 47, and an image sensor drive unit 48.

  The photographing lens 40 includes a focus lens, and is driven by the lens driving unit 46 to move back and forth along the optical axis. The CPU 12 controls the driving of the lens driving unit 46 to control the position of the focus lens and adjust the focus so that the subject is in focus.

  The diaphragm 41 is constituted by, for example, an iris diaphragm, and is driven by the diaphragm driving unit 47 to operate. The CPU 12 controls the aperture amount (aperture value) of the aperture 41 via the aperture drive unit 47 and controls the amount of light incident on the image sensor 44.

  The image sensor 44 is constituted by a two-dimensional color image sensor. A large number of photodiodes are two-dimensionally arranged on the light receiving surface of the image pickup element 44, and RGB color filters are arranged in a predetermined arrangement (such as a Bayer arrangement) in each photodiode. The optical image of the subject imaged on the light-receiving surface of the image sensor through the photographing lens 40, the diaphragm, the IR cut filter 42, and the optical low-pass filter 43 is converted into a signal charge corresponding to the amount of incident light by the photodiode. . The signal charge accumulated in each photodiode is sequentially read out from the image sensor 44 as a voltage signal (image signal) corresponding to the signal charge based on a drive pulse supplied from the image sensor driver 48 in accordance with a command from the CPU 12. The image sensor 44 has an electronic shutter function, and the exposure time (shutter speed) is controlled by controlling the charge accumulation time in the photodiode.

  The image signal read from the image sensor 44 is converted into a digital signal by the A / D converter 45 and then temporarily stored in the main memory 20 via the data bus 38.

  The image pickup unit 2 may have the same configuration as the image pickup unit 1, but generally a color image pickup device having a smaller number of pixels than the image pickup unit 1 is used, and the photographing lens does not have a focus adjustment function. used. This is because the imaging unit 2 is used for self-portrait, and the subject and the shooting distance are fixed.

  When imaging by the imaging unit 1 is selected, the CPU 12 drives the imaging unit 2 together with the imaging unit 1 at the same time.

  The operation unit 14 includes known operation input means such as a shutter button, a cross button, a determination key, and a numeric keypad. The shutter button is composed of a two-stage stroke type switch composed of so-called “half press” and “full press”. When the shutter button is pressed halfway in the shooting mode, shooting preparation processing (that is, AE (Automatic Exposure), AF (Auto Focus), AWB (Automatic White Balance)) is performed. When the shutter button is fully pressed, image capturing / recording processing is performed.

  The main memory 20 is used as a work area when the CPU 12 executes a program, and as a storage unit that temporarily stores digital image signals captured and acquired by the imaging units 1 and 2.

  The image processing unit 22 functions as a digital signal processing unit including a color shading correction unit, a noise suppression unit, a white balance processing unit, a demosaic processing unit, a linear matrix unit, a gamma correction unit, a contour correction unit, and the like. Predetermined signal processing is performed on the R, G, and B image data stored in. The detailed operation of the image processing unit 22 will be described later.

  The accumulating unit 24 calculates a focus evaluation value necessary for AF control based on an image signal captured when the shutter button is half-pressed, and RGB necessary for subject brightness, color shading correction, and white balance correction necessary for AE control. The integrated value for each color signal is calculated. The CPU 12 searches for a position where the focus evaluation value calculated by the integrating unit 24 is maximized during AF control, moves the focus lens to that position, thereby focusing on the subject (main subject), and performs AE control. Sometimes, exposure setting is performed to obtain an appropriate exposure amount based on the subject brightness calculated by the integrating unit 24. That is, the shooting sensitivity, aperture value, and shutter speed are set.

  The compression / decompression processing unit 26 performs compression processing on the input image data in accordance with a command from the CPU 12 to generate compressed image data in a predetermined format. For example, compression processing conforming to the JPEG standard is applied to still images, and MPEG2, MPEG4, H.264, and the like are applied to moving images. A compression process conforming to the H.264 standard is performed. In addition, the compression / decompression processing unit 26 performs decompression processing on the input compressed image data in accordance with a command from the CPU 12 to generate uncompressed image data.

  The external recording unit 28 creates an image file storing the image data in JPEG format or the like generated by the compression / decompression processing unit 26, and records the file on a removable external recording medium such as a memory card, or from the external recording medium. Read image file.

  The display unit 30 is configured by, for example, a display device including a color liquid crystal panel. When the camera-equipped mobile phone is used as a camera, the display unit 30 is used as an electronic viewfinder for confirming the angle of view, and a captured image. Is used as a GUI for various settings.

[Image processing method]
<First Embodiment>
FIG. 4 is a flowchart showing the processing flow of the first embodiment of the image processing method according to the present invention, and particularly shows the case of processing a still image taken by the imaging unit 1.

  RGB mosaic images corresponding to the color filters captured by the imaging units 1 and 2 are temporarily stored in the main memory 20.

  In FIG. 4, the shading correction unit of the image processing unit 22 reads the mosaic image captured by the imaging unit 1 from the main memory 20 and performs color shading correction on the mosaic image (step S <b> 10). This color shading correction is preferably performed after the R, G, and B signals of the mosaic image are offset corrected.

  The shading correction unit calculates the color temperature of the ambient light from the mosaic image. First, using the R, G, and B signals of the mosaic image, an average integrated value for each color of the RGB signal is calculated for each of 256 divided areas in which one screen is divided into 16 × 16, and R for each divided area. , G and B, the color information comprising the ratio of the average integrated values, that is, the ratio of R / G and B / G is calculated. Then, the centroid position of the distribution of the color information for each of the 256 divided areas in the color space of the R / G and B / G axis coordinates is calculated, and the color temperature of the ambient light is estimated from the color information indicated by the centroid position. To do. In addition, instead of the color temperature, a light source type having color information indicated by the center of gravity position, for example, blue sky, shade, clear, fluorescent lamp (daylight color, neutral white, white, warm white), tungsten, low tungsten, etc. The light source type at the time of shooting may be estimated (see Japanese Patent Application Laid-Open No. 2007-53499), or the color temperature may be estimated from the estimated light source type.

  Subsequently, the shading correction unit determines a color shading correction coefficient for each RGB based on the estimated color temperature of the ambient light. For example, the color shading correction amount can be expressed by a polynomial expression assuming a shading correction characteristic that is symmetrical in the vertical and horizontal directions with respect to the screen center, and using a distance x from the screen center as a variable as shown in the following equation.

[Equation 1]
Color shading correction amount = ax ⁶ + bx ⁴ + cx ² + d
The color shading correction coefficient means the coefficient (a, b, c, d) in the above [Expression 1], and the color shading correction coefficient for each RGB according to the color temperature (or light source type) of the ambient light. (A to d) are prepared in advance for the imaging unit 1 and stored in a nonvolatile memory (not shown).

  The shading correction unit reads the color shading correction coefficient for each RGB corresponding to the estimated color temperature of the ambient light, and calculates the color shading correction amount based on the position and color of the correction target pixel from the equation [1]. Then, color shading correction is performed based on the calculated color shading correction amount. The color shading correction amount is not limited to the case where the color shading correction amount is calculated from the function shown in [Expression 1]. The color shading correction amount is determined in advance for each section divided into screens, and interpolation values are used for pixels between the sections. May be.

  When color shading correction is performed on all the pixels of the mosaic image, noise is subsequently suppressed by applying a noise reduction filter by the noise suppression unit (step S12). Next, white balance correction is performed on the image whose noise is suppressed by the white balance processing unit (step S14). That is, a WB gain for each RGB or RB for performing an appropriate white balance correction corresponding to the color temperature of the ambient light is prepared in advance, and the white balance processing unit determines the color temperature of the ambient light at the time of shooting. Based on the WB gain, the corresponding WB gain for each RGB or RB is read out, and white balance correction is performed using the read WB gain.

  Next, the color shading amount analysis unit analyzes the color shading amount of the image after the white balance processing (step S16). The analysis of the amount of color shading begins with areas where color shading is conspicuous from the image after white balance processing (for example, areas with a flat object with a high flatness such as a whiteboard or wall and having a certain area or more). Extract. Such a planar object can be extracted based on the area of the region divided by the contour, the luminance difference, the color difference, and the like by binarizing the image to obtain the contour of the image.

  FIG. 5 is a diagram illustrating an example of an image including a region X of a planar object having color shading. Subsequently, the color shading amount of the extracted area X of the planar object is obtained. The color shading amount of the area X of the planar object includes the color ratio (R / G, B / G) of the area closest to the screen center in the area X of the planar object and the color ratio (R of the area farthest from the screen center). / G, B / G) or the difference. This is because color shading gradually changes from the center of the screen toward the periphery as shown in FIG.

  If it is determined that the color shading amount obtained as described above is small, it is determined that the color shading correction in step S10 has been properly performed, and normal image processing is performed as it is. That is, a demosaic process (step S18) for converting the color signal into a simultaneous expression by interpolating a spatial shift of the color signal associated with the color filter array of the single-plate image sensor from the mosaic image after the white balance process (step S18), Images such as linear matrix processing using a matrix (step S20), gamma correction (step S22) for gradation conversion so as to obtain a required gradation characteristic (gamma curve), and contour correction (step S24) for edge enhancement processing. Processing is performed.

  On the other hand, if it is determined that the color shading amount is large, the process proceeds to step S32 to correct the color shading correction coefficient.

  That is, the light source type is estimated based on the color image captured by the self-portrait imaging unit 2, and the color temperature is estimated from the estimated light source type (step S30). The light source type estimation method in step S30 is not limited to the method described in the processing in step S10, and the following method may be applied.

  (1) As shown in FIG. 1, in the image B photographed by the imaging unit 2, there is a high possibility that the light source illuminating the subject of the imaging unit 1 is reflected. In view of this, in the image B captured by the imaging unit 2, an extremely high brightness area is considered as an area where the light source is reflected, and the light source is estimated from the RGB signal intensity ratio of the image in that area. If the luminance of the light source is too high, the luminance of the light source may be reduced using a diffusion plate or a dimming plate.

  In addition, there may be a case where the imaging unit 2 captures the light source for 5 to 10 seconds (imaging preparation period) before determining the composition and focusing. Therefore, it is preferable to estimate the light source type using an image having the highest luminance among images continuously captured by the imaging unit 2 during the shooting preparation period.

  (2) The face of the photographer is greatly reflected in the image B photographed by the imaging unit 2 for self-portrait. Therefore, a face area is extracted from the image B photographed by the imaging unit 2, and a light source type is estimated from the color of the extracted face area.

  A color shading correction coefficient is determined based on the color temperature of the light source type estimated as described above, and the determined color shading correction coefficient is applied to the calculation of the color shading correction amount in step S10 (step S32). In step S10, a color shading correction amount is newly calculated based on the above-described [Equation 1], and color shading correction is performed using the calculated color shading correction amount.

  The color shading correction may be performed on the image after the white balance processing as in the flow line indicated by the dotted line in FIG. In this case, the color shading correction is performed by the difference between the color shading correction amount in step S10 and the color shading correction amount calculated from the color shading correction coefficient obtained in step S32.

  For the color shading correction, in addition to the method of changing the color shading correction amount (Gain) for each RGB color in the shading correction unit in step S10, the B pixel and the R pixel of the image after the white balance processing are completed. There is a way to change the shading amount. In either method, color shading correction is generally performed on the entire screen, but when there are two planar areas, short distance and long distance, spots may appear in the remaining color shading. Therefore, it is possible to extract only a portion where the color shading is conspicuous (a planar object having a certain area or more) and apply color shading correction only to that portion. This method can also be used when there are a plurality of light source types.

  When the color shading correction image is determined to be below the threshold by the color shading amount analysis unit, or when the color shading is the smallest among the processes so far, normal image processing is performed. And output.

  Further, when color shading correction is performed on the area X of the planar object as shown in FIG. 5, the color shading correction amount is determined so that the color of the peripheral portion matches the color of the area close to the center of the area X. Specifically, for the R and B pixels so that the color ratio (R / G, B / G) of the peripheral portion matches the color ratio (R / G, B / G) of the central portion of the area X of the planar object. Each color shading correction amount is obtained, and the obtained color shading correction amount is applied to R and B pixels to perform color shading correction so that the area of the planar object has a uniform color. In this case, the obtained color shading correction amount may be applied to the R and B pixels of the entire screen, or may be applied only to the R and B pixels of the extracted planar object region.

  In order to distinguish from a planar object having a color gradation, the amount of color shading in the central portion and the peripheral portion is set to about 0.975 to 1.025. This is because a flat object having a color gradation is not corrected by color shading correction.

<Second Embodiment>
FIG. 6 is a flowchart showing the flow of processing of the second embodiment of the image processing method according to the present invention, and particularly shows the case of processing a still image taken by the imaging unit 1. In addition, the same step number is attached | subjected to the part which is common in 1st Embodiment shown in FIG. 4, The detailed description is abbreviate | omitted.

  In FIG. 6, the shading correction unit in step S <b> 10 ′ reads out the mosaic image captured by the imaging unit 1 from the main memory 20 and acquires the color shading correction coefficient calculated in step S <b> 32. Then, color shading correction is performed on the mosaic image with the color shading correction amount calculated corresponding to the color shading correction coefficient.

  The mosaic image after the color shading correction is output after being subjected to image processing in steps S12 to S24.

  As in the first embodiment, a region of a planar object in which color shading is conspicuous is extracted from an image after white balance processing, and color shading correction is performed so that the extracted region of the planar object has a uniform color. It may be. Further, during the white balance process, the color temperature or the light source type of the ambient light may be estimated based on the color image acquired from the imaging unit for self-portrait, and the estimation result may be used.

  Furthermore, it goes without saying that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention.

  DESCRIPTION OF SYMBOLS 1, 2 ... Imaging part, 10 ... Imaging apparatus, 12 ... Central processing part (CPU), 14 ... Operation part, 20 ... Main memory, 22 ... Image processing part, 24 ... Integration part, 30 ... Display part

Claims (16)

In an imaging apparatus having a first imaging unit and a second imaging unit for self-taking provided on the opposite side of the first imaging unit,
First estimation means for estimating a color temperature or a light source type of ambient light based on a color image acquired from the first imaging unit;
Second estimation means for estimating a color temperature or a light source type of ambient light based on a color image acquired from the second imaging unit;
First shading correction amount determining means for determining a first color shading correction amount from the color temperature or light source type estimated by the first estimating means;
Second shading correction amount determining means for determining a second color shading correction amount from the color temperature or light source type estimated by the second estimating means;
Color shading for performing color shading correction for each color pixel of the color image with respect to the color image acquired from the first imaging unit based on the determined first color shading correction amount and second color shading correction amount Correction means;
An imaging apparatus comprising: Color shading amount analysis for extracting a planar object having a certain area or more from a color image corrected by color shading based on the first color shading correction amount by the color shading correction means and detecting a color shading amount in the planar object With means,
2. The imaging according to claim 1, wherein the color shading correction unit performs color shading correction based on the second color shading correction amount when the color shading amount analysis unit detects a color shading amount of a certain value or more. apparatus.   When the color shading correction unit detects a color shading amount equal to or greater than a predetermined value by the color shading amount analysis unit, the color shading correction unit performs shading correction only on the extracted planar object region based on the second color shading correction amount. The imaging device according to claim 2 to be performed. Color shading amount analysis for extracting a planar object having a certain area or more from a color image corrected by color shading based on the first color shading correction amount by the color shading correction means and detecting a color shading amount in the planar object Means,
A third shading correction amount determining means for determining a third color shading correction amount for reducing the color shading amount when a color shading amount of a certain value or more is detected by the color shading amount analyzing means;
4. The color shading correction unit according to claim 1, wherein the color shading correction unit performs color shading correction based on the third color shading correction amount when the color shading amount analysis unit detects a color shading amount of a certain value or more. The imaging apparatus of Claim 1.   When the color shading correction unit detects a color shading amount greater than a certain value by the color shading amount analysis unit, the color shading correction unit performs shading correction only on the extracted planar object region based on the third color shading correction amount. The imaging device according to claim 4 to be performed. In an imaging apparatus having a first imaging unit and a second imaging unit for self-taking provided on the opposite side of the first imaging unit,
Second estimation means for estimating a color temperature or a light source type of ambient light based on a color image acquired from the second imaging unit;
A shading correction amount determining means for determining a color shading correction amount from the estimated color temperature or light source type;
E Bei and a color shading correction means for performing color shading correction for each color pixel of the color image to the color image obtained from the first image pickup unit based on the color shading correction amount the determined,
The second estimating means is an imaging that estimates a color temperature or a light source type of ambient light based on a color image having the highest luminance among a plurality of color images acquired from the second imaging unit during a shooting preparation period. apparatus. The second estimating means estimates a color temperature or a light source type of ambient light based on a color image having the highest luminance among a plurality of color images acquired from the second imaging unit during a shooting preparation period. Item 6. The imaging device according to any one of Items 1 to 5 .   The second estimation means extracts a face area from a color image acquired from the second imaging unit, and estimates a color temperature or a light source type of ambient light based on the image of the face area. The imaging device according to any one of the above. An image processing method in an imaging apparatus having a first imaging unit and a second imaging unit for self-taking provided on the opposite side of the first imaging unit,
A first estimation step of estimating a color temperature or a light source type of ambient light based on a color image acquired from the first imaging unit;
A second estimation step of estimating a color temperature or a light source type of ambient light based on a color image acquired from the second imaging unit;
A first shading correction amount determining step for determining a first color shading correction amount from the color temperature or the light source type estimated in the first estimating step;
A second shading correction amount determination step of determining a second color shading correction amount from the color temperature or the light source type estimated in the second estimation step;
Color shading for performing color shading correction for each color pixel of the color image with respect to the color image acquired from the first imaging unit based on the determined first color shading correction amount and second color shading correction amount A correction process;
An image processing method comprising: The color shading correction step performs color shading correction of the color image based on the first color shading correction amount,
A color shading amount analyzing step of extracting a planar object having a certain area or more from the color image subjected to the color shading correction and detecting a color shading amount in the planar object;
The color shading correction step performs color shading correction again based on the second color shading correction amount only when a color shading amount equal to or greater than a predetermined value is detected by the color shading amount analysis step. Image processing method.   In the color shading correction step, when a color shading amount of a predetermined value or more is detected by the color shading amount analysis step, shading correction is performed only on the extracted planar object region based on the second color shading correction amount. The image processing method according to claim 10 to be performed. The color shading correction step performs color shading correction of the color image based on the first color shading correction amount,
A color shading amount analyzing step of extracting a planar object having a certain area or more from the color image corrected for color shading, and detecting a color shading amount in the planar object;
A third shading correction amount determining step for determining a third color shading correction amount for reducing the color shading amount when a color shading amount of a certain value or more is detected by the color shading amount analyzing step;
The color shading correction step performs color shading correction based on the third color shading correction amount only when a color shading amount of a certain value or more is detected by the color shading amount analysis step. The image processing method according to claim 1.   In the color shading correction step, when a color shading amount of a certain value or more is detected by the color shading amount analyzing step, only the extracted planar object region is subjected to shading correction based on the third color shading correction amount. The image processing method according to claim 12 to be performed. An image processing method in an imaging apparatus having a first imaging unit and a second imaging unit for self-taking provided on the opposite side of the first imaging unit,
A second estimation step of estimating a color temperature or a light source type of ambient light based on a color image acquired from the second imaging unit;
A shading correction amount determining step for determining a color shading correction amount from the estimated color temperature or light source type;
Look containing a color shading correction step of performing color shading correction for each color pixel of the color image to the color image obtained from the first image pickup unit based on the color shading correction amount the determined,
The second estimating step is an image for estimating a color temperature or a light source type of ambient light based on a color image having the highest luminance among a plurality of color images acquired from the second imaging unit during the shooting preparation period. Processing method. The second estimating step estimates a color temperature or a light source type of ambient light based on a color image having the highest luminance among a plurality of color images acquired from the second imaging unit during a shooting preparation period. the image processing method according to any one of claim 9 or al 13.   15. The second estimating step extracts a face area from a color image acquired from the second imaging unit, and estimates a color temperature or a light source type of ambient light based on the image of the face area. The image processing method according to any one of the above.