JP7114648B2 - Image processing device, image processing method, imaging device, program and recording medium - Google Patents

Description

The present invention relates to an image processing apparatus that synthesizes a plurality of images with different focus positions.

2. Description of the Related Art Conventionally, a so-called focus stacking technique is known, in which a plurality of images with different focus positions are combined to generate a composite image in which the entire imaging area is in focus. Focus stacking technology is used as a means to solve the problem that only a part of the subject can be focused due to insufficient depth of field when capturing an image with an imaging device such as a digital camera. .

For example, in Patent Document 1, a plurality of images with different focus positions are captured, only the region with the highest contrast value is extracted from each image, combined into one image, and the entire imaging region is focused. A technique for generating a composite image is disclosed.

JP-A-11-261797

However, when the focus stacking method described above is used, the contrast is erroneously detected when capturing an image of a glossy subject whose part has a high reflectance, and an image that is originally out of focus is synthesized. may be lost. For example, in an out-of-focus image, the glossy part of the subject may appear as a ball blur. If the contour of the out-of-focus ball is erroneously determined to have the highest contrast value, another image, which is not the focused image that should have been originally selected, is selected as a composition target.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to obtain a suitable composite image by suppressing the synthesis of an out-of-focus image, even for a subject having a glossy area, in a process of picking up images with different focus positions and combining the images. It is to provide an image processing apparatus capable of

In order to solve the above problems, the present invention acquires a gloss component for each of a plurality of images having different focus positions in the optical axis direction, using a polarization image corresponding to each of the plurality of images. and synthesizing means for generating a synthetic map for the plurality of images using the plurality of images from which the gloss component has been removed, and for generating a synthetic image, wherein the synthetic image is the plurality of images. To provide an image processing device with a deep depth of field.

According to the present invention, in the process of capturing images with different focus positions and synthesizing the images, a glossy area can be recognized as an out-of-focus area, and the use of the out-of-focus area for synthesis is suppressed. It is possible to provide an image processing device capable of obtaining a depth stacking image.

1 is a block diagram showing a configuration example of an imaging device according to an embodiment of the present invention; FIG. FIG. 4 shows different configurations of polarizers in embodiments of the present invention; FIG. 2 is a schematic diagram of an imaging device in which a patterned polarizer is arranged according to an embodiment of the present invention; 4 is a flowchart for explaining image composition processing according to the embodiment of the present invention; 4 is a flowchart for explaining imaging processing according to the embodiment of the present invention; 4 is a flowchart for explaining polarization imaging processing according to the embodiment of the present invention; 5 is a flow chart describing generation processing of a non-polarized image in the embodiment of the present invention. FIG. 4 is a relational diagram between the azimuth angle and the luminance value in the embodiment of the present invention; 5 is a flow chart describing alignment processing in the embodiment of the present invention. 4 is a flow chart describing generation of a composite image in the embodiment of the present invention;

An image processing apparatus according to an embodiment of the present invention will be described in detail below with reference to the drawings. In the following description, an example in which the image processing means of the present invention is applied to an imaging device will be used, but the present invention is not limited to this.

FIG. 1 is a block diagram for explaining the configuration of an imaging device 1000 according to this embodiment.

The imaging lens 101 has a plurality of lenses.

The shutter 102 is, for example, a mechanical shutter having a diaphragm function. In addition to the mechanical shutter of the shutter 102, it is possible to control the accumulation time as an electronic shutter by controlling the reset timing of the imaging element 105, and it can be used for moving image imaging.

The imaging device 103 converts the optical image into an electrical signal.

An A/D converter 104 converts an analog signal output from the image sensor 103 into a digital signal.

A timing generation circuit 105 supplies clock signals and control signals to the image sensor 103 and A/D converter 104 and is controlled by a memory control circuit 106 and a system control circuit 107 .

The image processing circuit 108 performs predetermined pixel interpolation processing and color conversion processing on the data from the A/D converter 104 or the data from the memory control circuit 106 . Further, the image processing circuit of the image processing circuit 108 implements an electronic zoom function by clipping an image and performing variable magnification processing. Further, in the image processing circuit 108, the captured image data is used to perform predetermined arithmetic processing. I do. Examples of control performed by the image processing circuit 108 include TTL AF processing, AE processing, and EF processing. Further, in the image processing circuit 108, predetermined arithmetic processing is performed using captured image data, and TTL AWB (auto white balance) processing is also performed based on the obtained arithmetic processing result.

The memory control circuit 106 controls the A/D converter 104 , timing generation circuit 105 , image processing circuit 108 , memory 109 and compression/decompression circuit 112 . Data from the A/D converter 104 is written to the memory 109 via the image processing circuit 108 and the memory control circuit 106 or directly via the memory control circuit 106 .

The image display unit 110 is composed of, for example, a TFT-LCD or the like, and image data for display written in the memory 109 is displayed by the image display unit 110 via the memory control circuit 106 . An electronic finder function can be realized by sequentially displaying captured image data using the image display unit 110 . In addition, the image display unit 110 can arbitrarily turn on/off the display according to an instruction from the system control circuit 107. When the display is turned off, the power consumption of the imaging apparatus 1000 can be greatly reduced. I can.

A memory 109 is a memory for storing captured still images and moving images, and has a storage capacity sufficient to store a predetermined number of still images and moving images for a predetermined time. The memory 109 makes it possible to write a large amount of images to the memory 109 at high speed even in the case of continuous shooting or panorama shooting in which a plurality of still images are continuously captured.

The memory 109 can also be used as a work area for the system control circuit 107 .

A non-volatile memory 111 is a non-volatile memory configured by a Flash ROM or the like. Program codes to be executed by the system control circuit 107 are written in the nonvolatile memory 111, and the program codes are executed while being sequentially read out. In addition, the nonvolatile memory 111 is provided with an area for storing system information and an area for storing user setting information, so that various information and settings can be read and restored at the next startup.

A compression/expansion circuit 112 compresses and expands image data by adaptive discrete cosine transform (ADCT) or the like. write to

A compression/decompression circuit 112 compresses and decompresses image data by adaptive discrete cosine transform (ADCT) or the like. Write to 109.

The exposure control means 113 controls the shutter 102 having a diaphragm function and also has a flash dimming function by interlocking with the flash 112 .

A distance control unit 114 controls focusing of the imaging lens 101 , and a zoom control unit 115 controls zooming of the imaging lens 101 .

The flash 116 also has an AF auxiliary light projection function and a flash dimming function. The exposure control means 113 and the distance measurement control means 114 are controlled using the TTL method. It controls the control means 114 .

A control circuit 107 controls the entire imaging apparatus 1000 .

Operation means 118, 119, 120, 121, 122, and 123 are operation means for inputting various operation instructions for the system control circuit 107, and include switches, dials, touch panels, pointing by line-of-sight detection, voice recognition devices, and the like. Composed of singular or plural combinations.

Next, the operating means 118, 119, 120, 121, 122 and 123 will be specifically described.

The operation unit 118 is a mode dial switch, and can switch and set each function mode such as power off, automatic imaging mode, imaging mode, HDR imaging mode, panorama imaging mode, moving image imaging mode, playback mode, PC connection mode, and the like.

The operating means 119 is a shutter switch SW1, which is turned ON during operation of the shutter button, and instructs the start of operations such as AF (autofocus) processing, AE (automatic exposure) processing, AEB (automatic white balance) processing, and the like.

The operating means 120 is a shutter switch SW2, which is turned ON when the operation of the shutter button is completed. In the case of flash photography, after performing EF (flash pre-emission) processing, the image sensor 103 is exposed for the exposure time determined by AE processing. At the same time as the end of the exposure, the exposure control unit 113 shields the light, thereby ending the exposure of the image sensor 103 . reading processing of writing image data in the memory 109 via the A/D converter 104 and the memory control circuit 106 from the signal read out from the image sensor 103; The image data is read out from the memory 109 , compressed by the compression/decompression circuit 112 , and then the image data is written to the recording medium 129 .

The operation means 121 is a display switching switch, and can switch the display of the image display section 110 . With this function, power can be saved by cutting off the current supply to the image display unit such as a TFT-LCD when taking an image using the optical viewfinder 104 .

An operation unit 122 is an operation unit including various buttons, a touch panel, a rotary dial, and the like, and includes a menu button, a set button, a macro button, a multi-screen playback page break button, a flash setting button, a single-shot/continuous-shot/self-timer switching button, and the like. There is Menu movement + (plus) button, menu movement - (minus) button, playback image movement + (plus) button, playback image - (minus) button, image quality selection button, exposure compensation button, date/time setting button, etc. be.

The operating means 123 is a zoom switch section as zoom operating means for the user to instruct to change the magnification of the captured image. Hereinafter, it will also be referred to as a zoom switch 123 . The zoom switch 123 is composed of a tele switch for changing the imaging angle of view to the telephoto side and a wide switch for changing the angle of view to the wide angle side. By using the zoom switch 123, the zoom control means 115 is instructed to change the imaging angle of view of the imaging lens 101, thereby triggering an optical zoom operation. It also serves as a trigger for image clipping by the image processing circuit 108 and for electronic zooming change of the imaging angle of view by pixel interpolation processing or the like.

The power supply means 117 is a power supply means comprising a primary battery such as an alkaline battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li ion battery, an AC adapter, or the like.

An interface 124 is an interface with a recording medium such as a memory card or a hard disk, and a connector 125 is a connector for connecting with a recording medium such as a memory card or a hard disk.

With the optical viewfinder 126 , it is possible to take an image using only the optical viewfinder without using the electronic viewfinder function of the image display unit 110 .

The communication means 127 has various communication functions such as USB, IEEE1394, LAN, and wireless communication.

A connector 128 is a connector that connects the imaging device 1000 to other equipment via the communication means 127 . An antenna may be used instead of the connector 128 for wireless communication.

A recording medium 129 is a memory card, a hard disk, or the like. A connector 130 is a connector for connecting with the imaging apparatus 1000, an interface 131 is an interface with the imaging apparatus 1000, and a recording unit 132 is a recording unit for a recording medium such as a memory card or hard disk.

Next, a method for acquiring a polarization image according to this embodiment will be described. Here, the polarization image is an image obtained by acquiring luminance information of only some of the polarization components of the reflected light from the object.

When the light incident on the subject from the light source is directly reflected (specularly reflected) on the subject and the linearly polarized light component is captured as it is, the surface of the subject in the captured image becomes glossy or "shiny". On the other hand, by inserting a polarizing plate in the optical system, it is possible to block the linearly polarized light component and obtain a polarized image in which gloss and "shininess" are removed or reduced.

As a method for obtaining a polarized image, a polarizer may be inserted before or after the imaging optical system or in the imaging optical system, and an image may be captured while changing the principal axis angle direction of the polarizer.

FIG. 2 is a diagram for explaining an example of an imaging optical system in this embodiment. FIG. 2 shows a digital camera body 40, a lens 30 with a common mount, and an adapter 20 with a built-in polarizer. The polarizer may be detachable from or built into the imaging device 1000 .

Also, as a method of acquiring a polarized image, a method of placing a patterned polarizer in front of the imaging device is also conceivable. FIG. 3 is a diagram for explaining the arrangement of the imaging element and the patterned polarizer in this embodiment. Polarizers 1101, 1102, 1103, and 1104 shown in FIG. 3 are different types of polarizers, for example, polarizers with transmission axes that differ by 45 degrees. Four types of polarizers are arranged as one set, and by approximating that the reflected light from the same point on the subject is observed in the neighboring pixels, images are captured while changing the principal axis angle direction of the polarizers. It is possible to obtain polarization information similar to that of . Needless to say, the image sensor 103 may acquire luminance information including all polarization components.

Next, focus stacking processing will be described.

FIG. 4 is a flow chart showing processing for generating a composite image according to this embodiment. The image generation processing of this embodiment is executed by the system control circuit 107 and the image processing circuit 108 according to a processing program for causing the computer to function as a processing device. Note that the processing program may be recorded in a computer-readable recording medium, for example.

In step S401, the image sensor 103 captures a plurality of images with different focus positions. In step S402, the image pickup device 103 uses the polarizer or the like described above to pick up a plurality of images with different focus positions in polarized light. In step S403, the image processing circuit 108 generates a non-polarized image from the multiple images obtained in step S102. In step S404, the system control circuit 107 aligns the multiple images captured by the image sensor 103 in step S401 and the multiple unpolarized images generated in step S403. In step S405, the image processing circuit 108 performs image synthesis on the aligned images to generate a synthesized image. Each step will be described in detail below.

FIG. 5 is a flowchart for explaining imaging in step S401 in this embodiment.

In step S501, the system control circuit 107 sets the focus position. For example, the user designates a focus position through the touch panel of the operation unit 122, and designates a plurality of focus positions at equal intervals before and after the focus position corresponding to the designated focus position in the optical axis direction. At the same time, the system control circuit 107 determines the imaging order in order of distance at the set focus position.

In step S502, the image sensor 103 captures an image at the focus position that is the first in the imaging order among the focus positions set in step S501 that have not yet been imaged.

In step S503, the system control circuit 107 determines whether or not imaging has been performed at all the focus positions set in step S501. If the image has been captured at all the focus positions, the processing in the flowchart shown in FIG. 5 is terminated.

Further, when the imaging apparatus 1000 is a multi-lens camera having a plurality of imaging lenses 101, mechanical shutters 102, and imaging elements 103, it is possible to simultaneously capture images at a plurality of focus positions set in step S201. Determination and discrimination of the order to be performed are unnecessary.

FIG. 6 is a flowchart for explaining polarization imaging in step S402 in this embodiment.

In step S601, the image sensor 103 performs polarization imaging at the focus position that is the first in the imaging order among the focus positions set in step S501 that have not yet been imaged. In this embodiment, the system control circuit 107 controls the imaging unit configured by the imaging lens 101, the mechanical shutter 102, and the imaging device 103 to obtain a polarized image of the subject under a plurality of conditions corresponding to mutually different polarization components. input image). The polarizer may be moved manually by the user or controlled by the system control circuit 107 . For example, when using the image pickup device shown in FIG. 3, the image pickup device 103 performs polarization imaging four times at each of the focus positions set in step S501 to obtain four polarization images having different polarization components.

In step S602, the system control circuit 107 determines whether polarization imaging has been performed at all the focus positions set in step S501. When polarization imaging has been performed at all the focus positions, the processing in the flowchart shown in FIG. 6 is terminated.

FIG. 7 is a flowchart for explaining generation of a non-polarized image in step S403 in this embodiment.

In step S701, the image processing circuit 108 acquires the polarization information of the subject from the plurality of acquired polarization images. The image processing circuit 108 acquires polarization information of the captured image from the polarization image acquired in step S102. A method for obtaining polarization information will be described below with reference to the drawings.

FIG. 8 is a graph for explaining an example of the relationship between the azimuth angle (polarization angle) φ and the luminance value (light intensity) I(φ) in this embodiment. The azimuth angle is the angle formed by the polarizing axis direction of the polarizing element with respect to an arbitrarily determined reference axis. The white circles in FIG. 8 indicate the brightness values I(0), I(45), and I(90) of one predetermined pixel of the captured images with the polarization directions of 0 degrees, 45 degrees, and 90 degrees. Also, the solid line curve shows the change in the luminance value I(φ) with respect to the azimuth angle φ calculated by fitting from the three measured values. Let I _MAX be the maximum luminance value among the calculated luminance values I, I _MIN be the minimum luminance value, and α (maximum azimuth angle) be the azimuth angle (maximum azimuth angle) of the maximum luminance value I _MAX . (φ) is represented by the following equation (1).

I(φ)=I _MAX −I _MIN )·cos ² (θ−α)+I _MIN Equation (1)
Therefore, by measuring three luminance values, polarization information (maximum luminance value I _MAX , minimum luminance value I _MIN , maximum azimuth angle α) is calculated for each pixel, and luminance value I(φ) with respect to azimuth angle φ. You can ask for change.

In this embodiment, the polarization information acquiring unit acquires the polarization information from the luminance values of the three polarized captured images, but the present invention is not limited to this. Polarization information may be calculated by fitting luminance values of four or more captured images using Equation (1). In that case, for example, the method of least squares or the like can be used. I _MAX −I _MIN is an angle-dependent component, that is, a luminance component (first component) that changes according to the polarization angle formed by the polarization direction with respect to the reference axis, and I _MIN is an angle-independent component that does not depend on the polarization angle (no polarization component).

In this embodiment, the first component and the non-polarized light component are regarded as the specular reflection component and the diffuse reflection component from the object, respectively. This is because the specular reflection component is generated by Fresnel reflection and contains more s-polarization components than p-polarization components, so it exhibits polarization angle dependence, while the diffuse reflection component loses its angular dependence during the scattering process inside the object. This is because there is a tendency not to exhibit angle dependence because the

Here, the polarization information is obtained using three or more polarization captured images, but as a simpler method, the polarization information may be obtained from two polarization captured images. Unlike the case where three or more polarized images are used, I _MAX −I _MIN cannot be calculated, but by taking the absolute value of the difference between two polarized images with different polarization directions, it is can be obtained. Since this luminance difference component is the difference between specific polarization components, it is part of the angle dependent component and has a value smaller than I _MAX -I _MIN . However, the amount still reflects the specular reflection component, which may be used as the first component.

In step S702, an image reflecting only the non-polarized component I _MIN is generated as a non-polarized image.

FIG. 9 is a flowchart for explaining alignment in step S104 in this embodiment.

In step S901, the system control circuit 107 acquires a reference image for alignment from each of the image captured by the image sensor 103 in step S502 and the non-polarized image generated in step S403. Assume that the reference image for alignment has the earliest imaging order. Alternatively, since the angle of view slightly changes between captured images by capturing images while changing the focus position, the angle of view may be the narrowest among the captured images.

In step S902, the system control circuit 107 acquires a target image for alignment processing. It is assumed that the target image is an image other than the reference image acquired in step S901 and has not undergone alignment processing. If the system control circuit 107 takes the one with the earliest imaging order as the reference image, the system control circuit 107 may sequentially acquire the target images in the imaging order.

In step S903, the system control circuit 107 calculates the positional deviation amount between the reference image and the target image. An example of the calculation method is described below. First, the system control circuit 107 sets a plurality of blocks in the reference image. System control circuit 107 preferably sets the size of each block to be the same. Next, the system control circuit 107 sets a search range wider than the blocks of the reference image at the same positions of the respective blocks of the reference image in the target image. Finally, the system control circuit 107 calculates a corresponding point in each search range of the target image that has the minimum sum of absolute differences in luminance (Sum of Absolute Difference, hereinafter referred to as SAD) from the block of the reference image. do. The system control circuit 107 calculates the positional deviation in step S503 as a vector from the center of the block of the reference image and the corresponding point described above. In calculating the corresponding points described above, the system control circuit 107 calculates the sum of squared differences (hereinafter referred to as SSD), normalized cross correlation (hereinafter referred to as NCC), etc. in addition to the SAD. may be used.

In step S904, the system control circuit 107 calculates a transform coefficient from the amount of positional deviation between the reference image and the target image. The system control circuit 107 uses, for example, a projective transform coefficient as the transform coefficient. However, the transform coefficients are not limited to projective transform coefficients, and affine transform coefficients or simplified transform coefficients with only horizontal and vertical shifts may be used.

In step S905, the image processing circuit 108 transforms the target image using the transform coefficients calculated in step S904.

For example, system control circuit 107 can perform the transformation using the equation shown in equation (2).

式（２）では、（ｘ´、ｙ´）は変形を行った後の座標を示し、（ｘ、ｙ）は変形を行う前の座標を示す。行列ＡはステップＳ９０４でシステム制御回路１０７が算出した変形係数を示す。

In equation (2), (x', y') indicate coordinates after deformation, and (x, y) indicate coordinates before deformation. Matrix A indicates the deformation coefficients calculated by the system control circuit 107 in step S904.

In step S906, system control circuit 107 determines whether alignment has been performed for all images other than the reference image. If all the images other than the reference image have been aligned, the processing in this flow chart ends, and if there is an unprocessed image, the process returns to step S902.

Further, when aligning a plurality of images captured by the above-described multi-lens camera, the amount of parallax caused by the difference in the position of the optical system 103 can be obtained by calculating the amount of deviation in step S903. Alignment can be done.

FIG. 10 is a flowchart for explaining image composition in step S405 in this embodiment.

At step 1001, the image processing circuit 108 calculates a contrast value for the non-polarized image (including the reference image) after alignment. As an example of a method of calculating the contrast value, for example, first, the image processing circuit 108 calculates the luminance Y from the color signals Sr, Sg, and Sb of each pixel using Equation (3) below.

Y=0.299S _r +0.587S _g +0.114S _b Expression (3)
Next, a contrast value I is calculated using a Sobel filter on the matrix L of luminance Y of 3×3 pixels, as shown in the following equations (4) to (6).

また、上述のコントラスト値の計算方法は一例にすぎず、たとえば、使用するフィルタをラプラシアンフィルタ等のエッジ検出フィルタや所定の帯域を通過するバンドパスフィルタを用いることも可能である。

Further, the method of calculating the contrast value described above is merely an example. For example, it is also possible to use an edge detection filter such as a Laplacian filter or a bandpass filter that passes a predetermined band as the filter to be used.

In step S1002, image processing circuitry 108 generates a composite map. As a method of generating a synthesis map, the image processing circuit 108 compares the contrast values of pixels at the same position in each image and calculates a synthesis ratio according to the magnitude of the contrast values.

An example of a specific calculation method is shown below.

A synthetic map Am(x, y) is generated using the contrast value Cm(x, y) calculated in step S1001. Note that m is the m-th image among a plurality of images with different focus positions, x is the horizontal coordinate of the image, and y is the vertical coordinate. A synthetic map is calculated by a weighted average based on contrast values obtained for each pixel. Assuming that M images with different focus positions are taken, the composite map Am(x, y) for the m-th image is expressed by Equation (7).

A _m (x, y)=max _k=1 C _k (x, y) Expression (7)
However, in step S1002, it is necessary to appropriately adjust the composition ratio so that the boundary does not look unnatural. As a result, the composition ratio of the composition map in one image is not binarized between 0% and 100%, but continuously changes.

In step S1003, the image processing circuit 108 generates a synthesized image O(x, y) by synthesizing a plurality of images captured in step S401 based on the synthesized map calculated in step S602. Assuming that the focused image is Im(x, y), the image is generated by the following equation (9) using the composite map generated by the equation (7).

以上で、ステップＳ４０５の画像合成処理を終了する。

Thus, the image synthesizing process in step S405 ends.

As described above, according to the present invention, it is possible to calculate a composite ratio from an input image obtained by polarization imaging and excluding a specular reflection component. As a result, in the focus stacking processing of a glossy subject, it is possible to obtain a suitable focus stacking image in which the out-of-focus area is suppressed from being used for stacking.

(Other embodiments)
In addition, the present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a recording medium, and one or more processors in a computer of the system or device executes the program. can also be realized by a process of reading and operating the It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

The above-described embodiments merely show specific examples for carrying out the present invention, and the technical scope of the present invention should not be construed to be limited by these. That is, the present invention can be embodied in various forms without departing from its technical spirit or main features.

1000 imaging device

Claims (17)

Acquisition means for acquiring a gloss component for each of a plurality of images with different focus positions in the optical axis direction, using a polarization image corresponding to each of the plurality of images;
synthesizing means for generating a synthetic map for the plurality of images using the plurality of images from which the gloss component has been removed to generate a synthetic image;
The image processing apparatus, wherein the composite image has a deeper depth of field than the plurality of images. 2. The image processing apparatus according to claim 1, wherein said acquisition means acquires said gloss component using a polarization component of said polarization image. 3. The image processing apparatus according to claim 1, wherein said synthesizing means generates said synthetic map based on the contrast of said plurality of images from which said gloss component has been removed. Acquisition means for acquiring a polarization component for each of a plurality of images with different focus positions in the optical axis direction, using a polarization image corresponding to each of the plurality of images;
synthesizing means for generating a synthetic map for the plurality of images using the plurality of images excluding the angle -dependent component to generate a synthetic image;
The image processing apparatus, wherein the composite image has a deeper depth of field than the plurality of images. 5. The image processing apparatus according to claim 4, wherein the polarization images corresponding to each of the plurality of images are captured at the same focal position as the images. 6. The image processing apparatus according to claim 4, wherein a plurality of said polarization images correspond to each of said plurality of images. 7. The image processing apparatus according to claim 6, wherein the three polarized images correspond to each of the plurality of images. 8. The image processing apparatus according to claim 6, wherein said plurality of polarized images corresponding to one image among said plurality of images have mutually different polarization angles. 8. The image processing apparatus according to claim 2, wherein said polarization component is a luminance component that changes with respect to a polarization angle. 10. The image processing apparatus according to any one of claims 4 to 9, wherein said synthesizing means generates said synthesized map based on contrasts of said plurality of images excluding said polarization components. obtaining a gloss component for each of a plurality of images with different focus positions in the optical axis direction, using a polarization image corresponding to each of the plurality of images;
a synthesizing step of synthesizing the plurality of images from which the gloss component has been removed to generate a synthetic image;
The image processing method, wherein the composite image has a deeper depth of field than the plurality of images. Acquisition step of acquiring a polarization component for each of a plurality of images with different focus positions in the optical axis direction, using a polarization image corresponding to each of the plurality of images;
a synthesizing step of synthesizing the plurality of images excluding the polarization components to generate a synthesized image;
The image processing method, wherein the composite image has a deeper depth of field than the plurality of images. imaging means for capturing a plurality of images with different focus positions in the optical axis direction;
obtaining means for obtaining a gloss component for each of the plurality of images using a polarization image corresponding to each of the plurality of images;
synthesizing means for synthesizing the plurality of images from which the gloss component has been removed to generate a synthetic image;
The imaging apparatus, wherein the composite image has a deeper depth of field than the plurality of images. imaging means for capturing a plurality of images with different focus positions in the optical axis direction;
obtaining means for obtaining a polarization component for each of the plurality of images using a polarization image corresponding to each of the plurality of images, and combining the plurality of images excluding the polarization component; and synthesizing means for generating a synthetic image,
The imaging apparatus, wherein the composite image has a deeper depth of field than the plurality of images. A computer program to be operated by a computer of an image processing device,
obtaining a gloss component for each of a plurality of images with different focus positions in the optical axis direction, using a polarization image corresponding to each of the plurality of images;
a synthesizing step of synthesizing the plurality of images from which the gloss component has been removed to generate a synthetic image;
A program, wherein the composite image has a deeper depth of field than the plurality of images. A computer program to be operated by a computer of an image processing device,
Acquisition step of acquiring a polarization component for each of a plurality of images with different focus positions in the optical axis direction, using a polarization image corresponding to each of the plurality of images;
performing a synthesizing step of synthesizing the plurality of images excluding the polarization components to generate a synthesized image;
The image processing method, wherein the composite image has a deeper depth of field than the plurality of images. A computer-readable recording medium recording the program according to claim 15 or 16.

Images