JP6961423B2 - Image processing equipment, imaging equipment, control methods for image processing equipment, programs and recording media - Google Patents

Description

The present invention relates to an image processing device that synthesizes a plurality of images having different focus positions.

When shooting multiple subjects whose distances from imagers such as digital cameras are significantly different from each other, or when shooting a subject that is long in the depth direction, the depth of field is insufficient, so focus is only on a part of the subject. It may not be possible to match. In order to solve this problem, in Patent Document 1, a plurality of images having different focus positions are captured, only the focusing region is extracted from each image, combined into one image, and the entire imaging region is focused. A so-called depth-of-field composition technique for generating a composite image is disclosed. Therefore, in Patent Document 1, among the regions at the same position of each image, the region having the highest contrast value is defined as the focusing region.

Japanese Unexamined Patent Publication No. 2015-216532

However, when the method of depth composition as described above is used, a blurred region may appear at the edge portion of the composite image.

FIG. 10 is a diagram for explaining an example of performing imaging for depth synthesis. In the case shown in FIG. 10, the imaging device captures five images 1010 to 1050 of an X-shaped subject in front of a white background while changing the focus position, and synthesizes the captured images. Is performed to generate the composite image 1000. In images 1010 to 1050, it is assumed that the pixels 1011 to 1051 are in the same position and the pixels 1012 to 1052 are in the same position. When a composite image is created using the technique disclosed in Patent Document 1, the pixel 1011 to 1051 having the highest contrast value is selected and used at the same position in the composite image. Therefore, it is illustrated that the image pickup apparatus uses the pixel 1031 for the composite image 1000. However, if the image pickup apparatus selects the pixel 1012 to 1052 having the highest contrast value and uses it at the same position in the composite image in the same manner, the pixel 1012 or 1052 is selected and used in the composite image. As a result, as shown in the figure, the composite image 1000 is blurred around the edge portion. This is a problem common to all image processing devices having a function of acquiring a plurality of images captured by changing the focus position and synthesizing the plurality of images, not limited to the image pickup device.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an image processing apparatus capable of reducing blur included in an image synthesized by using a plurality of images having different focus positions. ..

In order to solve the above problems, the present invention relates to an acquisition means for acquiring contrast values from each of a plurality of regions of an image, and for a plurality of images having at least a part of the angle of view overlapping and different focus positions. Each region has a compositing means that sets a compositing ratio according to the contrast value acquired by the acquisition means and synthesizes the plurality of images based on the set compositing ratio to generate a compositing image. The synthesis means is such that the amount of change in the contrast value between the first regions of the plurality of images is equal to or less than a predetermined first threshold value, and the first region When there is a second region in the vicinity of the image in which the amount of change in the contrast value between the regions of the plurality of images is larger than the second threshold value, the composition ratio of the first region is set to the above. Provided is an image processing apparatus characterized in that the composition ratio is set to be equal to the composition ratio of the second region.

According to the configuration of the present invention, it is possible to provide an image processing apparatus capable of reducing blur included in an image obtained by capturing and synthesizing a plurality of images having different focus positions.

It is a block diagram which shows the structure of the digital camera which concerns on embodiment of this invention. It is a flowchart for demonstrating the generation of the composite image in embodiment of this invention. It is a flowchart for demonstrating the imaging in Embodiment of this invention. It is a flowchart for demonstrating alignment in Embodiment of this invention. It is a flowchart for demonstrating composition of an image in Embodiment of this invention. It is a flowchart for demonstrating the correction of the synthetic map in 1st Embodiment. It is a figure for demonstrating the amount of change of the contrast value in embodiment of this invention. It is a figure for demonstrating the point spread function in embodiment of this invention. It is a flowchart for demonstrating the correction of the synthetic map in 2nd Embodiment. It is a figure for demonstrating an example of performing an image pickup for depth synthesis.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

(First Embodiment)
FIG. 1 is a block diagram showing a structure of a digital camera as an image processing device according to the present embodiment. The digital camera 100 can capture a still image, record information on the in-focus position, calculate a contrast value, and synthesize an image. Further, the digital camera 100 can perform enlargement processing or reduction processing on an image captured and saved or an image input from the outside.

The control unit 101 is, for example, a signal processor such as a CPU or an MPU, and controls each part of the digital camera 100 while reading a program built in the ROM 105 described later in advance. For example, as will be described later, the control unit 101 issues a command to the imaging unit 104, which will be described later, regarding the start and end of imaging. Alternatively, an image processing command is issued to the image processing unit 107, which will be described later, based on the program built in the ROM 105. The command from the user is input to the digital camera 100 by the operation unit 110 described later, and reaches each part of the digital camera 100 through the control unit 101.

The drive unit 102 is composed of a motor or the like, and mechanically operates the optical system 103, which will be described later, under the command of the control unit 101. For example, based on the command of the control unit 101, the drive unit 102 moves the position of the focus lens included in the optical system 103 to adjust the focal length of the optical system 103.

The optical system 103 includes a zoom lens, a focus lens, an aperture, and the like. The aperture is a mechanism that adjusts the amount of transmitted light. The focusing position can be changed by changing the position of the lens.

The imaging unit 104 is a photoelectric conversion element, and performs photoelectric conversion that converts an incident optical signal into an electric signal. For example, a CCD sensor, a CMOS sensor, or the like can be applied to the image pickup unit 104. The imaging unit 104 is provided with a moving image imaging mode, and can capture a plurality of images that are continuous in time as each frame of the moving image.

The ROM 105 is a read-only non-volatile memory as a recording medium, and stores, in addition to the operation program of each block included in the digital camera 100, parameters and the like necessary for the operation of each block. The RAM 106 is a rewritable volatile memory, and is used as a temporary storage area for data output in the operation of each block included in the digital camera 100.

The image processing unit 107 performs various image processing such as white balance adjustment, color interpolation, and filtering on the image output from the image pickup unit 104 or the image signal data recorded in the built-in memory 109 described later. .. Further, the image signal data captured by the imaging unit 104 is compressed according to a standard such as JPEG.

The image processing unit 107 is composed of an integrated circuit (ASIC) that collects circuits that perform specific processing. Alternatively, the control unit 101 may perform processing according to the program read from the ROM 105 so that the control unit 101 also has a part or all of the functions of the image processing unit 107. When the control unit 101 also has all the functions of the image processing unit 107, it is not necessary to have the image processing unit 107 as hardware.

The display unit 108 is a liquid crystal display, an organic EL display, or the like for displaying an image temporarily stored in the RAM 106, an image stored in the built-in memory 109 described later, a setting screen of the digital camera 100, or the like. be.

The built-in memory 109 is a place for recording an image captured by the imaging unit 104, an image processed by the image processing unit 107, information on the focusing position at the time of image imaging, and the like. A memory card or the like may be used instead of the built-in memory.

The operation unit 110 is, for example, a button, a switch, a key, a mode dial, or the like attached to the digital camera 100, or a touch panel that is also used as the display unit 108. The command from the user reaches the control unit 101 via the operation unit 110.

FIG. 2 is a flowchart for explaining the generation of the composite image in the present embodiment. In step S201, the imaging unit 104 captures a plurality of images having different focus positions. In step S202, the control unit 101 aligns the plurality of images captured by the imaging unit 104 in step S201. In step S203, the image processing unit 107 synthesizes the image after the alignment is performed, and generates a composite image. Each step will be described in detail below.

FIG. 3 is a flowchart for explaining the imaging in step S201 in the present embodiment.

In step S301, the control unit 101 sets the focus position. For example, the user specifies the focusing position through the touch panel that is also used by the display unit 108, and specifies a plurality of focusing positions at equal intervals before and after the focus position corresponding to the focusing position in the optical axis direction. At the same time, the control unit 101 determines the imaging order in the order of distance at the set focus position.

In step S302, the imaging unit 104 takes an image at the focus position in which the imaging order is the earliest among the unimpressed focus positions set in step S301.

In step S303, the control unit 101 determines whether or not imaging has been performed at all the focus positions set in step S301. When all the focus positions are imaged, the process in this flowchart is finished, and if there is a focus position that has not been imaged yet, the process returns to step S302.

FIG. 4 is a flowchart for explaining the alignment in step S202 in the present embodiment.

In step S401, the control unit 101 acquires a reference image for alignment from the images captured by the imaging unit 104 in step S201. The reference image for alignment shall have the earliest imaging order. Alternatively, by taking an image while changing the focus position, the angle of view changes slightly between the captured images, so that the angle of view may be the narrowest among the captured images.

In step S402, the control unit 101 acquires the target image for the alignment process. It is assumed that the target image is an image other than the reference image acquired in step S401 and the alignment process has not been completed. If the image having the earliest imaging order is used as the reference image, the control unit 101 may acquire the target images in the order of imaging.

In step S403, the control unit 101 calculates the amount of displacement between the reference image and the target image. An example of the calculation method will be described below. First, the control unit 101 sets a plurality of blocks in the reference image. The control unit 101 is preferably set so that the size of each block is the same. Next, the control unit 101 sets a search range of the target image at the same position as each block of the reference image, in a range wider than the block of the reference image. Finally, the control unit 101 calculates a corresponding point in each search range of the target image that minimizes the sum of the absolute differences in brightness from the block of the reference image (Su of Absolute Difference, hereinafter referred to as SAD). .. The control unit 101 calculates the deviation of the position in step S403 as a vector from the center of the block of the reference image and the corresponding point described above. In addition to SAD, the control unit 101 uses Sum of Squared Difference (hereinafter referred to as SSD), normalized cross-correlation (hereinafter referred to as NCC), and the like in the calculation of the corresponding points described above. You may.

In step S404, the control unit 101 calculates the conversion coefficient from the amount of positional deviation between the reference image and the target image. The control unit 101 uses, for example, a projective conversion coefficient as the conversion coefficient. However, the conversion coefficient is not limited to the projection conversion coefficient, and a simplified conversion coefficient of only the affine transformation coefficient or the horizontal / vertical shift may be used.

In step S405, the image processing unit 107 converts the target image using the conversion coefficient calculated in step S404.

For example, the control unit 101 can be transformed by using the equation shown in (Equation 1).

In (Equation 1), (x', y') indicates the coordinates after the transformation, and (x, y) indicates the coordinates before the transformation. The matrix A shows the deformation coefficient calculated by the control unit 101 in step S404.

In step S406, the control unit 101 determines whether or not all the images other than the reference image have been aligned. When the alignment is performed on all the images other than the reference image, the process in this flowchart is finished, and if there is an image that has not been processed yet, the process returns to step S402.

FIG. 5 is a flowchart for explaining the composition of images in step S203 in the present embodiment.

In step 501, the image processing unit 107 calculates the contrast value for each image (including the reference image) after the alignment is performed. As an example of the method of calculating the contrast value, for example, first, the image processing unit 107 calculates the luminance Y from the color signals Sr, Sg, and Sb of each pixel using the following (Equation 2).
Y = 0.299Sr + 0.587Sg + 0.114Sb ... (Equation 2)

Next, the contrast value I is calculated using the Sobel filter in the matrix L of the brightness Y of the 3 × 3 pixels as shown in the following (Equation 3) to (Equation 5).

Further, the above-mentioned method of calculating the contrast value is only an example, and for example, an edge detection filter such as a Laplacian filter or a bandpass filter that passes through a predetermined band can be used as the filter to be used.

In step S502, the image processing unit 107 generates a composite map. As a method of generating a composite map, the image processing unit 107 compares the contrast values of the pixels at the same position in each image, sets the composite ratio of the pixel with the highest contrast value to 100%, and sets another one at the same position. The pixel composition ratio is 0%. The image processing unit 107 sets such a composition ratio for all positions of the image.

In step S503, the control unit 101 corrects the composite map generated by the image processing unit 107 in step S502. The specific correction method will be described later.

In step S504, the image processing unit 107 replaces the pixels according to the composite map corrected by the control unit 101 in step S503, and generates a composite image. When the composition ratio changes from 0% to 100% (or changes from 100% to 0%) between adjacent pixels with respect to the composition ratio calculated in this way, the unnaturalness at the composition boundary becomes conspicuous. become. Therefore, a filter having a predetermined number of pixels (tap number) is applied to the composite map so that the composite ratio does not change suddenly between adjacent pixels.

Hereinafter, the correction of the composite map in step S503 will be described in detail.

FIG. 6 is a flowchart for explaining the correction of the composite map in step S503 in the present embodiment. In step S601, the control unit 101 selects the pixel of interest from the unprocessed pixels.

Next, in step S602, the control unit 101 calculates the amount of change in the contrast value of the pixel of interest. The difference between the maximum value and the minimum value of the contrast value among the pixels at the same position as the pixel of interest in each image (including the reference image) after the alignment of the amount of change in the contrast value referred to here. Point to.

In step S603, the control unit 101 compares the amount of change in the contrast value calculated in step S602 with a predetermined threshold value, and if it is equal to or less than the threshold value, proceeds to step S604 to change the composition ratio of the pixel of interest. On the other hand, if it is not equal to or less than the threshold value, the process proceeds to step S605. Here, the control unit 101 has determined that the pixels in which the amount of change in the contrast value is equal to or less than the threshold value are likely to have edge blur of the subject as in pixel 1001 of FIG.

In step S604, the control unit 101 changes the composition ratio of the pixel of interest. Here, the control unit 101 changes the composition ratio of the pixel of interest with reference to a pixel in the vicinity of which the amount of change in the contrast value is larger than the threshold value. For example, the control unit 101 sets the composition ratio of the pixel of interest to the composition ratio of the pixel in which the amount of change in the contrast value in the vicinity thereof is larger than the threshold value. Alternatively, the composition ratio of the closest pixel among the pixels in which the amount of change in the contrast value in the vicinity is larger than the threshold value is set. The threshold value in step S604 may be the same value as the threshold value used for comparison in step S603, or may be a value larger than the threshold value used for comparison in step S603. If there is no pixel in the vicinity where the amount of change in the contrast value is larger than the threshold value, there is no edge in the vicinity of the pixel, and it is unlikely that the pixel has edge blur, and the composition ratio is not corrected. The range of "neighborhood" referred to here will be described later. FIG. 8 is for explaining the amount of change in the contrast value in the present embodiment. FIG. 7A shows the relationship between the contrast values of the pixels 711 to 715 at the same position of a plurality of images and the focus position at the time of image imaging. The contrast value when an image with the pixel 713 is captured is the maximum, and it is determined that the pixel is in focus at this focus position, and this focus position is also referred to as the best focus. Here, the difference between the contrast value of the pixel 513 having the largest contrast value and the contrast value of the pixel 715 having the smallest contrast value, that is, the amount of change in the contrast value described above is calculated and assumed to be larger than the threshold value described above.

On the other hand, FIG. 7B shows the relationship between the contrast value and the focus position of the pixels 721 to 725 at the same position in the vicinity of the pixels 711 to 715. In FIG. 7B, the contrast value of the pixel 725 is the highest, but the difference between it and the contrast value of the pixel 723 having the lowest contrast value, that is, the amount of change in the contrast value is set to be equal to or less than the threshold value. When the amount of change in the contrast value is less than or equal to the threshold value, as described above, the control unit may be affected by the blurring of the edges of the surrounding subject because the pixel is located in the area of the subject having a small contrast value. Assuming that there is a property, the composition ratio is changed. As a method of changing, the control unit 101 determines that the focus position that is the best focus of the pixel 711 in FIG. 7 (a) is the best focus in the pixel 723 of FIG. 7 (b). Then, the control unit 101 sets the composition ratio of the pixel 723 at the focus position to 100%, and sets the composition ratio of the pixels at the same position of other images including the pixel 725 to 0%. For example, in FIG. 10, when the image processing unit 107 creates the composite image 1000, the pixel 1031 is used for the composite image instead of the pixel 1051 in FIG. 10 to prevent edge blurring. Become.

In step S605, the control unit 101 determines whether or not all the pixels have been processed. When all the pixels have been processed, this flow ends, and if there are pixels that have not been processed yet, the process returns to step S601.

As described above, the above processing is based on the premise that the pixels shown in FIG. 7A are in the vicinity of the pixels shown in FIG. 7B. The determination of "neighborhood" here can be performed using a point spread function (hereinafter referred to as PSF).

FIG. 8 is a diagram for explaining PSF in this embodiment. In FIG. 8, the horizontal axis represents the pixel coordinates and the vertical axis represents the contrast value. Although the actual PSF has a two-dimensional spread, it is represented in one dimension in FIG. 8 for the sake of brevity. FIG. 8A shows a PSF when a point image is formed at the best focus position using an ideal lens. In FIG. 8B, the image formation in the defocused state (the state where the position of the imaging surface is separated from the best focus position) in which the focus position is moved becomes as shown by the curve 820, and the image spreads (blurred spreads) pixels. The number N is as shown at distance 821. In FIG. 8C, when the defocus amount is further increased, the image formation becomes a curve 830, and the number of pixels N at which the image spreads becomes a distance 831.

Since the defocus amount (change amount of the focus position) can be calculated from the setting in step S301, it is possible to calculate how much the image spreads according to the PSF. It is preferable to set the "neighborhood" of the pixel of interest described above to be within this number of pixels N. That is, if the amount of change in the contrast value of the pixel of interest is equal to or less than the threshold value, the control unit 101 confirms whether or not there is a pixel in which the amount of change in the contrast value is larger than the threshold value in the range set based on the PSF calculated in advance. .. If it is in the vicinity of the pixel of interest, the control unit 101 changes the composition ratio described above.

It should be noted that the above-mentioned method of determining the neighborhood is only an example, and various changes can be made. For example, after making a judgment based on the number of pixels N obtained by spreading the PSF, the judgment is made using the distance information of the subject. That is, unless the difference between the subject distances of the pixels shown in FIGS. 7 (a) and 7 (b) is equal to or less than a predetermined amount, the control unit 101 does not determine that the two pixels are in the "neighborhood". The distance information can be calculated by a known technique, and as an example, it can be calculated from a stereo image acquired by an image sensor having a plurality of photoelectric conversion units in one pixel.

Further, it is also possible to consider the characteristics of the change in the contrast value among the pixels determined to be "nearby". For example, the pixel shown in FIG. 7A has a characteristic that the contrast value is maximum at the best focus position and the contrast value decreases when the focus position changes. On the other hand, the edge-blurred pixel has a characteristic that the contrast value is the minimum at the best focus position in FIG. 7 (a) and the contrast value increases when the focus position changes. Therefore, the composition ratio of the pixel of interest may be corrected by using only the pixel having the characteristic of changing the contrast value opposite to that of the pixel of interest.

According to the first embodiment, the edge blur can be reduced by calculating the amount of change in the contrast value of the pixels and changing the composition ratio of the pixels below the threshold value.

(Second Embodiment)
In the first embodiment, when the amount of change in the contrast value of the pixel of interest is equal to or less than the threshold value, the composition ratio of the pixel of interest is changed, but in the second embodiment, the amount of change in the contrast value of the pixel of interest is When it is larger than the threshold value, the composition ratio of the peripheral pixels is changed. Hereinafter, the second embodiment will be described focusing on the points different from those of the first embodiment. The same parts as in the first embodiment will be omitted.

FIG. 9 is a flowchart for explaining the correction of the composite map in step S503 in the present embodiment. In step S901, the control unit 101 selects the pixel of interest from the unprocessed pixels.

In step S902, the control unit 101 calculates the amount of change in the contrast value of the pixel of interest. The calculation method is the same as in step S602 of the first embodiment.

In step S903, the control unit 101 proceeds to step S904 if the amount of change in the contrast value calculated in step S902 is equal to or greater than a predetermined threshold value, and proceeds to step S905 if it is smaller than the threshold value. The "threshold value" referred to here is different from the "threshold value" in the description of the correction of the composite map of the first embodiment.

In step S904, the control unit 101 refers to the composition ratio of the pixel of interest and changes the composition ratio of the neighboring pixels. For example, the control unit 101 sets the composition ratio of the pixels in the vicinity of the pixel of interest whose contrast value change amount is equal to or less than the threshold value to the same composition ratio as the pixel of interest. Here, if there is another pixel whose contrast value change amount is equal to or greater than a predetermined threshold value at a position closer to the pixel of interest with respect to a pixel in the vicinity thereof, priority is given to the composition ratio based on that pixel. You may try to do it. Further, if there is no pixel in the vicinity of the pixel of interest whose contrast value change amount is equal to or less than the threshold value, the composition ratio is not changed in the range in the vicinity. The method of determining the "neighborhood" referred to here may be the same as the method described in step S604 of the first embodiment.

In step S905, the control unit 101 determines whether or not all the pixels have been processed. When all the pixels have been processed, this flow ends, and if there are pixels that have not been processed yet, the process returns to step S901.

In step S903, when the contrast value of the pixel of interest is equal to or greater than the threshold value, there is a high possibility that the edge blur will spread by changing the focus position, and if the composition ratio is simply determined by the maximum value of the contrast value in the neighboring pixels, the edge blurred image will be obtained. May be output. Therefore, when a pixel with a contrast value equal to or higher than the threshold value is detected, if there is a pixel in the vicinity where the amount of change in the contrast value that is easily affected by edge blur is small, the composition ratio of that pixel is changed to the same composition ratio as the pixel of interest. To do.

According to the second embodiment, the edge blur can be reduced by calculating the amount of change in the contrast value of the pixels and changing the composition ratio of the neighboring pixels of the pixels above the threshold value.

(Other embodiments)
Although the above embodiments have been described based on the implementation with a digital camera, the above embodiment is not limited to the digital camera. For example, it may be carried out by a portable device having a built-in image sensor, or may be a network camera capable of capturing an image.

In the present invention, a program that realizes one or more functions of the above-described embodiment is supplied to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device program. It can also be realized by the process of reading and operating. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 Digital camera 101 Control unit 102 Drive unit 103 Optical system 104 Imaging unit 105 ROM
106 RAM
107 Image processing unit 108 Display unit 109 Built-in memory 110 Operation unit

Claims (16)

An acquisition method for acquiring contrast values from each of a plurality of areas of an image,
For a plurality of images in which at least a part of the angles of view overlap and the focus positions are different, a composition ratio is set for each area according to the contrast value acquired by the acquisition means, and the composition ratio is based on the set composition ratio. It has a compositing means for generating a composite image by compositing the plurality of images.
The combining means, the amount of change in the contrast value between each of the first region of the plurality of images is equal to or less than the first threshold value predetermined and, in the vicinity of the first region, said plurality When there is a second region in which the amount of change in the contrast value between the respective regions of the image is larger than the second threshold value, the composite ratio of the first region is used as the composition ratio of the second region. An image processing device characterized in that it is set to be equal to the composition ratio. In the synthesis means, the amount of change in the contrast value in the first region of the plurality of regions is equal to or less than a predetermined first threshold value, and the contrast value is located in the vicinity of the first region. Claim 1 is characterized in that when there is no region in which the amount of change is larger than the second threshold value, the synthesis ratio of the first region is set based on the contrast value of the first region. The image processing apparatus according to. An acquisition method for acquiring contrast values from each of a plurality of areas of an image,
For a plurality of images in which at least a part of the angles of view overlap and the focus positions are different, a composition ratio is set for each area according to the contrast value acquired by the acquisition means, and the composition ratio is based on the set composition ratio. It has a compositing means for generating a composite image by compositing the plurality of images.
The combining means, the amount of change in the contrast value between each of the first region of the plurality of images is equal to or less than the first threshold value predetermined and, in the vicinity of the first region, said plurality When there is a second region in which the amount of change in the contrast value between the regions of the image is larger than the second threshold value, the composite ratio of the first region is used as the composition ratio of the second region. The contrast value is set based on the contrast value, and the amount of change in the contrast value in the first region of the plurality of regions is equal to or less than a predetermined first threshold value and is located in the vicinity of the first region. When there is no region in which the amount of change in the contrast value is larger than the second threshold value, the synthesis ratio of the first region is set based on the contrast value of the first region. Image processing device. When the amount of change in the contrast value of the third region among the plurality of regions is larger than the first threshold value, the synthesis means sets the synthesis ratio of the third region to the third region. The image processing apparatus according to any one of claims 1 to 3, wherein the image processing apparatus is set based on the contrast value of the above. The image processing apparatus according to any one of claims 1 to 4 , wherein the second threshold value is the same value as the first threshold value or a value larger than the first threshold value. The image processing apparatus according to any one of claims 1 to 5 , wherein the region is a region composed of one pixel. Any one of claims 1 to 6 , wherein the amount of change in the contrast value in the region is the difference between the maximum value and the minimum value of the contrast value in the plurality of image regions corresponding to the region. The image processing apparatus according to the section. The image processing apparatus according to any one of claims 1 to 7 , wherein the region of the plurality of images corresponding to the region is a region at the same position in each of the images. The image processing apparatus according to any one of claims 1 to 8 , wherein the synthesis means defines a range of the vicinity using a point spread function. The image processing apparatus according to any one of claims 1 to 9 , wherein the synthesizing means defines a range of the vicinity using distance information of a subject. The image processing apparatus according to any one of claims 1 to 10 , wherein the synthesizing means defines a range of the vicinity using the change characteristic of the contrast value. The image processing apparatus according to any one of claims 1 to 11 , wherein the focus positions are set at equal intervals in the optical axis direction. It has an imaging means for capturing an image,
An acquisition means for acquiring a contrast value from each of a plurality of regions of the image, and
For a plurality of images in which at least a part of the angles of view overlap and the focus positions are different, a composition ratio is set for each area according to the contrast value acquired by the acquisition means, and the composition ratio is based on the set composition ratio. It has a compositing means for generating a composite image by compositing the plurality of images.
The combining means, the amount of change in the contrast value between each of the first region of the plurality of images is equal to or less than the first threshold value predetermined and, in the vicinity of the first region, said plurality When there is a second region in which the amount of change in the contrast value between the respective regions of the image is larger than the second threshold value, the composite ratio of the first region is used as the composition ratio of the second region. An image processing device characterized in that it is set to be equal to the composition ratio. An acquisition step to acquire contrast values from each of multiple areas of the image,
For a plurality of images in which at least a part of the angles of view overlap and the focus positions are different, a composition ratio is set according to the contrast value for each region, and the composition ratio of the plurality of images is set based on the set composition ratio. It has a compositing step to perform compositing and generate a compositing image,
In the above synthesis step, the is a plurality of first threshold value or less amount of change in the contrast value between each of the first region of the image is predetermined, and, in the vicinity of the first region, the When there is a second region in which the amount of change in the contrast value between the respective regions of the plurality of images is larger than the second threshold value, the composite ratio of the first region is set to that of the second region. A control method for an image processing apparatus, characterized in that the composition ratio is set to be equal to the composition ratio. A computer program that runs on the computer of an image processing device.
An acquisition step to acquire contrast values from each of multiple areas of the image,
For a plurality of images in which at least a part of the angles of view overlap and the focus positions are different, a composition ratio according to the contrast value is set for each region, and the composition ratio of the plurality of images is set based on the set composition ratio. Perform the compositing step to generate a compositing image by compositing,
In the above synthesis step, the is a plurality of first threshold value or less amount of change in the contrast value between each of the first region of the image is predetermined, and, in the vicinity of the first region, the When there is a second region in which the amount of change in the contrast value between the respective regions of the plurality of images is larger than the second threshold value, the composite ratio of the first region is set to that of the second region. A program characterized in that it is set to be equal to the synthesis ratio. A computer-readable recording medium on which the program according to claim 15 is recorded.

Images