JP2016099836A - Imaging device, image processor, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To eliminate a level difference in image characteristics while maintaining the original image quality of a composite image.SOLUTION: An image processor includes acquisition means for acquiring a plurality of images having different image characteristics from each other, generation means for generating a composite image by compositing the plurality of images acquired by the acquisition means, and processing means for performing correction processing of pixel values of the composite image. The processing means uses a correction amount corresponding to a distance from a composition boundary of the composite image, about a pixel to be an object of pixel value correction to perform the correction processing.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a technique for improving the image quality of an image obtained by combining a plurality of images.

  There is an image composition technique that realizes angle-of-view control after image capturing, HDR, and the like by combining a plurality of image data captured by a plurality of image capturing units. However, when the imaging conditions of each imaging unit are different, image characteristics such as noise and resolution are different among a plurality of imaging data, so that there is a problem that a difference in image characteristics appears clearly at the synthesis boundary in the synthesized image. there were.

  As a technique for eliminating the noise level difference at the composite boundary in the composite image, there is a technique of aligning the noise amount of a plurality of image data by switching the noise reduction strength for each image according to the gain value at the time of imaging (patent) Reference 1).

JP 2010-154478 A

  However, when the technique described in Patent Document 1 is used, if the noise level difference that appears in the composite image data is large, excessive noise reduction is performed on an image area having a large amount of noise, and the image quality of the composite image decreases. May end up. Accordingly, an object of the present invention is to eliminate the difference in image characteristics while maintaining the image quality of the composite image as much as possible.

  In order to solve the above-described problems, an image processing apparatus according to the present invention has a common image area corresponding to the same area of a subject with at least one image characteristic among noise characteristics, resolution, brightness, and color balance being different from each other Acquisition means for acquiring a plurality of images, and generation means for generating a composite image by aligning and synthesizing the plurality of images acquired by the acquisition means so that the common image regions overlap. And processing means for correcting pixel values of the composite image based on differences in the image characteristics of the plurality of images, the processing means being used for generating pixel values in the composite image For at least one of the two regions sandwiching the composite boundary that is the boundary at which the image switches, the pixel is continuously changed up to a pixel of a predetermined distance according to the distance from the composite boundary. Using the correction amount, and performs the correction processing.

  According to the present invention, it is possible to eliminate a difference in image characteristics while maintaining the quality of a composite image.

1 is a diagram illustrating an example of a configuration of an image processing apparatus. FIG. 3 is a diagram illustrating a configuration of an image processing unit according to the first embodiment. 3 is a flowchart illustrating a flow of processing performed by the image processing apparatus according to the first embodiment. The figure which shows the outline | summary of synthetic | combination image data. The figure which shows the ideal model of noise amount change. The figure which shows panorama composition. FIG. 3 is a diagram illustrating an application example of the first embodiment. The figure which shows the relationship between pixel pitch, a focal distance, and object distance. FIG. 10 is a diagram illustrating an example of resolution change in the second embodiment. FIG. 10 is a diagram illustrating an application example of the second embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Example 1>
In the present embodiment, an example will be described in which a noise level difference occurring at the synthesis boundary when image data corresponding to image regions in different ranges is synthesized is eliminated. FIG. 1A is a diagram illustrating a configuration of an image processing apparatus 100 (hereinafter referred to as a processing apparatus 100) according to the present embodiment. The processing apparatus 100 includes imaging units 101 to 105, a CPU 106, a RAM 107, a ROM 108, an operation unit 109, a display control unit 110, and a display unit 111. The processing apparatus 100 also includes an imaging unit control unit 112, a digital signal processing unit 113, an encoder unit 114, an external memory control unit 115, an image processing unit 117, and a bus 118.

  The imaging units 101 to 105 are cameras each having an independent imaging optical system and an imaging sensor, and are arranged on the same plane of the casing of the processing apparatus 100 as shown in FIG. The imaging units 101 to 104 have telephoto lenses having the same focal length, and the imaging unit 105 has a wide-angle lens having a shorter focal length than the imaging units 101 to 104. In addition, the number of pixels of the imaging elements included in the imaging units 101 to 105 is equal to each other. The imaging units 101 to 105 capture an image corresponding to the operation of the operation unit 109 and output the acquired image data to each component of the processing apparatus 100 via the bus 118. Note that the configuration of the imaging unit is not limited to the above, and the number of imaging units, the position and orientation of the imaging unit may be changed, and the focal lengths of all the lenses may be the same or different. . In addition, the processing apparatus according to the present embodiment is not limited to a multi-view camera provided with a plurality of cameras in one casing, and may take the form of a computer connected to a plurality of cameras. Further, an example in which image processing is performed on image data obtained by taking a plurality of images while changing the viewpoint in the form of a camera having only one imaging unit is also included in the processing device in this embodiment.

  The CPU 106 is a processing circuit that comprehensively controls each unit described below. A RAM 107 is a memory that functions as a main memory of the CPU 106 and temporarily stores data such as captured image data. The ROM 108 is a memory that stores a control program executed by the CPU 106. The operation unit 109 is a button for the user to perform an input operation on the processing apparatus 100, and can input an image capturing instruction and the like. The operation unit 109 includes a mode dial and a cross key in addition to the shutter button. The display control unit 110 is a processing circuit that performs display control of images and characters displayed on the display unit 111. The imaging unit control unit 112 is a processing circuit that controls the imaging units 101 to 105 based on an instruction from the CPU 106 or the operation unit 109.

  The digital signal processing unit 113 is a processing circuit that performs various processes such as white balance processing, gamma processing, and noise reduction processing on digital data received via the bus 118. The encoder unit 114 is a processing circuit that performs processing for converting image data into a file format such as JPEG or MPEG. The external memory control unit 115 is an interface that connects the processing apparatus 100 and a PC or other media 116 (for example, a hard disk, a memory card, a CF card, an SD card, a USB memory). The processing apparatus 100 can output data to the PC or other media 116 via the external memory control unit 115.

  The image processing unit 117 is a processing circuit that performs image processing on the image data acquired by the imaging units 101 to 105 and the image data output from the digital signal processing unit 113. In the processing apparatus 100, the CPU 106 executes the program stored in the ROM 108 using the RAM 107 as a work memory, thereby controlling the image processing unit 117 as each block shown in FIG. 2 and executes the following processing.

  Hereinafter, the flow of processing performed in the processing apparatus 100 of the present embodiment will be described with reference to the block diagram of FIG. 2 and the flowchart of FIG.

  In step S301, the acquisition unit 201 acquires an imaging data group captured by the imaging units 101 to 105. Here, the imaging data group refers to image data captured by each imaging unit, focal length of the lens of each imaging unit, imaging parameters such as F value and exposure time at the time of imaging, and each imaging unit stored in the RAM 107. Calibration parameters including position and orientation. It is assumed that the calibration parameters here are obtained by performing calibration in advance. The acquisition unit 201 outputs the acquired imaging data group to the synthesis unit 202.

In step S <b> 302, the composition unit 202 generates composite image data using the imaging data group input from the acquisition unit 201. In this embodiment, by combining the image data of the imaging units 101 to 105 included in the imaging data group, from the imaging data of the imaging unit 105 at the same viewpoint as the lens of the imaging unit 105 from the same viewpoint as the imaging unit 105. An example of generating high-quality composite image data is shown. Here, the focal length of the lenses of the imaging units 101 to 104 is f _T , and the focal length of the lens of the imaging unit 105 is f _W. In the combining unit 202 first, from the image data of the image pickup unit 101 - 104, by the viewpoint interpolation processing using the calibration parameters, viewpoint position, and orientation is the same as the imaging unit 105, four focal length image data of f _T Generate. An existing method can be used for the viewpoint interpolation processing. For example, the image data of the imaging units 101 to 104 is generated by shifting the image data of the imaging units 101 to 104 to the viewpoint of the imaging unit 105 based on the distance of the subject in each pixel determined by performing block matching between the image data of the imaging units 101 to 104. it can.

Next, the combining unit 202, by averaging the pixel values of the four image data generated, to generate combined image data T having a field angle corresponding to the focal length f _T. Then, the composition unit 202, by combining the image data of the imaged focal length f _W of synthetic image data T and the imaging unit 105, generates the synthetic image data W having a field angle corresponding to the focal length f _W To do. Since both the image data have the same viewpoint position and optical axis direction, they can be combined by enlarging or reducing according to the focal length ratio. An example of the composite image data W is shown in FIG. The image shown in FIG. 4A is the composite image data W, and a region surrounded by a gray line at the center is a region corresponding to the composite image data T. The gray line indicates the composite boundary. In the present embodiment, the composite image data T and the image data captured by the image capturing unit 105 are combined with the composite image data T inserted into the image data captured by the image capturing unit 105 (the pixel value of the composite image data T). This is performed by replacing the pixel value of the image data of the imaging unit 105). That is, in this process, the composite image data T is combined with the image data captured by the image capturing unit 105 so that the image regions corresponding to the same subject region overlap each other. The combining method is not limited to this, and a form in which the combined image data T and the pixel value of the image data captured by the imaging unit 105 are added and averaged in the combining region may be used. When the composite image data T is inserted, it is possible to obtain composite image data W having a higher resolution at the central portion of the image. However, in the synthesis method in which pixel values are added and averaged, the central portion of the composite image data W and Differences in image quality at the periphery are alleviated, and noise at the center of the image is also reduced.

  The composition unit 202 outputs the composite image data W generated here to the determination unit 203 and the correction unit 204, and outputs the image data of the imaging unit 105 and the composite image data T used for composition to the determination unit 203. Note that the image composition processing performed here is not limited to the above. For example, the process of the present embodiment can also be applied to a case where an image captured by the image capturing unit 105 and an image captured by any of the image capturing units 101 to 104 are combined.

In step S <b> 303, the determination unit 203 acquires a noise characteristic at the composite boundary of the composite image data W as the image characteristic of the composite image data W input from the composite unit 202. The noise level difference will be described with reference to FIG. FIG. 4B is a diagram showing the composite image data W in the same manner as FIG. An area 402 is an area corresponding to the composite image data T, an area 403 is an area corresponding to the image data captured by the imaging unit 105, and an area 404 is an area near the composite boundary. In the region 404, the left side of the composite boundary is a region obtained by averaging a plurality of images, and thus is a region with less noise than the right region having only one image information. . Therefore, a difference in the amount of noise occurs on the left and right of the synthesis boundary, which appears as a noise step in the image. When the signal component of the image is S and the noise component is N, the actual measurement value U acquired by the imaging sensor can be expressed as in Expression (1). At this time, the standard deviation of the noise component N is σ.
U = S + N (1)
Further, the value U ′ obtained when K times of signals are acquired under the same conditions and the addition average is taken can be expressed by Expression (2).

  According to the central limit theorem, when random numbers generated from the distribution of standard deviation σ are added K times, the standard deviation of the distribution of values after addition is

Therefore, the standard deviation of the noise component N ′ after the averaging is obtained from the equation (3).

  In other words, by adding K times and taking the average, the standard deviation of the noise component is

It turns out that it becomes. Accordingly, assuming that the noise standard deviations of the imaging units 101 to 105 are all the same σ, in this embodiment, noise steps as shown in FIG. Become. In this embodiment, since four images are averaged in the region 402, the standard deviation of the noise in the region on the left side of the region 404 is

The standard deviation of noise in the right region is σ.

  The determination unit 203 acquires the noise characteristics corresponding to the regions 402 and 403 from the pixel values in the composite image data W. In the present embodiment, the determination unit 203 acquires the standard deviation of the pixel values of the solid part of the region 402 and the region 403 (the region where the subject colors are substantially the same) as the noise standard deviation. Although a well-known technique can be widely used as a method for extracting a solid part, in this embodiment, edge extraction processing is performed on the composite image data W, and an area from which no edge is extracted is extracted as a solid part. To do. In addition, the acquisition method of a noise characteristic is not restricted to this, You may acquire a standard deviation with the same method from the pixel value of the captured image data before a synthesis | combination. If the relationship between the camera imaging parameters and the noise intensity is known in advance, it may be calculated from the F value at the time of imaging, lens characteristics, ISO sensitivity, etc. without using the pixel values of the image data. .

  In step S304, the determination unit 203 determines the amount of change in the pixel value in the synthesis region for eliminating the noise level difference based on the noise characteristics acquired in step S303. Here, FIG. 5A shows an ideal model of the noise amount change in the x-axis direction in the region 404 in this embodiment. In this embodiment, the noise characteristic at the synthesis boundary is added to the pixel on the left side of the synthesis boundary with a small amount of noise by adding a noise amount corresponding to the distance d from the synthesis boundary to the pixel to be corrected. Is changed as shown in FIG. 5A to eliminate the noise step. In this embodiment, the area inside the pixel with the distance D from the boundary is set as the noise amount changing area, and the pixel value is corrected so that the noise amount continuously increases as the distance from the boundary is approached. When the noise amount is changed in this way, the noise level difference becomes inconspicuous in the corrected image. The noise amount difference σ ′ between the region 402 and the region 403 shown in FIG. 5A can be calculated by Expression (4) based on the additiveness of dispersion.

  Further, the amount of additional noise in a pixel whose distance from the synthesis boundary is d [pixel] can be obtained by Expression (5).

  The distance D from the boundary indicating the range in which the amount of noise is changed can be implemented with, for example, 30 pixels, but the value of D is not limited thereto. In this embodiment, a certain value stored in the ROM 108 is used as the value of D, but it may be changed according to the ratio of the angle of view of each image. For example, when the angle of view of the telephoto image is smaller than a predetermined threshold, it is possible to prevent noise from being added to a wide range of the telephoto image by narrowing the range in which the amount of noise is changed. In this embodiment, the linear model shown in FIG. 5A is used as an ideal model for noise amount change, but the shape of the ideal model for noise amount change is not limited to this. For example, by changing to a smooth curve system as shown in FIG. 5B, it is possible to smoothly correct the noise step while suppressing the total amount of added noise as compared with the linear ideal model. The determination unit 203 outputs the correction amount of the noise amount determined here to the correction unit 204.

  In step S305, the correction unit 204 performs a noise amount correction process on the composite image data W based on the correction amount input from the determination unit 203, and outputs the generated corrected image data to the bus 118 for processing. finish. Known techniques can be widely used as a noise addition technique. In this embodiment, the correction unit 204 corrects the pixel value of each pixel of the composite image data W by adding additive white Gaussian noise using a standard deviation of σd according to the distance d from the composite boundary. Generate image data. In this embodiment, the corrected composite image data is generated by adding white Gaussian noise with an average of 0 and standard deviation σd. However, if the noise amount of the corrected composite image data conforms to the ideal change model of the noise amount, a noise addition method is used. Is not limited to this.

  The above is the process performed by the processing apparatus 100 of the present embodiment. According to the above processing, the noise level difference at the synthesis boundary can be eliminated without performing noise reduction over a wide range in the image, and image quality deterioration due to noise reduction can be suppressed. Note that the present embodiment is not limited to the case where images having different angles of view are combined, and for example, as illustrated in FIG. In the example of FIG. 6, the region 601 surrounded by the dotted line is a composite region, and a noise step occurs at the boundary with other regions. Therefore, the noise step can be corrected by applying this embodiment.

  An example in which this embodiment is applied to an actual image is shown in FIG. FIG. 7A is an image obtained by joining two types of images having different standard deviations σ (noise amount), and a noise level difference can be confirmed at the center of the image. FIG. 7B is a result image obtained by applying the present embodiment to the image of FIG. It can be seen that the change in the amount of noise is smooth.

  In this embodiment, the acquisition unit 201 functions as an acquisition unit that acquires a plurality of images having different image characteristics. The synthesizing unit 202 functions as a generating unit that generates a synthesized image by synthesizing a plurality of images acquired by the acquiring unit. The correction unit 204 functions as a processing unit that performs a correction process on the pixel value of the composite image. The determination unit 203 functions as a determination unit that determines the correction amount of the pixel value in the correction process.

<Example 2>
In the first embodiment, the embodiment in which the composite image data is corrected so that the noise amount smoothly changes at the composite boundary of the corrected image data has been described. In this embodiment, an embodiment will be described in which, when images having different angles of view are synthesized, the synthesized image data is corrected so that the resolution changes smoothly at the synthesis boundary of the corrected image data. In addition, description is abbreviate | omitted about the part which is common in Example 1, and only the difference with Example 1 is demonstrated.

  First, the difference in resolution between image data captured by imaging units having different focal lengths will be described. FIG. 8 shows a relationship between the pixel pitch x of the sensor, the focal length f of the lens, and the subject distance d. Here, when the resolution in this embodiment is defined as how many pixels the length a on the subject surface is on the sensor surface, the resolution R can be calculated by Expression (6).

Therefore, assuming that the sensor size and the distance to the subject are the same between the imaging units, it can be seen that the resolution is determined by the focal length of the lens. Therefore in the same manner as in Example 1, when synthesizing the image data of: focal length f _T and the focal length f _W, resulting in the resolution step is generated at the synthesis boundary. The present embodiment relates to processing for solving this problem.

In the present embodiment, a case where the combining unit 202 fits the combined image data T into the center of the imaging unit 105 as in the first embodiment will be described. The composite image data W is shown in FIG. 4B as in the first embodiment. Since the focal length corresponding to the region 402 is f _T and the focal length corresponding to the region 403 is f _W , a resolution step as shown in FIG. In the present embodiment, the correction unit 204 eliminates this resolution step by performing correction for degrading the resolution as shown in FIG. 9B on the right side of the synthesis area.

  Hereinafter, details of processing performed in the processing apparatus 100 of the present embodiment will be described. The configuration and processing flow of the processing apparatus 100 of this embodiment are basically the same as those of the first embodiment, but the details of the processes performed in steps S303 to S305 are different from those of the first embodiment. The difference will be described.

  In step S <b> 303 of this embodiment, the determination unit 203 acquires the difference in resolution between the area 402 and the area 403 as the image characteristics of the composite image data W. Since the resolution can be calculated from the imaging parameters of the imaging units 101 to 105, the determination unit 203 substitutes the imaging parameter indicated by the imaging data group input from the acquisition unit 201 into Equation 6 to obtain the difference, thereby obtaining the difference in resolution. calculate.

  In step S304 of this embodiment, the determination unit 203 determines an ideal change model for how the resolution should change in the vicinity of the synthesis boundary. Then, the amount of additional blur to satisfy that is determined. In this embodiment, the resolution of the image data is smoothly changed by applying the Gaussian filter represented by Expression (7) to the region in the region 403 near the synthesis boundary. Here, x and y are pixel positions in the image. Also, the resolution of the image data

The resolution

The standard deviation σ ′ used for the deterioration can be calculated by equation (8).

  Furthermore, the amount of additional blur in the region where the distance from the synthesis boundary is d can be obtained by Expression (9).

Note that the filter for adding blur is not limited to the Gaussian filter, and various blur filters can be used. The determination unit 203 outputs the additional blur amount σ _d determined here to the correction unit 204.

In step 305, the correction unit 204 performs processing using a Gaussian filter on the composite image data W based on the additional blur amount σ _d determined in step S 304, and outputs the corrected image data to the bus 118 for processing. finish.

  The above is the process performed by the processing apparatus 100 of the present embodiment. According to the above processing, it is possible to alleviate the resolution step that occurs at the synthesis boundary when the image data having different resolutions are synthesized. An example in which the present embodiment is applied to an actual image is shown in FIG. FIG. 10A is an image obtained by joining two types of images having different resolutions, and a resolution step can be confirmed at the center of the image. FIG. 10B is a result image obtained by applying the present embodiment to the image of FIG. It can be seen that the resolution change is smooth.

<Other examples>
Embodiments of the present invention are not limited to those listed in the above examples. For example, when images having different noise characteristics are synthesized, unlike the first embodiment, a denoising process for reducing noise with an intensity corresponding to the distance from the synthesis boundary is performed on a region with a larger amount of noise. May be. In addition, when images with different resolutions are combined, unlike the second embodiment, a sharpening filter corresponding to the distance from the combination boundary may be applied to a region with a lower resolution. Further, a process for changing the amount of noise and a process for changing the resolution may be performed on both sides of the synthesis boundary. That is, a process for correcting the pixel value may be performed for at least one of the two regions sandwiching the composite boundary. Further, the processing of changing both the noise amount and the resolution may be performed by combining the first and second embodiments. The image having different image characteristics is not limited to an image having different noise characteristics and resolution, and the present invention can be applied to an example using images having different brightness and color balance.

  In the above-described embodiments, the process of correcting the pixel value is performed on the composite image data W. However, the order in which the pixel value is corrected is not limited to the above. For example, the composite image data W and the image data captured by the imaging unit 105 may be preliminarily subjected to pixel value correction processing and then combined to generate the composite image data W.

  The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Claims (15)

Acquisition means for acquiring a plurality of images having a common image area corresponding to the same area of the subject, wherein at least one image characteristic among noise characteristics, resolution, brightness, and color balance is different from each other;
Generating means for generating a composite image by aligning and synthesizing the plurality of images acquired by the acquisition means so that the common image areas overlap;
Processing means for performing correction processing of pixel values of the composite image based on the difference in the image characteristics of the plurality of images;
The processing means includes a pixel having a predetermined distance according to a distance from the composite boundary, regarding at least one of two regions sandwiching a composite boundary, which is a boundary at which an image used for generating a pixel value in the composite image is switched. The image processing apparatus is characterized in that the correction processing is performed using a correction amount that continuously changes until.   The processing unit corrects the pixel value of the composite image so that the image characteristics continuously change in two regions sandwiching the composite boundary in the composite image. The image processing apparatus described. Acquisition means for acquiring a plurality of images having a common image area corresponding to the same area of the subject, wherein at least one image characteristic among noise characteristics, resolution, brightness, and color balance is different from each other;
Processing means for performing correction processing on pixel values of the plurality of images based on the difference in the image characteristics of the plurality of images;
Generating means for generating a composite image by aligning and synthesizing the plurality of images whose pixel values have been corrected by the processing means so that the common image areas overlap;
The processing unit is configured to process the composite image with respect to a region in the plurality of images corresponding to at least one of two regions sandwiching a composite boundary that is a boundary at which an image used for generating a pixel value in the composite image is switched. An image processing apparatus, wherein the correction processing is performed using a correction amount that continuously changes to a pixel having a predetermined distance according to a distance from the synthesis boundary.   The processing means corrects pixel values of the plurality of images so that the image characteristics continuously change in two regions sandwiching the composite boundary in the composite image. An image processing apparatus according to 1.   The said processing means makes the said correction amount large, so that the distance from the said synthetic | combination boundary of the pixel used as the object of the said pixel value correction | amendment is small. Image processing device. A determination unit for determining a correction amount of the pixel value in the correction process;
The image characteristic is a noise characteristic including a noise amount in an image,
The image processing apparatus according to claim 1, wherein the determination unit determines the correction amount based on a noise characteristic of each of the plurality of images.   The correction process performed by the processing unit includes a process of adding an amount of noise corresponding to a distance from the synthesis boundary to an image having a small noise amount among the plurality of images, and a noise amount of the plurality of images. The image processing apparatus according to claim 6, wherein the image processing apparatus is at least one of processing for reducing an amount of noise corresponding to a distance from the synthesis boundary from an image having a large size.   The determining means extracts solid portions of the plurality of images, acquires a standard deviation of pixel values of the extracted solid portions as the noise characteristics, and determines the correction amount using the acquired noise characteristics. The image processing apparatus according to claim 6, wherein: A determination unit for determining a correction amount of the pixel value in the correction process;
The image characteristic is the resolution of the image;
The image processing apparatus according to claim 1, wherein the determination unit determines the correction amount based on a resolution of each of the plurality of images.   The correction process performed by the processing means includes a process of applying a sharpening filter having an intensity corresponding to a distance from the synthesis boundary to an image having a low resolution among the plurality of images, The image processing apparatus according to claim 9, wherein the image processing apparatus is at least one of processes for applying a blur filter having an intensity corresponding to a distance from the synthesis boundary to an image having a high resolution.   The determination unit acquires resolutions of the plurality of images based on at least one of a focal length corresponding to the plurality of images, a pixel pitch of an imaging sensor, and a subject distance, and the correction processing is performed based on the acquired resolutions The image processing apparatus according to claim 9 or 10, wherein:   An image pickup apparatus comprising the image processing apparatus according to claim 1, and comprising a plurality of image pickup units that pick up the plurality of images and have different characteristics.   A program that causes a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 11. An acquisition step of acquiring a plurality of images having a common image area corresponding to the same area of the subject, wherein at least one image characteristic of noise characteristics, resolution, brightness, and color balance is different from each other;
A generation step of generating a composite image by aligning and synthesizing the plurality of images acquired in the acquisition step so that the common image region overlaps;
A process for correcting pixel values of the composite image based on the difference in the image characteristics of the plurality of images,
In the processing step, for at least one of two regions sandwiching a composite boundary, which is a boundary at which an image used for generating a pixel value in the composite image is switched, a pixel having a predetermined distance according to the distance from the composite boundary An image processing method characterized in that the correction processing is performed using a correction amount that continuously changes until An acquisition step of acquiring a plurality of images having a common image area corresponding to the same area of the subject, wherein at least one image characteristic of noise characteristics, resolution, brightness, and color balance is different from each other;
A processing step of performing correction processing of pixel values of the plurality of images based on the difference in the image characteristics of the plurality of images;
Including a generation step of generating a composite image by aligning and synthesizing the plurality of images whose pixel values have been corrected by the processing means so that the common image regions overlap.
In the processing step, the composite image is a region in the plurality of images corresponding to at least one of two regions sandwiching a composite boundary that is a boundary at which an image used to generate a pixel value in the composite image is switched. An image processing method comprising the step of performing the correction process using a correction amount that continuously changes to a pixel having a predetermined distance in accordance with a distance from the synthesis boundary.