JP6378496B2 - Image processing apparatus, control method, and recording medium - Google Patents

Description

  The present invention relates to an image processing apparatus, a control method, and a recording medium, and more particularly to a technique for reducing noise included in an image.

  An image obtained by photographing with an imaging device such as a digital camera may contain noise depending on photographing conditions. Since noise becomes conspicuous when amplified by a gain process or the like in the development process, noise reduction processing (NR processing) for reducing noise is applied to the image. Since NR processing corrects pixels that are determined to be noise using surrounding pixels, when the processing is applied to the entire image, the image may be blurred depending on the application level. sell. In particular, when such processing is performed on the edge of the subject image in the image, it may cause a decrease in the resolution of the image.

  Therefore, when applying NR processing to an image, there is a method in which processing is different depending on whether the pixel to be applied is an edge portion or a so-called flat portion other than that (Patent Document 1 and 2).

JP 2000-331152 A JP 2008-61217 A

  On the other hand, some recent imaging apparatuses employ an imaging element that can handle high-sensitivity imaging. Since high-sensitivity shooting has a higher signal amplification factor in the development process, random noise generated in the image may be higher than the signal level of the subject image. Such random noise may be identified as an edge portion in determining whether or not it is an edge portion, and a suitable correction result cannot be obtained when NR processing as in Patent Documents 1 and 2 is performed. There was a possibility.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide an image processing apparatus, a control method, and a recording medium that suitably detect an included edge in an image including noise. .

In order to achieve the above object, an image processing apparatus of the present invention is characterized by having the following configuration. Specifically, the image processing apparatus acquires an edge of a subject image in each image from an acquisition unit that acquires a plurality of input images captured within a predetermined time, and a plurality of input images acquired by the acquisition unit. The first generation unit that generates the emphasized edge image, the correction unit that corrects the positional deviation of the subject image between the plurality of edge images generated by the first generation unit, and the positional deviation is corrected by the correction unit. In a plurality of edge images, noise is detected from a plurality of edge images based on information on a pixel in which noise is detected as an edge, and information on a pixel in which the noise specified by the specifying means is detected as an edge. possess a second generating means for generating a reduced output edge image, a specific means, the upper of the plurality of edge images, the absolute value of the difference between the pixel values of the image to threshold Pixels that, noise and identifies a pixel that is detected as an edge.

  With such a configuration, according to the present invention, it is possible to suitably detect an included edge in an image including noise.

1 is a block diagram showing a functional configuration of an image processing apparatus 100 according to Embodiment 1 of the present invention. The block diagram which illustrated the detailed structure of the 1st edge image generation part 101 which concerns on embodiment of this invention, and the 2nd edge image generation part 102 The figure for demonstrating the production | generation of the output edge image which concerns on embodiment of this invention The block diagram which showed the function structure of the image processing apparatus 100 which concerns on Embodiment 2 of this invention. The block diagram which illustrated the detailed structure of the edge NR process part 401 which concerns on Embodiment 2 of this invention. The figure which illustrated the direction template used in the edge direction detection part 501 which concerns on Embodiment 2 of this invention. The figure which illustrated the pixel used for NR processing in adaptive NR processing part 502 concerning Embodiment 2 of the present invention. The figure for demonstrating the NR process in the 1st flat NR process part 402 and the 2nd flat NR process part 403 which concern on Embodiment 2 of this invention. The block diagram which illustrated the detailed structure of the edge process part 406 which concerns on Embodiment 2 of this invention. The block diagram which illustrated the detailed composition of the image composition part 407 concerning Embodiment 2 of the present invention.

[Embodiment 1]
Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. In an embodiment described below, an example in which the present invention is applied to an apparatus such as a PC that can generate an edge image obtained by detecting an edge portion of a subject image from an image will be described as an example of an image processing apparatus. However, the present invention can be applied to any device that can detect an edge portion of a subject image from an image. In this specification, the “edge portion” refers to a region where the spatial frequency is high and the amplitude undulation in the frequency domain is steep, and the “flat portion” refers to a gradual amplitude undulation in the frequency domain. It shall refer to an area.

<< Configuration of Image Processing Apparatus 100 >>
FIG. 1 is a block diagram illustrating a functional configuration related to edge image generation in the image processing apparatus 100 according to the embodiment of the present invention. In the present embodiment, the image processing apparatus 100 uses two images obtained by continuously shooting the same subject under the same shooting conditions, reduces random noise, and generates an edge image that preferably shows an edge portion. An example will be described. However, the plurality of images to be processed in the embodiment of the present invention are not limited to this, and may be a plurality of images obtained by photographing the same subject within a predetermined time. Further, the configuration of the image processing apparatus 100 described below may be configured as a circuit that sequentially processes each input image, or a predetermined operation program recorded on a recording medium by a control unit (not shown). The operation may be controlled based on this.

  The first input image and the second input image to be processed are input to the first edge image generation unit 101 or the second edge image generation unit 102, respectively. The first edge image generation unit 101 and the second edge image generation unit 102 generate an edge image by applying a predetermined filter to the input image. For example, the first edge image generation unit 101 and the second edge image generation unit 102 may have a detailed configuration as shown in FIG.

  In FIG. 2A, a horizontal Sobel filter 201 and a vertical Sobel filter 203 are applied to the input image, and the image is converted into an image in which edge components are emphasized in each of the horizontal direction and the vertical direction. The filters applied in the horizontal Sobel filter 201 and the vertical Sobel filter 203 may be filters as shown in FIGS. 2B and 2C, respectively. The images in which the edges are emphasized in each of the horizontal direction and the vertical direction are input to the absolute value filter 202 or the absolute value filter 204 and converted into absolute values, and then added in the adder 205 to obtain the first It is output as an edge image or a second edge image.

  The alignment unit 103 performs a process for aligning the subject image in the first edge image and the second edge image output from the first edge image generation unit 101 and the second edge image generation unit 102. The first input image and the second input image to be processed are photographed continuously, but the captured subject image is moved two-dimensionally during that time due to the camera shake of the photographer. Since there is a possibility, the alignment unit 103 corrects the positional deviation. Specifically, the alignment unit 103 acquires the amount of positional deviation by calculating the Euclidean distance between the first edge image and the second edge image, for example, so that the first edge image matches the subject image. The second edge image is changed and output. The alignment unit 103 outputs the first edge image as it is. In the present embodiment, the alignment unit 103 is described as performing alignment of the second edge image based on the first edge image, but the embodiment of the present invention is not limited to this. The alignment unit 103 may align the first edge image with the second edge image, or may align both images with the intermediate position of displacement.

  The edge synthesizing unit 104 synthesizes the first edge image and the second edge image from which the positional deviation has been eliminated, and generates an output edge image with reduced random noise.

<Generation of output edge image>
Here, the generation method and principle of the output edge image generated in the edge synthesis unit 104 will be described in detail below.

The edge synthesis unit 104 first generates a difference image from the input first edge image and the second edge image after alignment. In the generation of the difference image, it is only necessary to know the difference between the images. Therefore, the pixel value d of each pixel of the difference image is calculated using an absolute value.
And Here, e1 and e2 are pixel values (pixel values related to the same subject image) at the same pixel position in the first edge image and the second edge image, respectively.

  As described above, when the S / N ratio of the first input image and the second input image is low, random noise generated in the image can remain as an edge after edge imaging. However, while the portion that is originally an edge in the subject image always appears as an edge, the portion that remains as an edge due to erroneous recognition of random noise does not always appear as an edge. That is, random noise does not always occur at the same position and at the same signal level. Therefore, in this embodiment, the edge synthesis unit 104 calculates a difference between the edge images, thereby estimating a portion where random noise is erroneously recognized as an edge. That is, when the difference between edge images is taken for the part that is essentially an edge, the difference is close to 0, and when the difference between edge images is taken for the part that is a random noise, the difference is 0. Rather, it indicates a value corresponding to the noise. Therefore, in this embodiment, the edge composition unit 104 determines whether each pixel is an edge by comparing the absolute value of the difference with a threshold value.

  Note that if two types of input images are taken under the same shooting conditions as in the present embodiment, such a phenomenon appears, but the two types of input images may have different shooting conditions. . In this case, since the signal level rises and falls according to the shooting conditions, the difference absolute value may be calculated after normalization according to the shooting conditions. That is, the plurality of images input to the image processing apparatus 100 are preferably captured under the same imaging conditions, but this is not an essential requirement in the practice of the present invention.

  The edge synthesis unit 104 compares the pixel value of the obtained difference image with the threshold T, and determines whether each pixel is an edge that has appeared due to erroneous recognition of random noise. Specifically, for the pixels of the difference image whose pixel value is equal to or less than the threshold value T, the edge composition unit 104 determines that the pixel at the same pixel position in the first edge image and the second edge image is the original edge portion or It is judged that the flat part recognized correctly is shown. In addition, for the pixels of the difference image in which the pixel value exceeds the threshold value T, the edge composition unit 104 generates an edge that appears due to erroneous recognition of random noise in the pixel at the same pixel position in the first edge image and the second edge image. That is, it is determined that it was originally a flat part. Here, it is assumed that the threshold value T is determined by the ISO sensitivity and the luminance level in an area (photographing environment) corresponding to a pixel among the photographing conditions of the first input image or the second input image. For example, noise having a high signal level can occur in an area with high luminance, and therefore T is set to be large accordingly.

In this embodiment, the edge composition unit 104 assigns a pixel from the first edge image to a pixel at the same pixel position in the output edge image for a pixel that is determined to be an original edge portion or a flat portion that is correctly recognized. . In addition, the edge synthesis unit 104 performs the weighted average of the first edge image and the second edge image for the pixel that is determined to be an edge that appears due to erroneous recognition of random noise, so that the same pixel position of the output edge image is obtained. Are generated. The weight of the weighted average may be determined linearly using the threshold value T, and the weight w assigned to the pixel of the second edge image when both cases are combined is w = d−T (d> T)
w = 0 (d ≦ T)
Can be expressed as That is, each pixel e of the output edge image has the weights of the first edge image and the second edge image as shown in FIGS. 3A and 3B.
e = {e1 · (T−w) + e2 · w} / T
Can be calculated as

  That is, the output edge image generated in the image processing apparatus 100 of the present embodiment determines that there is no random noise for pixels whose difference between a plurality of edge images generated for the same subject is equal to or less than a threshold value, and one edge image Only these pixels are used. In addition, it is judged that random noise is generated for pixels whose difference between multiple edge images generated for the same subject exceeds the threshold, and the influence of noise is reduced by weighted averaging of the pixels of multiple edge images. Generated pixels. By doing so, it is possible to reduce the random noise and obtain an edge image that can suitably identify the original edge portion. For pixels determined to be noise, the smallest pixel value of the plurality of edge images may be selected instead of weighted averaging of the pixel values of the plurality of edge images.

  In this embodiment, a method for generating one output edge image from two input images has been described. However, it will be easily understood that the number of input images is not limited to this in the implementation of the present invention. That is, the output edge image generated by the present invention is a weighted average of pixel values of a plurality of edge images for pixels determined to be noise between the plurality of edge images, and any pixel determined to be not noise. The pixel value of one edge image may be used. Therefore, it goes without saying that the present invention is similarly applied to three or more input images.

  As described above, the image processing apparatus according to the present embodiment can preferably detect included edges in an image including noise. Specifically, the image processing apparatus acquires a plurality of input images photographed within a predetermined time, and generates an edge image in which the edge of the subject image in each image is emphasized from the acquired plurality of input images. . Further, the positional deviation of the subject image between the plurality of generated edge images is corrected, and the pixel in which noise is detected as an edge is specified. Then, the image processing apparatus generates an output edge image with reduced noise from a plurality of edge images based on information on pixels in which noise is detected as an edge.

[Embodiment 2]
In the above-described embodiment, the method of generating an edge image with reduced influence of noise has been described. However, in the present embodiment, an image in which noise is appropriately reduced in each of the edge portion and the flat portion while generating a similar edge image. A method of generating the will be described.

<< Functional Configuration of Image Processing Apparatus 100 >>
FIG. 4 is a block diagram illustrating a functional configuration related to generation of an image obtained by performing different noise reduction (NR) processing on each of the edge portion and the flat portion of the image processing apparatus 100 according to the embodiment of the present invention. In this embodiment, as in the first embodiment, an example of generating an image with reduced random noise in an input image using two images obtained by continuously shooting the same subject under the same shooting conditions. explain. In the following description of the functional configuration, blocks that perform the same processing as in the first embodiment are denoted by the same reference numerals, description thereof is omitted, and only the configuration unique to the present embodiment is described.

  The first input image to be processed is input to the edge NR processing unit 401, the first flat NR processing unit 402, the first edge image generation unit 101, the alignment unit 404, and the moving object detection unit 405. The second input image to be processed is input to the alignment unit 404, the second edge image generation unit 102, and the second flat NR processing unit 403.

  The edge NR processing unit 401 identifies an edge of the first input image, performs NR processing according to the type of the edge, and generates an edge part NR image. In the present embodiment, the edge NR processing unit 401 is described as having the configuration shown in FIG. 5, but any of the existing methods for reducing noise at the edge of the input image according to the type of edge may be used. Good.

  The edge direction detection unit 501 assumes that all the pixels of the first input image indicate edges, and detects the edge direction for each pixel. Specifically, the edge direction detection unit 501 refers to a 5 × 5 pixel region centered on the pixel of interest, and determines which of the direction templates for edge direction detection shown in FIG. The edge direction for the pixel of interest is output as a detection result. In the detection of the edge direction, the edge direction detection unit 501 extracts pixel values of pixels (hatched pixels in each pattern in FIG. 6) that form a straight line in the direction template from the referenced 5 × 5 pixels. Then, the sum (SAD value) of the absolute values of the pixel value differences between the extracted five pixels and the target pixel is calculated. The edge direction detection unit 501 calculates SAD values for all of the direction templates, and outputs the direction in which the SAD value is minimum among the 12 directions as the edge direction of the pixel of interest.

  The adaptive NR processing unit 502 performs NR processing corresponding to the edge direction on the first input image based on the detection result of the edge direction in the edge direction detection unit 501. Specifically, the adaptive NR processing unit 502 calculates the difference in pixel value from the target pixel among the seven pixels (pixels indicated by hatching) corresponding to the edge direction as illustrated in FIG. An average value of pixel values is calculated using pixels whose absolute value is equal to or less than a predetermined value. Then, the adaptive NR processing unit 502 determines the average value as the pixel value of the target pixel after NR processing. In this way, the adaptive NR processing unit 502 generates and outputs an edge part NR image to which NR processing adapted to the edge direction is applied to all pixels of the first input image.

  The first flat NR processing unit 402 and the second flat NR processing unit 403 perform NR processing that does not consider the edge direction for the input image, that is, processing different from the NR processing performed by the edge NR processing unit 401. Run. In the present embodiment, the first flat NR processing unit 402 and the second flat NR processing unit 403 focus on the pixel of interest as shown in FIG. 8 for all pixels of the first input image or the second input image. 7 × 7 pixels are referred to. Then, among the 7 × 7 pixels, the average value of the pixel values is calculated using the pixels whose absolute value of the difference between the pixel values of the target pixel is equal to or less than a predetermined value, and determined as the pixel value of the target pixel after the NR process. . By doing in this way, the first flat NR processing unit 402 and the second flat NR processing unit 403, with respect to the flat portion excluding an image that may cause a difference in pixel value from surrounding pixels such as edges, The first flat portion NR image or the second flat portion NR image to which smoothing is applied is output.

  The alignment unit 404 performs a process of aligning the subject image in the input first input image and second input image, and corrects the influence of camera shake between the images. Specifically, the alignment unit 404 acquires the amount of positional deviation from the Euclidean distance between the images as in the alignment unit 103 of the first embodiment, and the second input so that the first input image and the subject image match. The input image is changed by projective transformation and output. In addition, the alignment unit 404 changes and outputs the second edge image and the second flat portion NR image using the acquired displacement amount. The alignment unit 404 outputs the first input image as it is.

  The moving body detection unit 405 detects a region corresponding to a subject that has moved between images from the first input image and the second input image from which the positional deviation has been eliminated. Specifically, the moving object detection unit 405 calculates a pixel value difference for each pixel between the first input image and the second input image, and detects a pixel whose difference is equal to or greater than a predetermined value and its surrounding pixels as a moving object. Specify as an area. The moving object detection unit 405 outputs the detected moving object region information.

  As in the first embodiment, the edge processing unit 406 performs processing for making the edges thicker and gentler on the output edge image generated by the edge synthesis unit 104. For the processing, first, as shown in FIG. 9A, for example, a maximum value filter 901 applies a process of assigning the maximum pixel value of 5 × 5 pixels centered on the target pixel to the target pixel. Thereafter, the low-pass filter 902 applies a low-pass filter as shown in FIG. 9B to generate a normalized value.

  As described above, the image generated by the edge NR processing unit 401, the first flat NR processing unit 402, the second flat NR processing unit 403, the alignment unit 404, the moving object detection unit 405, and the edge processing unit 406 is an image. The signal is input to the synthesis unit 407 and used to generate a noise reduced image. The image composition unit 407 of the present embodiment has a detailed configuration as shown in FIG.

  The flat image generation unit 1001 generates a flat part NR image (smoothed image) in which the influence of random noise is reduced from the input first flat part NR image, second flat part NR image, and moving body region information. To do. Specifically, the flat image generation unit 1001 uses the flat part NR to extract pixels extracted only from the first flat part NR image in order to prevent the so-called double image from being generated based on the moving object area information. Used as image pixels. For the area where no moving object is detected, an average value of the pixel values of the first flat part NR image and the second flat part NR image is assigned as a pixel of the flat part NR image. In this way, the flat image generation unit 1001 reduces the influence of noise on pixels related to random noise remaining in the smoothing process of the first flat NR processing unit 402 or the second flat NR processing unit 403. To do.

  Then, the NR image generation unit 1002 combines the edge part NR image and the flat part NR image based on the processed output edge image to generate a noise reduced image. Specifically, the NR image generation unit 1002 extracts a corresponding pixel from the edge portion NR image and extracts a corresponding pixel from the edge portion NR image for pixels whose pixel values are classified into a range indicating the edge portion in the processed output edge image. Let it be a pixel. In addition, for pixels whose pixel values are classified into a value range indicating a flat portion in the processed output edge image, a corresponding pixel is extracted from the flat portion NR image and used as a pixel of the noise reduction image. Further, in the processed output edge image, for the pixel whose pixel value is classified into an intermediate value range between the value range indicating the edge portion and the value range indicating the flat portion, the edge portion NR image and the flat portion NR image according to the pixel value Are assigned pixel values obtained by weighted averaging. In this way, the NR image generation unit 1002 refers to the output edge image obtained by the method of the first embodiment, and suitably uses each of the edge part, the flat part, and the part that is affected by random noise. An image with reduced noise can be generated.

[Other Embodiments]
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (12)

Obtaining means for obtaining a plurality of input images photographed within a predetermined time;
First generation means for generating an edge image in which an edge of a subject image in each image is emphasized from a plurality of input images acquired by the acquisition means;
Correction means for correcting a positional deviation of a subject image between a plurality of edge images generated by the first generation means;
In a plurality of edge images whose positional deviation has been corrected by the correcting unit, a specifying unit that specifies a pixel in which noise is detected as an edge;
Wherein said noise specified by the specifying means based on the information of the pixel is detected as an edge, from said plurality of edge images, possess a second generating means for generating an output edge image with reduced noise, ,
The specifying means specifies a pixel in which an absolute value of a pixel value difference between images exceeds a threshold among the plurality of edge images as a pixel in which the noise is detected as an edge. An image processing apparatus. Obtaining means for obtaining a plurality of input images photographed within a predetermined time;
First generation means for generating an edge image in which an edge of a subject image in each image is emphasized from a plurality of input images acquired by the acquisition means;
Correction means for correcting a positional deviation of a subject image between a plurality of edge images generated by the first generation means;
In a plurality of edge images whose positional deviation has been corrected by the correcting unit, a specifying unit that specifies a pixel in which noise is detected as an edge;
Second generation means for generating an output edge image in which noise is reduced from the plurality of edge images based on information on a pixel in which the noise specified by the specifying means is detected as an edge. ,
The second generation means obtains a pixel value by performing a weighted average of pixel values of the plurality of edge images for the pixel of the output edge image having the same pixel position as the pixel where the noise is detected as an edge. images processor you and obtains. Obtaining means for obtaining a plurality of input images photographed within a predetermined time;
First generation means for generating an edge image in which an edge of a subject image in each image is emphasized from a plurality of input images acquired by the acquisition means;
Correction means for correcting a positional deviation of a subject image between a plurality of edge images generated by the first generation means;
In a plurality of edge images whose positional deviation has been corrected by the correcting unit, a specifying unit that specifies a pixel in which noise is detected as an edge;
Second generation means for generating an output edge image in which noise is reduced from the plurality of edge images based on information on a pixel in which the noise specified by the specifying means is detected as an edge. ,
The second generation means, for a pixel of the output edge image having the same pixel position as a pixel for which the noise is not detected as an edge, a pixel value of any one of the plurality of edge images images processor characterized by using the. Obtaining means for obtaining a plurality of input images photographed within a predetermined time;
First generation means for generating an edge image in which an edge of a subject image in each image is emphasized from a plurality of input images acquired by the acquisition means;
Correction means for correcting a positional deviation of a subject image between a plurality of edge images generated by the first generation means;
In a plurality of edge images whose positional deviation has been corrected by the correcting unit, a specifying unit that specifies a pixel in which noise is detected as an edge;
Second generation means for generating an output edge image with reduced noise from the plurality of edge images, based on information of pixels in which the noise specified by the specifying means is detected as an edge;
First noise reduction means for reducing noise according to an edge direction with respect to one input image of the plurality of input images;
Second noise reduction means different from the first noise reduction means for reducing noise of pixels excluding pixels detected as edges for each of the plurality of input images;
The one input image whose noise is reduced by the first noise reduction means and the plurality of input images whose noise is reduced by the second noise reduction means are synthesized based on the output edge image. , wherein one of the third generation means and, images processor you characterized in that have a generating a reduced noise reduced image noise of an input image. The third generation unit generates a single smoothed image by combining the plurality of input images with reduced noise, and uses the smoothed image and the single input image with reduced noise. The image processing apparatus according to claim 4 , wherein the noise reduction image is generated. The third generation means includes
For pixels recognized as edges in the output edge image, the pixels of one input image with reduced noise are used as pixels of the noise reduced image,
For pixels that are not recognized as edges in the output edge image, pixels of the smoothed image or pixels having a pixel value obtained by using one input image with the noise reduced and the smoothed image are The image processing apparatus according to claim 5 , wherein the image processing apparatus is used as a pixel of a noise reduced image. For the plurality of input images, further comprising detection means for detecting a moving body region in which movement of a subject occurs between the images,
When the third generation unit generates the smoothed image, for the moving object region detected by the detection unit, the pixel of one image of one input image in which the noise is reduced, the image processing apparatus according to claim 5 or 6, characterized by using as a pixel of the smoothed image. An acquisition step in which an acquisition unit of the image processing apparatus acquires a plurality of input images taken within a predetermined time;
A first generation step in which a first generation unit of the image processing apparatus generates an edge image in which an edge of a subject image in each image is emphasized from a plurality of input images acquired in the acquisition step;
A correcting step in which the correcting means of the image processing device corrects the positional deviation of the subject image between the plurality of edge images generated in the first generating step;
A specifying step of specifying a pixel in which noise is detected as an edge in the plurality of edge images in which the positional deviation is corrected in the correction step;
A second generation unit of the image processing device generates an output edge image in which noise is reduced from the plurality of edge images based on information on a pixel in which the noise specified in the specifying step is detected as an edge. a second generation step of generating, the possess,
In the specifying step, the specifying unit specifies a pixel in which an absolute value of a pixel value difference between images exceeds a threshold among the plurality of edge images as a pixel in which the noise is detected as an edge. A control method of the image processing apparatus. An acquisition step in which an acquisition unit of the image processing apparatus acquires a plurality of input images taken within a predetermined time;
  A first generation step in which a first generation unit of the image processing apparatus generates an edge image in which an edge of a subject image in each image is emphasized from a plurality of input images acquired in the acquisition step;
  A correcting step in which the correcting means of the image processing device corrects the positional deviation of the subject image between the plurality of edge images generated in the first generating step;
  A specifying step of specifying a pixel in which noise is detected as an edge in the plurality of edge images in which the positional deviation is corrected in the correction step;
  A second generation unit of the image processing device generates an output edge image in which noise is reduced from the plurality of edge images based on information on a pixel in which the noise specified in the specifying step is detected as an edge. A second generating step to generate,
  In the second generation step, the second generation means weights the pixel values of the plurality of edge images for the pixels of the output edge image having the same pixel position as the pixel where the noise is detected as an edge. Find pixel values by averaging
And a control method for the image processing apparatus. An acquisition step in which an acquisition unit of the image processing apparatus acquires a plurality of input images taken within a predetermined time;
  A first generation step in which a first generation unit of the image processing apparatus generates an edge image in which an edge of a subject image in each image is emphasized from a plurality of input images acquired in the acquisition step;
  A correcting step in which the correcting means of the image processing device corrects the positional deviation of the subject image between the plurality of edge images generated in the first generating step;
  A specifying step of specifying a pixel in which noise is detected as an edge in the plurality of edge images in which the positional deviation is corrected in the correction step;
  A second generation unit of the image processing device generates an output edge image in which noise is reduced from the plurality of edge images based on information on a pixel in which the noise specified in the specifying step is detected as an edge. A second generating step to generate,
  In the second generation step, the second generation unit may select one of the plurality of edge images for a pixel of the output edge image having the same pixel position as a pixel in which the noise is not detected as an edge. Use pixel value of one edge image
And a control method for the image processing apparatus. An acquisition step in which an acquisition unit of the image processing apparatus acquires a plurality of input images taken within a predetermined time;
  A first generation step in which a first generation unit of the image processing apparatus generates an edge image in which an edge of a subject image in each image is emphasized from a plurality of input images acquired in the acquisition step;
  A correcting step in which the correcting means of the image processing device corrects the positional deviation of the subject image between the plurality of edge images generated in the first generating step;
  A specifying step of specifying a pixel in which noise is detected as an edge in the plurality of edge images in which the positional deviation is corrected in the correction step;
  A second generation unit of the image processing device generates an output edge image in which noise is reduced from the plurality of edge images based on information on a pixel in which the noise specified in the specifying step is detected as an edge. A second generating step to generate;
  A first noise reduction step in which a first noise reduction unit of the image processing device reduces noise according to an edge direction with respect to one input image of the plurality of input images;
  The second noise reduction means of the image processing apparatus is different from the first noise reduction step in which the noise of pixels excluding the pixels detected as edges is reduced for each of the plurality of input images. 2 noise reduction processes;
  The third generation unit of the image processing apparatus includes the one input image in which noise is reduced in the first noise reduction step, and the plurality of input images in which noise is reduced in the second noise reduction step. And a third generation step of generating a noise-reduced image in which noise of the one input image is reduced, based on the output edge image
And a control method for the image processing apparatus. Computer, according to claim 1 to 7 recording medium recording a program for causing to function as each means of the image processing apparatus according to any one of.

Images