JP6245847B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method for performing noise reduction processing.

  Digital imaging devices such as digital still cameras and digital video cameras are widely used and are generally used. These digital imaging devices generate digital image data by converting light received by a photoelectric charge conversion device (imaging device) such as a CCD or CMOS sensor into a digital signal. In the process of generating digital image data, dark current noise, thermal noise, shot noise, and the like are generated due to the characteristics of the image sensor and circuit, and random noise (random noise) is mixed into the digital image data. Random noise is more conspicuous because the pixel pitch has been minimized with the recent downsizing and higher pixel size of the image sensor. Especially when the imaging sensitivity is increased, the random noise is noticeable and the image quality deteriorates. It has become a big factor. Therefore, various methods have been proposed to realize high image quality by removing random noise.

  For example, an area including a target pixel in one captured image (target area) is determined, the similarity between the target pixel and the reference pixel is obtained for each area, and noise reduction is performed using a weighted average corresponding to the similarity. Has been proposed (Patent Document 1). However, the method of Patent Document 1 has a problem that the noise reduction effect differs depending on the region in the case of an image in which a region including many edges such as a building and a flat region such as the sky are mixed. This is because, compared with the edge portion, the flat portion has more reference pixels similar to the pixel of interest and is more likely to reduce noise. That is, the noise reduction effect is higher in the flat portion than in the edge portion. On the other hand, a method has been proposed in which a plurality of images of the same imaging scene are captured, and the positional deviation due to each camera shake or moving subject is corrected after noise reduction in one captured image as described above and combined ( Patent Document 2). In the method of Patent Literature 2, first, noise reduction is performed on one captured image so that a certain amount of noise is obtained, and then a plurality of images on which one noise reduction has been performed are combined to generate an image. The noise reduction effect is made constant.

Special table 2007-536662 gazette JP 2010-171753 A

  According to the method of Patent Document 2, it is possible to avoid that the noise reduction effect in the image differs depending on the region. However, the method of Patent Document 2 is a technique for performing noise reduction processing in one image and then combining a plurality of pieces of image data subjected to noise reduction processing. In noise reduction processing in one image as in Patent Document 1 and Patent Document 2, the original pixel value of the target pixel is estimated from the relationship between the target pixel and surrounding pixels. Therefore, when the amount of noise is very large, it is difficult to estimate the original pixel of the pixel of interest. Therefore, when the S / N ratio is very bad, image quality problems such as blurring and artifacts are caused. When noise reduction is performed by synthesizing image data after noise reduction processing in such a single image as in Patent Document 2, an image quality failure that occurs in the noise reduction processing in one image. Noise reduction processing is performed while leaving the influence of the above. This may further cause image failure.

An image processing apparatus according to the present invention generates noise by combining a plurality of image data acquired by the acquisition unit with an acquisition unit that acquires a plurality of image data continuously captured, and generating composite image data. First noise reduction means for reducing, and second noise for performing noise reduction on the composite image data using a correction amount for each pixel used when the composite image data is generated in the first noise reduction means. possess a reduction means, said first noise reduction means, blocks interest any one of a plurality of image data image data, the reference image data to otherwise corresponds to the processed pixel of the focused image data and characterized by determining the area, according to the difference between the corresponding block area in the search position included in the search area of the reference image data, a correction amount for each pixel That.

  According to the present invention, it is possible to reduce image quality disturbances such as a reduction in noise reduction effect and occurrence of artifacts, while avoiding a difference in noise reduction effect in an image depending on a region.

1 is an external view illustrating a configuration of an image pickup apparatus in Embodiment 1. FIG. 1 is a block diagram illustrating an internal configuration of an imaging apparatus according to Embodiment 1. FIG. FIG. 3 is a block diagram illustrating an internal configuration of an image processing unit according to the first embodiment. 3 is a flowchart illustrating a flow of overall processing in the first embodiment. 3 is a flowchart illustrating a flow of a multiple sheet combining process according to the first exemplary embodiment. 6 is a flowchart illustrating a flow of similarity output processing in the first embodiment. FIG. 6 is a diagram for explaining a multiple-sheet composition process in the first embodiment. It is a figure for demonstrating the similarity calculation in Example 1. FIG. 3 is a flowchart illustrating a flow of noise reduction processing in the first embodiment. It is a figure for demonstrating the noise reduction process in Example 1. FIG. It is a figure for demonstrating the weight determination in Example 1. FIG.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

  An outline of the first embodiment will be described. In the first embodiment, noise reduction is performed by performing synthesis processing on a plurality of RAW image data captured continuously, and image data corresponding to one RAW image is generated. Hereinafter, the image data corresponding to one RAW image is referred to as one RAW image data. Thereafter, RAW noise reduction processing within one image is performed on the generated RAW image data, thereby generating image data with further noise reduction. Note that the RAW image data is a digital signal obtained via a color imaging element unit 201 and an A / D conversion unit 208 described later.

  In the present embodiment, any one of the images indicated by continuously captured image data is referred to as a focused image, and the other images are referred to as reference images.

  FIG. 1 is an external view showing a configuration of an image pickup apparatus in the present embodiment. FIG. 1A illustrates the front surface of the imaging apparatus, and FIG. 1B illustrates the rear surface of the imaging apparatus. The imaging device 101 includes an optical unit 102, an imaging button 103, a display unit 104, and operation buttons 105. The optical unit 102 includes a zoom lens, a focus lens, a shake correction lens, an aperture, and a shutter, and collects light information of the subject. The imaging button 103 is a button for the user to instruct the imaging apparatus 101 to start imaging. The display unit 104 uses a liquid crystal display or the like, and displays image data processed by the imaging apparatus 101 and various data. The operation button 105 is a button for the user to instruct the imaging apparatus 101 of imaging condition parameters and the like.

  FIG. 2 is a block diagram illustrating an internal configuration of the imaging apparatus 101 in the present embodiment. The color imaging element unit 201 is an element that converts light information collected by the optical unit 102 into a current value, and acquires color information by combining with a color filter or the like. The CPU 202 is involved in all the processes of each component, and sequentially reads and interprets instructions stored in a ROM (Read Only Memory) 203 and a RAM (Rondom Access Memory) 204, and executes processes according to the results. In addition, the imaging system control unit 205 performs control instructed by the CPU 202 such as adjusting the focus, opening the shutter, and adjusting the aperture of the optical unit 102. The control unit 206 controls the start and end of the imaging operation according to a user instruction from the imaging button 103 and the operation button 105. The character generation unit 207 generates characters and graphics. The A / D conversion unit 208 converts the amount of light of the subject detected by the color image sensor unit 201 into a digital signal value. The image processing unit 209 performs image processing on the image data (that is, RAW image data) of the digital signal. The encoder unit 210 converts the image data processed by the image processing unit 209 into a file format such as Jpeg. The media I / F 211 is an interface for transmitting / receiving image data to / from a PC / media 213 (for example, a hard disk, a memory card, a CF card, an SD card, etc.). The system bus 212 is a bus for transmitting and receiving data.

  FIG. 3 is an internal configuration block diagram of the image processing unit 209 according to the first embodiment. A focused RAW image data storage unit 301 stores RAW image data of a focused image (hereinafter referred to as focused RAW image data) among a plurality of RAW image data. The reference RAW image data storage unit 302 stores RAW image data of a reference image (hereinafter referred to as reference RAW image data).

  The output RAW image data storage unit 303 stores RAW image data obtained as a result of performing a combining process described later. The parameter storage unit 304 stores parameters necessary for the synthesis process and the noise reduction process in one image. The composite weight data storage unit 305 stores composite weight data at the time of synthesis. The synthesis weight data is data serving as an index indicating the degree (correction amount) of the noise reduction effect by the synthesis process. Details will be described later. The output image data storage unit 306 stores the image data output by each processing unit. The multiple RAW image composition processing unit 307 synthesizes the target RAW image data stored in the target RAW image data storage unit 301 and the reference RAW image data stored in the reference RAW image data storage unit 302. Then, the synthesized image data (RAW image data) obtained by the synthesis process is stored in the output RAW image data storage unit 303. The multiple RAW image composition processing unit 307 stores the composite weight data in the composite weight data storage unit 305. The RAW noise reduction processing unit 308 uses the combined weight data stored in the combined weight data storage unit 305 to generate one image for the combined RAW image data stored in the output RAW image data storage unit 303. RAW noise reduction processing is performed. The RAW noise reduction processing unit 308 stores the output RAW image data in the output RAW image data storage unit 303. The development processing unit 309 performs various development processes on the RAW image data after the RAW noise reduction processing stored in the output RAW image data storage unit 303. For example, processing such as white balance, demosaicing, color conversion, gamma, sharpness, and Jpeg conversion is performed, and the developed image data is stored in the output image data storage unit 306.

  FIG. 4 is a diagram showing an overall processing flow in the first embodiment. First, in step S <b> 401, the control unit 206 sets imaging conditions such as an aperture value and a focal length designated by the user via the operation button 105. In step S402, the control unit 206 determines whether or not the imaging button 103 has been pressed by the user. If it has been pressed, the process proceeds to step S403, and if it has not been pressed, the process returns to step S401.

  In step S403, the imaging system control unit 205 performs continuous imaging on the same subject. The image processing unit 209 stores the RAW image data of the target image in the captured RAW image data in the target RAW image data storage unit 301, and further stores the RAW image data of the reference image in the reference RAW image data storage unit 302. .

  In step S <b> 404, the multiple RAW image composition processing unit 307 uses the RAW image data of the image of interest, the RAW image data of the reference image, and the noise characteristic data of the imaging sensor stored in the parameter storage unit 304. Perform synthesis processing. Details of the multi-sheet composition process will be described later. The multiple RAW image composition processing unit 307 stores the RAW image data obtained as a result of the multiple image composition processing in the output RAW image data storage unit 303. Further, the multiple RAW image synthesis processing unit 307 stores the synthesis weight data at the time of synthesis in the synthesis weight data storage unit 305.

  In step S405, the RAW noise reduction processing unit 308 uses the RAW image data stored in the output RAW image data storage unit 303 and the composite weight data stored in the composite weight data storage unit 305 to reduce the RAW noise. Process. Details of the RAW noise reduction processing will be described later. The RAW noise reduction processing unit 308 stores the RAW image data obtained as a result of the RAW noise reduction processing in the output RAW image data storage unit 303.

  In step S <b> 406, the development processing unit 309 performs development processing on the RAW image data stored in the output RAW image data storage unit 303. The development processing unit 309 stores image data obtained as a result of the development processing in the output image data storage unit 306. In step S <b> 407, the control unit 206 displays the output image data stored in the output image data storage unit 306 on the display unit 104. Finally, in step S408, the control unit 206 outputs the image data stored in the output image data storage unit 306 to the PC / media 213 via the media I / F 211.

<Multi-sheet composition processing>
The multi-sheet combining process in S404 in FIG. 4 will be described. The multiple image combining process is a process for reducing noise by combining image data using the target image data and one or more reference image data. That is, the multiple sheet combining process can also be referred to as a first noise reduction process. In the following, for simplification of description, the data to be synthesized is simply referred to as a focused image and a reference image. In general, noise can be reduced by simply averaging the target image and the reference image. However, in the case of continuous imaging, when the images of the images do not match due to the positional deviation of the imaging device or the movement of the moving subject, image quality deterioration such as multiple images and blurring occurs if simple averaging is performed. Therefore, the composition is performed by comparing the image region including the target pixel of the target image (hereinafter referred to as a block region) and the block region of the reference image (hereinafter referred to as block matching).

  FIG. 5 is a processing flow of the multi-sheet composition processing S404. FIG. 7 is a diagram for explaining the multi-sheet composition process. FIG. 7A shows a target image 701, and FIG. 7B shows a reference image 702. In addition, the position of the pixel to be processed in the target image 701 is indicated by 703. A search area of the reference image 702 corresponding to the processing pixel position 703 of the target image 701 is indicated by 704. A block area corresponding to the processing pixel position 703 of the target image 701 is indicated by 705. A block area corresponding to the search position 707 of the reference image 702 is indicated by 706.

  Next, an outline of block matching will be described. In block matching, each pixel of the image of interest is sequentially processed as a processing pixel. In the example of FIG. 7, the search range corresponding to the processing pixel position 703 of the image of interest 701 is determined. In the example of FIG. 7, it is assumed that the search range is determined to be 5 × 5 pixels centering on the pixel of the reference image at the position corresponding to the processing pixel position of the target image data. Then, block matching is performed between the block area 705 corresponding to the processing pixel position 703 and the block area 706 corresponding to the search position 707. Block matching is performed for each search position within the search range.

  Next, the flow of the multi-sheet composition process will be described with reference to FIG.

  In step S <b> 501, the multiple RAW image composition processing unit 307 sets a search range stored in the parameter storage unit 304. In the example of FIG. 7, the search range is set to 5 × 5 pixels centering on the pixels of the reference image at the position corresponding to the processing target position of the image of interest. Note that general RAW image data has a Bayer arrangement with only one color of R, G, and B per pixel. Therefore, the search position 707 needs to have the same color as the processing pixel position 703. That is, in the example of FIG. 7, block matching is performed on the search positions for nine pixels indicating the R pixels included in the search range 704.

  In step S502, the multiple RAW image composition processing unit 307 initializes the output pixel value and the composition weight data to 1.

  In step S <b> 503, the multiple RAW image composition processing unit 307 reads one image data among the reference RAW image data stored in the reference image data storage unit 302.

  In step S504, the multiple RAW image composition processing unit 307 initializes the processing pixel position.

  In step S505, the multiple RAW image composition processing unit 307 determines an output pixel value and similarity that are the maximum similarity within the search range. Details will be described later.

  In step S506, the multiple RAW image composition processing unit 307 updates the output pixel value and composition weight data using the output pixel value and the similarity determined in step S505. Details will be described later.

  In step S507, the multiple RAW image composition processing unit 307 determines whether or not the processing in steps S505 to S506 has been performed on all the processing pixel positions of the target image data. If processing has been performed for all processing pixel positions, the process proceeds to step S509, and if not, the process proceeds to step S508.

  In step S508, the multiple RAW image composition processing unit 307 updates the processing pixel position, and proceeds to step S505.

  In step S509, the multiple RAW image composition processing unit 307 determines whether or not the processing in steps S504 to S508 has been performed on all the reference RAW image data stored in the reference RAW image data storage unit 302. If the process has been performed, the process proceeds to step S511; otherwise, the process proceeds to step S510.

  In step S510, the multiple RAW image composition processing unit 307 reads unprocessed RAW image data from the reference RAW image data stored in the reference RAW image data storage unit 302, and the process proceeds to step S504.

  In step S511, the multiple RAW image composition processing unit 307 outputs the output RAW image data to the output RAW image data storage unit 303. The output RAW image data is the RAW image data for which the output pixel value has been updated in step S506 (that is, obtained as a result of combining).

<Calculation of pixel value and similarity that have maximum similarity within the search range>
The process of determining the output pixel value and the similarity that are the maximum similarity within the search range in step S505 in FIG. 5 will be described. FIG. 6 shows an internal processing flow of step S505. The process of FIG. 6 is performed on one processing target pixel.

  In this embodiment, the minimum error between blocks in the search area is determined for the target image and the reference image, and the maximum similarity is determined based on the result.

  In step S601, the multiple RAW image composition processing unit 307 sets a block size Nb and a weight control parameter h (details will be described later) stored in the parameter storage unit 304.

  In step S602, the multiple RAW image composition processing unit 307 initializes the minimum error value Cmin to the maximum value in the range of possible values. In step S603, the multiple RAW image composition processing unit 307 initializes the output pixel value o to zero. In step S604, the multiple RAW image composition processing unit 307 initializes the search position i in the search range corresponding to the currently focused processing pixel. In step S605, the multiple RAW image composition processing unit 307 determines the error Ci based on Equation 1.

Here, b _t is a block area of the image of interest, and b _{r, i} is a block area at the search position i of the reference image.

  In step S606, the multiple RAW image composition processing unit 307 compares Ci determined in step S605 with the minimum error value Cmin. If Ci is smaller than Cmin, the process proceeds to step S607; otherwise, the process proceeds to step S609.

  In step S607, the multiple RAW image composition processing unit 307 updates the minimum error value Cmin to Ci.

  In step S608, the multiple RAW image composition processing unit 307 updates the output pixel value to the pixel value at the search position i.

  In step S609, the multiple RAW image composition processing unit 307 determines whether or not processing has been performed at all search positions. If processing has been performed at all search positions, the process proceeds to step S611. Otherwise, the process proceeds to step S610.

  In step S610, the multiple RAW image composition processing unit 307 updates the search position.

  In step S611, the multiple RAW image composition processing unit 307 calculates the similarity s that is the maximum similarity based on Expression 2.

<Weight control parameter h>
The weight control parameter h set in step S601 in FIG. 6 will be described. FIG. 8 shows an example in which the error C as the minimum error value Cmin and the similarity s are plotted by changing the weight control parameter h in Expression 2. As shown in FIG. 8, the similarity s increases with respect to the error as the value of h increases. On the other hand, in general, even if the same subject is imaged, the amount of noise increases as the ISO sensitivity increases. As a result, the error determined by Equation 1 increases as the amount of noise increases. Therefore, in this embodiment, the value of h is changed according to the amount of noise at the time of imaging, that is, the ISO sensitivity, in order to determine the similar degree of similarity for the same subject regardless of the amount of noise.

<Update of output pixel value and composite weight data>
The update of the output pixel value and composite weight data in step S506 will be described. If the output pixel value determined in step S505 is v and the similarity is s, the updated output pixel value O (x, y) is obtained by weighted addition of the output pixel value and the similarity as shown in Equation 3. It is done. Also, the updated composite weight data α (x, y) is obtained by Equation 4. The update in step S506 is performed for each reference image data.
O (x, y) + = v · s (Formula 3)
α (x, y) + = s (Formula 4)

<RAW noise reduction processing>
The RAW noise reduction process in step S405 in FIG. 4 will be described. In the RAW noise reduction processing, in addition to the conventional noise reduction processing method in one image, more effective noise reduction processing is performed by using the synthesis weight data α determined in step S404. Specifically, the composite weight data α determined in step S404 has an effect of reducing noise in each pixel of the output RAW image data. Therefore, the effect of the noise reduction process in one image is set to a large value for a pixel whose noise reduction effect is small by the multi-sheet composition process, and the noise reduction process in one image is set for a large pixel. Set the effect of. By processing in this way, it can be avoided that the noise reduction effect varies depending on the region. In the present embodiment, the composition processing is performed using a plurality of RAW image data captured continuously. Since the plurality of RAW image data are similar to each other, the pixels whose noise has been reduced by combining the plurality of RAW image data are used. On the other hand, noise reduction processing within one piece of image data is performed for pixels that have not been able to reduce noise by combining a plurality of RAW image data. Thus, according to the present embodiment, it is possible to obtain an equivalent noise reduction effect for each pixel. Note that the noise reduction processing in one piece of image data in step S405 is referred to as second noise reduction processing.

  FIG. 9 is a diagram showing a flow of internal processing of the RAW noise reduction processing in step S405 of FIG.

  In step S <b> 901, the RAW noise reduction processing unit 308 acquires and sets parameters necessary for processing from the parameter storage unit 304.

  In step S902, the RAW noise reduction processing unit 308 reads the RAW image data obtained by combining a plurality of sheets in step S404, which is stored in the output RAW image data storage unit 303.

  In step S903, the RAW noise reduction processing unit 308 initializes a processing pixel position.

  In step S904, the RAW noise reduction processing unit 308 performs noise reduction processing on the processing pixel position. Details will be described later.

  In step S906, the RAW noise reduction processing unit 308 determines whether or not the noise reduction processing in step S904 has been performed on all processing pixel positions. If noise reduction processing has been performed for all processing pixel positions, the process proceeds to step S908, and if not, the process proceeds to step S907.

  In step S907, the RAW noise reduction processing unit 308 updates the processing pixel position, and the process returns to step S904.

  In step S <b> 908, the RAW noise reduction processing unit 308 outputs the RAW image data as a processing result to the output RAW image data storage unit 303.

<Noise reduction processing>
In this process, a technique called a non-local-means (hereinafter, NLM) method is used as the noise reduction process in step S904. This method multiplies the pixel values of a plurality of reference pixels that exist around the pixel of interest, including the pixel of interest that is subject to noise reduction, adaptively weights the pixel values of the pixel of interest by adding all the results. Noise is reduced by replacement. When the number of reference pixels is N _S , the pixel value of the reference pixel is I _j (j = 1 to N _S ), and the weight of the reference pixel is w _j (j = 1 to N _S ), attention after noise reduction processing The pixel value I _new of the pixel is given by Equation 5.

  Next, a method for determining the weight of the reference pixel will be described with reference to FIGS.

FIG. 10A shows one piece of image data 1001. The pixel value of each pixel is represented as I (x, y) with the upper left pixel of the image data 1001 as the origin. Here, reference numeral 1002 denotes a target pixel, and its pixel value is I (4, 4). Reference numeral 1003 denotes a region of interest, which is a rectangular region of 5 × 5 pixels centered on the pixel of interest 1002 that is a noise reduction target. Reference numeral 1004 denotes a reference pixel, which is a pixel in a 5 × 5 pixel (N _S = 25) rectangular area including the target pixel 1002. Reference numeral 1005 denotes a reference region of the reference pixel I (2, 2), which is a 5 × 5 pixel rectangular region having the same size as the region of interest centered on the reference pixel I (2, 2). A reference area exists for each reference pixel, but only the reference area of the reference pixel I (2, 2) is shown here.

In order to obtain the weight of the reference pixel I (2, 2), first, the similarity is calculated by comparing the attention area 1003 with the reference area 1005 of the reference pixel I (2, 2). Note that the similarity may be obtained by a desired method. For example, as shown in FIG. 10B, the pixel of the attention area 1003 is b _s (p, q), and the pixel of the reference area 1005 is b _j (p, q) (j = 1 to N _S ). If the difference between the spatially corresponding pixels between the region of interest 1003 and the reference region 1005 is taken as an error, the error C _j is expressed by Equation 6.

The smaller the value of the error C _j is, the higher the similarity between the attention area and the reference area is. Therefore, the weight is determined according to the error. The weight may be determined so that the weight is larger as the error C _j is smaller and the weight is smaller as the error C _j is larger as in the function shown in FIG.

  h is a variable for controlling the magnitude of the weight. When h is increased, the weight is increased and the noise reduction effect is increased. On the other hand, if h is reduced, the weight is reduced and the noise reduction effect is reduced. Here, the noise can be effectively reduced by changing h in accordance with the amount of noise included in the image data. Therefore, in the present embodiment, the value of h is changed according to the combination weight data when the combination process is performed in the combination process in step S404. Here, the composition weight data output in step S404 is the number of images used for composition. At this time, the standard deviation σ ′ of noise when n images are combined is expressed by Equation 8, where the standard deviation of one noise is σ.

  Therefore, if the combined weight data at the processing pixel position j is α (j), Expression 7 is changed to Expression 9.

  Similarly, the weight of each reference pixel can be obtained by sequentially comparing the region of interest 1003 and the reference region of each reference pixel. Then, the obtained weight is applied to Equation 5 to obtain a pixel value after noise reduction.

  Note that the noise reduction processing in this embodiment may be processing in which the weight of the reference pixel is determined based on the error between the focus area and the reference area and the combined weight data, and the calculation method of the error and weight is the method described here. It is not limited to.

  As described above, in this embodiment, first, noise reduction is performed by combining a plurality of pieces of image data, and further, noise reduction of one piece of image data is performed based on the combination result. As a result, it is possible to reduce image quality problems such as a reduction in noise reduction effect and the occurrence of artifacts. Further, since noise reduction of one piece of image data is performed only on the combined image data, the processing load can be reduced.

<Other examples>
In the above embodiment, an example in which a Bayer image before demosaic processing (pixel interpolation processing) is synthesized has been described. By synthesizing the Bayer image before demosaic processing, it is possible to prevent image degradation that may occur during demosaic processing. However, the present invention combines image data before performing noise reduction processing within one image, and performs noise reduction processing within one image on the image data obtained by combining. Applicable to the case. For example, a process of combining RGB image data after demosaic processing may be performed.

  Further, image data obtained by combining pixels having RGB planes obtained from sensors such as a multilayer imaging sensor or a 3CCD sensor, and reducing noise in one image with respect to the obtained image data Processing may be performed. In this case, the search range in the synthesis process and the reference pixel settings in the noise reduction process are set according to the image data to be synthesized, and the other processes may be performed in the same manner.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (13)

An acquisition means for acquiring a plurality of image data captured continuously;
First noise reduction means for reducing noise by combining a plurality of image data acquired by the acquisition means to generate composite image data;
Have a second noise reduction means for performing noise reduction for the synthetic image data by using the correction amount for each pixel used to generate the composite image data in said first noise reduction means,
The first noise reduction means uses any one of the plurality of image data as the target image data and the other as the reference image data, a block region corresponding to a processing pixel of the target image data, and the reference image data An image processing apparatus that determines a correction amount for each pixel according to a difference from a block area corresponding to a search position included in the search area . The image processing apparatus according to claim 1 , wherein the first noise reduction unit generates the composite image data by performing weighted addition of pixel values of reference image data using the determined correction amount. . The image processing apparatus according to claim 2 , wherein the first noise reduction unit performs the weighted addition using a pixel value of a pixel having a minimum error in a search position included in the search area. . The first noise reduction unit determines a correction amount for each pixel used when generating the synthesized image data by adding a correction amount for each reference image data used for synthesis. Item 4. The image processing device according to any one of Items 1 to 3 . The second noise reduction unit determines a difference between a target area including a target pixel of the composite image data and a reference area including a reference pixel of the composite image data, and determines the difference and a correction amount for each pixel. depending determines the weights of the reference pixels, the image processing according to any one of claims 1 to 4, characterized by updating the pixel value of the target pixel by performing a weighted addition in accordance with the weight apparatus. The image processing apparatus according to any one of claims 1 to 5 , wherein the image data acquired by the acquisition unit is image data indicating a Bayer image. The image data acquired by the acquisition means, image processing device according to claim 1, wherein in any one of the 5 that the image data before demosaic processing. The image data acquired by the acquisition means, image processing device according to claim 1, wherein in any one of the 5 that the image data obtained by the stacked image sensor. The image data acquired by the acquisition means, image processing device according to claim 1, wherein in any one of the 5 that the image data obtained by 3CCD sensor. The image processing apparatus according to any one of claims 1 to 6, further comprising a demosaic unit for pixel interpolation on the noise reduced synthesized image data by the second noise reducing means. Imaging means for continuously imaging a plurality of image data;
First noise reduction means for reducing noise by combining a plurality of image data picked up by the image pickup means to generate composite image data;
Have a second noise reduction means for performing noise reduction for the synthetic image data by using the correction amount for each pixel used to generate the composite image data in said first noise reduction means,
The first noise reduction means uses any one of the plurality of image data as the target image data and the other as the reference image data, a block region corresponding to a processing pixel of the target image data, and the reference image data A correction amount for each pixel is determined in accordance with a difference from a block area corresponding to a search position included in the search area . An acquisition step of acquiring a plurality of image data continuously captured;
A first noise reduction step of reducing noise by combining a plurality of image data acquired in the acquisition step to generate composite image data;
Have a second noise reduction step for performing noise reduction for the synthetic image data by using the correction amount for each pixel used to generate the composite image data in the first noise reduction step,
The first noise reduction step uses any one of a plurality of image data as target image data and the other as reference image data, and a block region corresponding to a processing pixel of the target image data; and the reference image data A correction amount for each pixel is determined according to a difference from a block area corresponding to a search position included in the search area . The program for functioning a computer as an image processing apparatus as described in any one of Claim 1 to 10 .

Images