JP2010181951A - Image processor and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce or remove random noise without deteriorating resolution from a photographic image. <P>SOLUTION: The image processor is provided with: an image storage part 20 for receiving and storing a plurality of images in a frame unit which are captured at a different time; an image correlation calculation part 30 for defining one of a plurality of images 2<SB>1</SB>and 2<SB>2</SB>captured at neighboring different times read from the image storage part 20 as a reference image, and for calculating image correlation data 3<SB>1</SB>by performing correlation processing between the reference image 2<SB>1</SB>and the image 2<SB>2</SB>other than the reference image; and an image composition part 40 for comparing the calculated image correlation data 3<SB>1</SB>with a correlation threshold specified in advance, and for combining the reference image 2<SB>1</SB>with the image 2<SB>2</SB>other than the reference image only when the image correlation data 3<SB>1</SB>are equal to or less than the correlation threshold. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing program for suppressing random noise appearing on a photographed digital image.

  As is well known, in recent years, the number of pixels of an image sensor has increased, and a digital camera can acquire a high-definition image as close as a film camera. However, in the case of a digital image taken using an image sensor, noise that appears as a random graininess on the image (hereinafter referred to as random noise) is generated, which greatly affects the quality of the image. Random noise includes luminance noise (or illuminance noise) and color noise (or chrominance noise) .Color noise occurs particularly in shadow areas and areas where the color is uniform. It looks uncomfortable. There are various causes of random noise, but the physical properties of light and the image sensor are said to be the biggest factors. There are various causes of sensor noise such as readout noise, light receiving element noise, photon noise, and dark current. One simple method of denoising is to replace each pixel value on the image with the average value of the neighboring pixels, but this can reduce the variation caused by random noise, but at the same time smooth the image details. Therefore, there is a problem that the sharpness of the image is remarkably lost.

  In addition, as a technique for removing the random noise, an image processing method has been proposed in which random noise in captured image data acquired by a film scanner is suppressed by a weighted average filtering process (see, for example, Patent Document 1). In this method, a target pixel is sequentially set from a group of pixels constituting captured image data, and a pixel between each peripheral pixel and the target pixel located in a calculation area for weighted average filtering processing centered on the target pixel. The difference value of the value is calculated, the weighting factor used for the weighted average filtering process is determined according to the calculated difference value, the weighted average filtering process is executed using the determined weighting factor, and the corrected pixel value of the target pixel is determined. It is what you want. Further, in this method, when the weighting coefficient having the definition range of 0 to 1 is determined according to the difference value, the weighting coefficient is set to 1 when the difference value is 0, and the absolute value of the difference value is set. A numerical value determined from a numerical value group in which the weighting factor is 0 when the threshold value is greater than or equal to the threshold value, and when the difference value is greater than 0 and smaller than the threshold value, monotonically decreases from 1 and increases monotonically to 1 as the absolute value of the difference value increases. Is a weighting factor. According to this method, in the captured image data, random noise can be sufficiently suppressed not only in a flat region but also in the contour and its vicinity, and the fine details and texture of the photograph are maintained.

JP 2006-302023 A

In the image processing method described in Patent Document 1, a weighted average filtering process is performed as a blurring (smoothing) process in order to reduce random noise. At that time, by calculating the difference value of the pixel value between the target pixel and each peripheral pixel, and applying the weighted average filtering process according to the difference value, random noise suppression that maintains the texture of the contour part is possible It is said that. In addition, the method of Patent Document 1 performs so-called spatial filtering processing, in which filtering processing is performed using a target pixel and its surrounding pixels in a target frame image. For this reason, the texture of the contour portion is maintained by performing the weighted average filtering process according to the difference value. However, the region other than the contour portion is blurred together with the random noise reduction. That is, there is a problem that the sense of resolution is lowered.
In the present invention, as described later, random noise is reduced without impairing the sense of resolution by image processing using a plurality of temporally different images. There is no description suggesting so-called time-direction image processing using other images.

  The present invention has been made to solve the above-described problems, and an object thereof is to obtain an image processing apparatus and an image processing program capable of reducing or removing random noise from a captured image without losing a sense of resolution. .

  The image processing apparatus according to the present invention uses, as a reference image, an image storage unit that inputs and stores a plurality of frames taken at different times, and a plurality of neighboring images read from the image storage unit at different times. An image correlation calculation unit that calculates correlation data between the reference image and an image other than the reference image to calculate image correlation data, compares the calculated image correlation data with a pre-specified correlation threshold value, and calculates image correlation data And an image composition unit that composes the reference image and an image other than the reference image only when is equal to or less than the correlation threshold.

  According to the present invention, since a single image is obtained by using a plurality of images that are temporally different from each other and synthesizing images having a high correlation, conventional blurring (smoothing) processing is performed. Therefore, random noise can be suppressed without impairing resolution, or resolution can be improved by noise suppression.

It is a block diagram which shows the function structure of the image processing apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the synthetic | combination processing method in the image synthetic | combination part which concerns on the same Embodiment 1. FIG. It is a block diagram which shows the function structure of the image processing apparatus by the same Embodiment 2. It is explanatory drawing which shows the synthetic | combination processing method in the image synthetic | combination part which concerns on the same Embodiment 2. FIG. It is a block diagram which shows the function structure of the image processing apparatus by the same Embodiment 3. It is a block diagram which shows the other example of a function structure of the image processing apparatus by the same Embodiment 3. FIG. It is explanatory drawing which shows the synthetic | combination processing method in the image synthetic | combination part which concerns on the same Embodiment 3. FIG. It is explanatory drawing shown about the method of the affine transformation which concerns on the same Embodiment 3. FIG. It is explanatory drawing shown about the comparison of the image after the affine transformation which concerns on the same Embodiment 3, and another input image. It is explanatory drawing which shows the image composition method of the image composition part which concerns on the same Embodiment 5. FIG. It is explanatory drawing which shows the image composition method of the image composition part which concerns on the same Embodiment 6. It is explanatory drawing which shows the image composition method of the image composition part which concerns on the same Embodiment 7. It is explanatory drawing which shows the image composition method of the image composition part which concerns on the same Embodiment 7.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a functional configuration of an image processing apparatus according to Embodiment 1 of the present invention.
In FIG. 1, the image processing apparatus according to the first embodiment includes an image storage unit 20, an image correlation calculation unit 30, and an image composition unit 40. Further, 1 ₁ is an input image, 2 ₁ is an image read from the image storage unit 20, and 2 ₂ is an image read from the image storage unit 20, but the time taken is different from the image 2 _1. image, 3 ₁ image correlation data calculated by the image correlation calculation section 30, 4 ₁ is a composite image output from the image synthesizing unit 40.

The image storage unit 20, for example, at a certain predetermined field angle, and to a different time (e.g., per two frames, and so three frames) are sequentially input to store a plurality of images 1 ₁ in units of frames taken in. The image correlation calculation unit 30 reads out, for example, two images 2 ₁ and 2 ₂ that are temporally different from the image storage unit 20, performs correlation processing between the two images, and calculates image correlation data 3 ₁ . . Here, for example, the image 2 ₁ as a reference image. Further, the sum of absolute differences between frames between the image 2 ₁ and the image 2 ₂ is calculated as the image correlation data 3 ₁ . The image synthesizing unit 40, image correlation data calculated by the image correlation calculation unit 30 3 ₁ pre-specified threshold (This is referred to as "correlation threshold") is compared with J1, images based on the comparison result Perform the synthesis process. Here, when the image correlation data 3 ₁ is equal to or less than the correlation threshold value J1, the images 2 ₁ and 2 ₂ are regarded as having a high correlation, and the two images are combined. On the other hand, when the image correlation data 3 ₁ is larger than the correlation threshold value J1, the image 2 ₁ and the image 2 ₂ are regarded as having a low correlation, and the image composition unit 40 does not perform the composition processing of the two images. As an image composition process in the image composition unit 40, an image averaging process is performed. That is, when the correlation between the image 2 ₁ and the image 2 ₂ is high, the addition averaging process is performed between the pixels at the same pixel position in the image 2 ₁ and the image 2 ₂ as shown in FIG. Thereby, random noise mixed in each of the image 2 ₁ and the image 2 ₂ is averaged and suppressed. Further, in the correlation image with a high 2 ₁ and the image 2 _2, because a averaging process using the pixels at the same pixel position, spatially without being affected by the pixels in the neighborhood, to maintain the resolution It becomes possible.

In the above example, in FIG. 1, the parallel transmission of the image 2 ₁ and the image 2 ₂ is performed between the image storage unit 20 and the image correlation calculation unit 30, but the image 2 ₁ , image 2 ₂ may be sequentially transmitted in time series. Further, regardless of the number of data bus between the image storage unit 20 and the image correlation calculation section 30, the image data handled image 2 ₁ and the image 2 not only ₂ of the two images, three or more temporally Different images may be handled. In this case, for example, if the data bus between the image storage unit 20 and the image correlation calculation unit 30 as a single, dealing with five image 2 ₁ - image 2 ₅ temporally different, the reference image example image 2 ₁ , image correlation data 3 ₁ (data obtained by indexing the correlation between reference image 2 ₁ and image 2 ₂ , the same applies hereinafter), image correlation data 3 ₂ , and the like as correlations with other images 2 ₂ to 2 ₅ . Image correlation data 3 ₃ and image correlation data 3 ₄ are calculated, and each image correlation data is compared with the correlation threshold value J1. Then, in the case of any image correlation data also below the correlation threshold J1, by performing averaging processing of the image 2 ₁ - image 2 ₅ all images to calculate a composite image. If only the image correlation data 33 is larger than the correlation threshold value J1, the composite image is calculated by performing the averaging process of the remaining four images 2 ₁ , 2 ₂ , 2 ₃ , 2 ₅ excluding the image 2 _4. do it.

In the above example, when calculating the image correlation data, the sum of absolute differences between frames of the target image is calculated. In this case, the absolute value of the inter-frame difference is calculated after combining the average values of the target images. A sum of values may be calculated. Also, the sum of absolute differences between frames may be calculated after adding an offset to each image so that the maximum values of the target images are matched, or conversely, the minimum value of each of the target images. The sum of absolute differences between frames may be calculated after adding an offset to each image so as to match. Alternatively, the sum of absolute differences between frames may be calculated after adding an offset to each image so that the median values of the target images are matched. Furthermore, the sum of absolute differences between frames may be calculated after adding an offset to each image so that the target image matches the dynamic range of the reference image, or by multiplying the dynamic range by multiplying the weighting coefficient. Further, the images of the image 2 ₁ , the image 2 _2, etc. are added and averaged as they are using the pixels at the same pixel position, but the above averaging processing is performed after the above processing is performed on the target image. It may be. That is, when calculating the image correlation data, the image correlation data may be calculated after matching the target image as much as possible with the characteristics of the reference image. By suppressing the level as much as possible, the identity of the image pattern can be evaluated.

In the above example, the interframe difference absolute value sum is calculated as the image correlation data. However, as an alternative calculation value, the interframe difference square sum, the interframe difference absolute maximum value, and the interframe difference absolute value are calculated. You may use the frequency | count of the maximum value, the value which expressed the correlation of the image used as object quantitatively, or the value which combined these.
Further, the value of the correlation threshold value J1 of the image correlation data may be arbitrary. For example, when the noise is strong and the image correlation data is large, the image synthesis process can be performed without excluding the target image by increasing the correlation threshold J1. The more images to be combined, the more random noise mixed in the image is suppressed, and the pixel value of the combined image tends to approach the true value, but this is a normal distribution or a normal distribution centered on the true value. This is because it is considered that there is a distribution tendency that conforms.
In the above example, the image 2 ₁ as a reference image, other images may be used as the reference image. For example, various methods such as setting the first image in the time series as the reference image, or setting the last image in the time series as the reference image are possible, but the method of determining the reference image is not limited. The same applies to the other embodiments described below.

  As described above, according to the first embodiment, a plurality of frames-by-frame images taken at different times are stored in the image storage unit, and one of a plurality of neighboring images at different times read from the image storage unit is stored. One image is used as a reference image, correlation processing between the reference image and other images is performed to calculate image correlation data, and the calculated image correlation data is compared with a predetermined correlation threshold value. Only when the correlation data is equal to or less than the correlation threshold, the reference image and the image other than the reference image are combined. Therefore, since the blurring (smoothing) processing as in the prior art is not used, random noise can be suppressed without impairing the resolution, or the resolution can be improved by noise suppression.

Embodiment 2. FIG.
In Embodiment 1 described above, image correlation and image composition processing are performed in units of frames, but this is a still image in which the input image is input in time series, and the pixel position of the image changes within the screen. It is effective when there is not. On the other hand, the input image is not always a still image, and a moving object may appear in a part of the screen. In such a case, the video of the moving object appears at different positions in each screen input in time series. In the second embodiment of the present invention, a method for suppressing random noise by combining a plurality of images in which a moving object is partially reflected in this way will be described.
FIG. 3 is a block diagram showing a functional configuration of an image processing apparatus according to Embodiment 2 of the present invention.
In FIG. 3, the image processing apparatus according to the second embodiment includes an image storage unit 20, a region search unit 50, and an image composition unit 60. Further, 5 ₁ region information data, 5 ₂ image correlation data, ₆₁ represents a composite image output from the image synthesizing unit 60. Note that the same reference numerals as those in FIG. The area search unit 50 includes a function corresponding to the image correlation calculation unit 30 in FIG. Further, in the image composition unit 60 of the second embodiment, the image composition processing is performed by adding and averaging the target images.

The area searching unit 50, first, the image 2 ₁ and the image 2 ₂ series when read from the image storage unit 20, the highest correlation region and target region moving object reflected in the image 2 _1, reference picture 2 ₂ searched from, generates the area information data 5 ₁ indicating the highest correlation area.
For the search of the target area, the reference image (image 2 ₁ ) is divided into a plurality of image blocks (rectangular blocks), and the image block having the smallest evaluation value (in this example, the sum of absolute differences) is selected as the image 2. Search from ₂ . The search is sequentially performed for each image block, and the motion vector distribution of each block in the screen is grouped. The same vector value or vector values within a specified range are grouped, and a grouping that satisfies the condition of the vector value range is set as an image synthesis target. At this time, the area information data 5 ₁ represent the grouping, [x1, y1] ~ rectangular area and that [x2, y2] [x11, y11] ~ [x12, y12], [x21, y21] ~ [x22 , Y22], [x31, y31] to [x32, y32].

Region searching unit 50, then the image 2 ₁ and the image 2 ₂ by performing correlation processing between images to calculate the image correlation data 5 _2. The grouped target area correlation data is the sum of the evaluation values (sum of absolute differences) of each block in the target area. For example, in a target region where the combined range of rectangular ranges [x11, y11] to [x12, y12], [x21, y21] to [x22, y22], [x31, y31] to [x32, y32] meets the conditions. some cases, the sum of the three blocks evaluation value image correlation data 5 _2.
The image synthesizing unit 60, when image correlation data 5 ₂ correlation threshold J2 hereinafter designated in advance, extracts the target area from the image 2 ₂ on the basis of the area information data 5 _1, corresponding image 2 ₁ is a reference image Perform the averaging process with the area to be processed.

In the above example of the second embodiment, the correlation data of the grouped target regions is the sum of the evaluation values (inter-block difference absolute value sum) of each block in the region search. The difference sum of squares in the target area, the maximum difference absolute value in the target area, the frequency of the maximum difference absolute value in the target area or the value that quantitatively represents the correlation between the target area images, and the difference between the other two images Values expressed quantitatively or a combination of these may be applied.
In the above example, only the grouped target area is subjected to the image composition process. However, the image composition process may be performed in addition to the grouped target area. For example, as shown in FIG. 4, the image of the compositing target images 2 _1, image 2 _2, an image 2 _3, the image 2 ₁ as a reference image, image 2 _1, image 2 _2, image 2 ₃ screen Assume that there are target areas A1, A2, and A3 that are images of the same moving object. In this case, the image 2 in ₁ and target area A1 and the target area A2 of the image 2 in the ₂ and the target region A3 of the image 2 ₃ with the synthesis (averaging process), the image 2 ₁ in other than the target area A1 and the region, the image 2 in the _second and the region other than the "target area A2 and the area of the target area A1 of the image 2 in _1", the image 2 in the ₃ 'region of the target area A1 of the object area A3 and the image 2 in ₁ " Image composition with other regions may be performed. Furthermore, in this case, if there are 3 pixels at the same position that are not excluded in the images 2 ₁ , 2 ₂ , and 2 ₃ (there are 3 images, 3 pixels is the maximum), the 3 pixels at the same position are combined (added average) 2), the image 2 ₁ , the image 2 ₂ , and the image 2 ₃ are two identical pixels excluding the target areas A1, A2, and A3 in each image and the area corresponding to the target area A1 in the image 2 ₁ . (In FIG. 4, since the moving target area has a small amount of movement, and none of the image 2 ₁ , the image 2 ₂ , and the image 2 ₃ has pixel positions corresponding to the target areas A1, A2, and A3, at least two pixels A background image other than the target region is obtained), and two pixels at the same position are synthesized.

  As described above, according to the second embodiment, the region having the highest correlation with the target region in which the moving object is reflected on the reference image is searched from the image other than the reference image and output as region information data. Image correlation data is calculated by performing correlation processing between images other than the reference image and the calculated image correlation data is lower than a correlation threshold specified in advance. If the region with the highest correlation on the image other than the reference image indicated by the data is combined, and the calculated image correlation data is higher than the specified correlation threshold, the image is obtained by removing the target region from the reference image. Then, an image obtained by excluding a region having the highest correlation from images other than the reference image is synthesized. Therefore, even when there is an object to be moved in the screen, the pixel position of the object to be moved in the reference image and the object to be moved in the image other than the reference image can be aligned. Even in a certain region, the resolution can be maintained to suppress random noise, or the resolution can be improved by suppressing noise.

Embodiment 3 FIG.
FIG. 5 is a block diagram showing a functional configuration of an image processing apparatus according to Embodiment 3 of the present invention.
5, the image processing apparatus according to the third embodiment includes an image storage unit 20, an image correlation calculation unit 70, an image conversion unit 80, a region search unit 90, and an image composition unit 100. 7 ₁ is image correlation data, 8 ₁ is image transformation parameter information, 8 ₂ is a transformed block image (affine transformation block image), 9 ₁ is area information data, 9 ₂ is image correlation data, and 9 ₃ is transformed image data. , 10 ₁ synthetic image data, 2 ₁₁ represents an image (affine transformation frame image) affine transformation frame image after the conversion. In addition, the part which attached | subjected the same code | symbol as FIG. 1 and FIG. 3 means the same thing. Similar to the area search unit 50 in FIG. 3, the area search unit 90 includes a function corresponding to the image correlation calculation unit 30 in FIG. 1.
Here is a description of image 2 ₁ temporally old image 2 ₂ for example of the reference image, it may be a new image in time to the reference image is a matter of course.

As a continuous image to be input, the entire screen is displayed in time series so that the entire screen moves from the front to the front, or the moving object approaches from the front to the front as shown in FIG. There is an image. In Embodiment 3 of the present invention, a method of suppressing random noise by combining a plurality of such images will be described.
Image correlation calculating unit 70, the image storage unit image 2 ₁ read different time from 20 and the image 2 _2, performs correlation processing of the image for each region by dividing the screen into a plurality of regions, the image correlation data 7 ₁ is calculated and given to the image conversion unit 80.
Here, there are cases where the entire screen moves from the front to the front or only the movement target B as shown in FIG. 7, but in any case, the movement area is compared with the reference image (image 2 ₂ ). The correlation between the target image (image 2 ₁ ) and the region at the same position tends to be low. This is because the image of the subject changes due to movement. Also, when a represents an image obtained by photographing the front from the vehicle of the car or the like 7 is advanced, the amount of movement of the appearance in the image 2 ₁ and image 2 ₂ and near infinity near the screen edge of the center of the screen Different. That is, the moving object B moves slowly as it is closer to the center of the screen, and tends to move faster as it approaches the screen edge. Therefore, if the object of interest is moving, because the position in the screen different, as shown in FIG. 7, the mobile object B at the position of the image 2 ₁ at a certain time, another time It has moved to the position of the image 2 _2, so that the shape of the moving object B changes reflected increased. At this time, the mobile object B as viewed from a vehicle in order seen in the image 2 ₁ and the image 2 ₂ position (angle) are different, similar in shape to the moving object B in the mobile in the image 2 ₁ target B and the image 2 ₂ It will not be.

Further, in FIG. 7, the entire screen in the image 2 ₁ and the image 2 ₂ is moving (e.g., if taken Aboard, actually to have moved is a vehicle itself, a vehicle The images outside the video taken from the image appear to move relatively), and the image 2 ₁ and the image 2 ₂ are correlated with the entire screen, or the screen is divided into a plurality of areas and the two images 2 ₁ and 2 correlation of the image 2 ₂ tends to decrease both. This is because the images 2 ₁ and 2 _2, which are outside images seen from the vehicle, are moving, so that the pixels at the same position are generally not the same target pixel unless they are flat and monotonous images, or This is because even the same object is a pixel at a different position in the pattern.

In the image conversion unit 80, a designated area (corresponding to the screen of the image 2 ₂ itself) in the image 2 ₂ determined from the image correlation data and used as a reference image, and an image area of the image 2 ₁ corresponding to the designated area. If the correlation varies depending on the position of the screen, based on the image transformation parameters _81, image conversion so as to match the image area of the image 2 ₁ image (reference image) to 2 ₂ (affine transformation), the converted image and it outputs the (affine transformation frame image) ₂₁₁ and the transform block image (affine transformation block image) 8 ₂ area searching unit 90.
As described in FIG. 7, the image 2 _1, image 2 ₂ is an image obtained by photographing the front from the vehicle, the image 2 ₁ image (reference image) to the image close to the front like the 2 ₂ the image of the image 2 ₁ single point in the broken line (image area corresponding to the designated area of the reference image) to affine transformation as shown in FIG. Dashed portions of the image (affine transformation frame image) 2 ₁₁ after the affine transformation by the target area B after the movement, which is photographed in the image 2 ₂ those described virtually actually the converted image 2 ₁₁ is not what exists.
Here the image transformation parameters ₈₁ for use in, for example, a horizontal and for the affine transformation coefficients for the vertical corresponding to the position of the screen. Note that the setting coefficient may be arbitrary. Set the coefficient according to the distance from the screen center (infinitely far point) within the screen (the apparent distance on the screen, not the distance in the real space), or the distance within the screen Four areas divided into dotted lines as shown in the diagonal lines of FIG. 7 by categorizing and setting the same coefficient for the same category, categorizing areas in the screen and setting the same coefficient for the same category A coefficient may be set for each.

Next, FIG. 9 is a diagram comparing the image 2 ₂ is an image 2 ₁₁ and the reference image after conversion. The solid line in the rectangular area in the central area of the image 2 ₂ is an area that has seen an image 2 _2, a region did not appear too small in the image 2 ₁₁ after conversion. Although FIG. 9 (right) shows a result of comparing the images 2 ₁₁ and the image 2 ₂ after conversion, relatively differences greater is the dashed region portion. That is, the part of the movement target B accompanying the movement (even if the part overlaps in the broken line area, the relative position in the area is different, so the difference is also relatively large in the overlapping area) and the image 2 _This is a rectangular area near the center that was newly seen in ₂ .

The area searching unit 90, the image 2 ₁₁ after conversion to image converted by the image converting unit 80, searches for a region to be image compositing target. In this case, the image 2 ₂ which is the reference image is divided into a plurality of image blocks area, from the image 2 ₁₁ after the image conversion a small image blocks of (a sum of absolute differences in the present example) the most evaluation value in blocks Explore. In 9, when the target B within the image 2 ₁₁ after the image conversion is moved in the image (reference image) 2 _2, an image block of the image 2 ₁₁ identical pixel position and the target B of the image 2 in the ₂ When the evaluation value is calculated, since the target B is moving, the difference increases and the evaluation value increases. Therefore, the image 2 for determining the object corresponding to the target B within ₂ from the image 2 _11, image 2 sequentially changing the pixel position of the block in ₁₁ image 2 in block evaluation value at a target B in ₂ (difference (Absolute value sum) is calculated. If the target B own image 2 in ₂ can not be specified, by dividing the image 2 ₂ block area, the search from the image 2 ₁₁ after the image conversion block which has the smallest evaluation value for each block of the image 2 ₂ And ask. At this time, the coordinates in the screen of each block of the image 2 ₂ (pixel position) and when the coordinates in the screen of the search block of the image ₂₁₁ matches, the target block is not moved block (hereinafter, a still Block).

On the other hand, if the coordinates of the image 2 blocks images 2 ₂ of the corresponding block and the screen in the image 2 ₁₁ that corresponds to the block including the subject B in ₂ do not match, the object B is seen to move . Therefore, whether stationary block for each block, sets a moving block of information as the area information data 9 _1. Also, setting the minimum evaluation value of each block determined by the block search the image correlation data 9 _2. Further, to set the image of the block unit as the converted image data 9 _3. At location connected downstream (the image combining unit 100), the converted image data 9 ₃ or the moving block by the area information data 9 _1, or a still block is found. Area information data 9 ₁ , image correlation data 9 ₂ , and converted image data 9 ₃ are prepared for the number of blocks (in H / W, registers and memory images, and in S / W, arrays and pointer images).
In addition, the shape of the block to be divided may be arbitrary. Further, in the above example, the image 2 ₂ but evaluation values for image 2 ₁₁ after the image converted into the block is divided into block areas is searched block having the smallest image 2 ₁₁ to the block area in the opposite divided and may be searching for a block which has the smallest evaluation value to the image 2 ₂ to the block, is not limited which image should be searched in block division.

In FIG. 9, since the moving target B is trapezoidal, the shape may be different from the screen-divided block. However, the correlation degree is evaluated without depending on the relationship between the moving target B and the shape of the screen-divided block. Since (evaluation value calculation) can be performed, this invention is not limited. When the block shape is rectangular and the moving object B is trapezoidal, for example, the moving object is composed of a block area composed of an affine transformation block image 8 ₂ including the moving object B composed of a plurality of rectangular blocks and an image (reference image) 2 _2. When a difference is taken with a block area corresponding to the block area including B, the difference tends to increase for the protruding area other than the movement target B. This is because the moving object B is moving, and when the image is cut into the block area, an image with another pattern is also cut out.

The image combining unit 100, obtained by the area searching unit 90 to the mobile block area or a still block area corresponding to the area information data 9 _1, the image 2 ₂ is converted image data 9 ₃ and the reference image, the image based on the value of the correlation data 9 ₂ Compose (averaging process). When the image correlation data 9 ₂ is smaller than the correlation threshold value J3 designated in advance for each of the moving block region and the still block region, the converted image data 9 ₃ and the corresponding block in the image 2 ₂ are regarded as having a high degree of correlation (for example, The object B and background searched by the area search unit 90 have a high degree of correlation), and the converted image data 9 ₃ and the image 2 ₂ are synthesized. On the other hand, image correlation data 9 ₂ for moving the block area or a still block area correlation threshold J3 larger area previously specified for each does not perform image synthesis. This larger area correlation threshold J3 is converted image data 9 ₃ and image 2 low correlation with the corresponding block in the _2, likely to be different images are synthesized by the image synthesis processing, noise reduction and resolution reduction suppression This is because it cannot fulfill its original purpose.

In the above example, the image 2 ₁ and has two frames of the image 2 ₂ subjected to affine transformation processing and image synthesizing processing using the reference image, may be used an image of three or more frames.
The image transformation parameter information 8 _1, affine transform coefficients, affine transform coefficients set, in addition to the affine transformation coefficient candidates corresponding to the moving target region, the position information of the moving object area, the relative amount of movement of the moving object area and the corresponding affine A conversion coefficient or the like may be included.
In addition, the image conversion unit 80 and the area search unit 90 dealt with blocks obtained by dividing an image into a plurality of blocks, but the number of divisions of blocks may be arbitrary, and the shape of the blocks is not limited to a rectangle, and other shapes such as a triangle It may be.
In addition, the affine transformation may be performed by limiting only the necessary area, not the entire screen, and the type, order, and area of the affine transformation are not limited.
In the above example, the correlation threshold J3 is a value specified in advance for each moving block area or a still block area, as shown in another example of FIG. 6, the image synthesizing unit area information data 9 ₁ 100 In this case, the image composition may be performed based on a predesignated value without depending on the moving block area or the still block area.

Further, in the above example, there is one moving object as shown in FIGS. 7, 8, and 9, but the same can be performed even when there are a plurality of moving objects.
Furthermore, in the above example, the sum of the evaluation values (inter-block difference absolute value sum) of each block in the area search is calculated as the correlation data of the movement target area. Quantitative sum, maximum difference absolute value within the movement target area, frequency of maximum difference absolute value within the movement target area or a value that quantitatively represents the correlation between the movement target area images, and the difference between the other two images Or a combination of these values.

  As described above, according to the third embodiment, the first image correlation data is calculated by the correlation process between the reference image and an image other than the reference image in a plurality of temporally adjacent images, and the first image When the correlation between the designated area in the reference image and the image area of the image other than the reference image corresponding to the designated area is different depending on the position in the screen as judged from the correlation data, the target area of the image other than the reference image is determined as the reference image. The image is converted so as to match the designated area, and the image area having the highest correlation with the movement target area in the reference image is searched from the converted image, and the converted image data and the movement target area are searched for each search unit. Only when the region information data indicating whether or not and the second image correlation data by the correlation process between the converted image and the reference image is calculated and the second image correlation data is smaller than the correlation threshold specified in advance, Above The conversion image data are to be combined with the corresponding regions of the reference image. Therefore, even when the entire screen changes and there are targets that move within the screen, random noise can be suppressed while maintaining the resolution for the moving target and the background area other than the moving target, or noise suppression The resolution can be improved.

Embodiment 4 FIG.
In the first to third embodiments, the averaging process is performed as the image synthesis process. However, instead of this, an adding process, a weighted addition process, a weighted average process, or a filter process may be performed.
The addition process may be performed when a low-illuminance image is synthesized. Although pixel values are distributed in a low pixel level range due to low illuminance, random noise can be suppressed and the luminance level range can be expanded by performing addition processing.
As described in the second embodiment and the third embodiment, when the number of pixels to be partially combined due to the movement of the target region is different, it can be dealt with by performing a weighted addition process. That is, when adding the pixels at the same position with three pixels to three images, the addition is performed as it is, and when adding one pixel excluding one pixel, the addition result is multiplied by 3/2. Thereby, the range of the pixel level with the periphery matches. In addition, each pixel to be added may be weighted for addition. Further, instead of simple averaging, weighted average processing may be performed in which each pixel to be averaged is weighted and averaged.
In the third embodiment, the image conversion performed by the image conversion unit 80 is affine transformation. However, enlargement, reduction, rotation, vertical and horizontal movement, inversion, or a combination thereof may be used.

  As described above, according to the fourth embodiment, the addition averaging process, the addition process, the weighted addition process, the weighted average process or the filter process is performed as the image synthesis process, and the enlargement, reduction, rotation, Since the affine transformation of either up / down / left / right diagonal movement, inversion, or a combination of these is performed, even if the entire screen changes and there is an object to move within the screen, the object to be moved and other than the object to be moved In the background area, random noise can be suppressed while maintaining the resolution, or the resolution can be improved by noise suppression.

Embodiment 5 FIG.
FIG. 10 is an explanatory diagram showing an image composition method of the image composition unit according to Embodiment 5 of the present invention.
In FIG. 10A, an image 2 ₁ , an image 2 ₂ ,..., An image 2 _N are input images that are temporally different and represent N images to be subjected to image synthesis. Here, the image 2 ₁ the reference image. In FIG. 10B, the composite image [N] is obtained by converting N images to be composited in FIG. 10A by any of the processes described in the first to fourth embodiments. An image synthesized by using the image is represented.

In the image composition unit (the image composition unit 40 in FIG. 1, the image composition unit 60 in FIG. 3, or the image composition unit 100 in FIG. 5), the absolute difference between the reference image (image 2 ₁ ) and the calculated composite image [N]. The sum is calculated as an evaluation value, and the calculated evaluation value is compared with a predetermined threshold value (this is referred to as a “pixel shift threshold value”) J4, and only when the pixel shift threshold value J4 is less than or equal to the composite image [N] Is output. This is for examining whether the composite image N has a pixel level shift equal to or greater than random noise. When the random noise of the composite image [N] is reduced or removed, the evaluation value based on the reference image (image 2 ₁ ) and the composite image N can be regarded as an evaluation value of random noise and shows a low value. When the pixel shift occurs due to the above, and the shift more than the random noise occurs, the evaluation value becomes large. Therefore, it is possible to determine whether or not the composite image N is appropriate by comparing the evaluation value with the pixel shift threshold J4. The pixel shift threshold J4 may be any value as long as it is within a range that can be tolerated as pixel shift other than random noise and random noise.

  As described above, according to the fifth embodiment, the image composition unit calculates the evaluation value of the composite image generated using N images and the reference image, and the evaluation value specifies the pixel shift threshold value J4 designated in advance. Since the composite image is output only in the following cases, the resolution can be maintained to suppress random noise, or the resolution can be improved by suppressing noise.

Embodiment 6 FIG.
FIG. 11 is an explanatory diagram showing an image composition method of an image composition unit according to Embodiment 6 of the present invention.
In FIG. 11A, an image 2 ₁ , an image 2 ₂ ,..., An image 2 _{N + 1} are input images that are temporally different and represent N + 1 images to be subjected to image composition. Here, the image 2 ₁ the reference image. In FIG. 11B, the composite image (second image) [N + 1] is the target of the composite of FIG. 11A by any of the processes described in the first to fourth embodiments. An image synthesized using N + 1 images is represented. In the sixth embodiment, similarly to the fifth embodiment, a composite image (first image) [N] generated using N images is also used.

In the image composition unit (the image composition unit 40 in FIG. 1, the image composition unit 60 in FIG. 3, or the image composition unit 100 in FIG. 5), first, the difference between the reference image (image 2 ₁ ) and the first composite image [N]. The absolute value sum is calculated as the evaluation value [N], and the difference absolute value sum between the reference image (image 2 ₁ ) and the second composite image [N + 1] is calculated as the evaluation value [N + 1] _ev . Next, an absolute difference value between the calculated evaluation value [N] _ev and the evaluation value [N + 1] _ev is calculated. Then, the absolute value of the difference between the calculated evaluation values is compared with a predetermined threshold value (this is referred to as “convergence threshold value”) J5, and when it is equal to or less than the convergence threshold value J5, the composite image [N] or [N + 1] is determined. Output. This is for checking that the evaluation value [N] _ev and the evaluation value [N + 1] _ev are close to each other and the evaluation value of the composite image is converging (saturating). If it is not always efficient to converge to the theoretical limit in order to reduce the processing time and reduce the size of the H / W to be configured while maintaining the performance, it is terminated as described above. It is more efficient to set a reference and interrupt the process.

In the above example, the difference process is calculated by the absolute difference value, but it may be a difference process with a sign and is not particularly limited. In this case, if the difference result is a positive value and smaller than the convergence threshold J5, the first composite image [N] or the second composite image [N + 1] is output, while if the difference result is a negative value, the evaluation is performed. since the value [N + 1] _ev means that has become larger than the evaluation value [N] _ev, outputs towards the minimum evaluation value [N] first composite image corresponding to _ev [N].

  As described above, according to the sixth embodiment, the image composition unit generates the first composite image [N] using N images and the second composite image using N + 1 images. [N + 1] is generated, and for the first composite image [N] and the second composite image [N + 1], an evaluation value of the composite image is calculated between each of the reference image and the evaluation value of the first composite image. And the evaluation value of the second composite image is obtained, and if the difference between the evaluation values is equal to or less than the convergence threshold value J5 specified in advance, the first composite image [N] or the second composite image [N + 1] is output. Thus, the resolution can be maintained to suppress random noise, or the resolution can be improved by suppressing noise.

Embodiment 7 FIG.
12 and 13 are explanatory diagrams showing an image composition method of the image composition unit according to the sixth embodiment of the present invention.
In FIG. 12A, an image 2 ₁ , an image 2 ₂ ,..., An image 2 _{N + i} are input images that are temporally different, and represent N + i images that are targets of image composition. In FIG. 12B, the composite image [N + i] is synthesized using the N + i images in FIG. 12A by any of the processes described in the first to fourth embodiments. Represents an image. On the other hand, in FIG. 13A, an image 2 ₁ , an image 2 ₂ ,..., An image 2 _{N + i + 1} are input images that are temporally different, and N + i + 1 images to be subjected to image composition. Represents. In FIG. 13B, the composite image [N + i + 1] is synthesized using the N + i + 1 images in FIG. 13A by any of the processes described in the first to fourth embodiments. Represents an image.
In the seventh embodiment, even when the difference between the evaluation values of the composite image [N] and the composite image [N + 1] in the sixth embodiment is equal to or greater than the convergence threshold value J5 specified in advance, the output can be performed. A method for regenerating a composite image will be described.

In the image composition unit (the image composition unit 40 in FIG. 1, the image composition unit 60 in FIG. 3, or the image composition unit 100 in FIG. 5), the reference image (image 2 ₁ ) and the first composite image [N ] _Is calculated as an evaluation value [N] _ev , and the difference absolute value sum of the reference image (image 2 ₁ ) and the second composite image [N + 1] shown in FIG. 11 is evaluated as an evaluation value [N + 1]. Calculate _ev . Next, an absolute difference value between the calculated evaluation value [N] _ev and the evaluation value [N + 1] _ev is calculated. Then, the absolute difference value of these evaluation values is compared with the specified convergence threshold value J5, and if it is equal to or greater than the convergence threshold value J5, the image is increased by one to generate the composite image [N + i] shown in FIG. Then, the sum of absolute differences between the reference image (image 21) and the composite image [N + i] is calculated as an evaluation value [N + i] _ev . At this time, i = 2. Next, the absolute value of the difference between the previously calculated evaluation value [N + 1] _ev and the _currently calculated evaluation value [N + i] _ev is calculated, and if the absolute value of the difference between the evaluation values is equal to or less than the specified convergence threshold value J5, The image [N + 1] or [N + i] is output.

On the other hand, if the difference absolute value between the evaluation value [N + 1] _ev and the evaluation value [N + i] _ev is equal to or greater than the specified convergence threshold J5, the number of images to be combined is further increased by one, and the combined image shown in FIG. [N + i + 1] is generated and the sum of absolute differences between the reference image (image 2 ₁ ) and the composite image [N + i + 1] is calculated as the evaluation value [N + i + 1] _ev , and then the evaluation value [N + i] _ev and the evaluation value [N + i + 1] _ev The absolute difference value of is calculated. If the absolute value of the difference between the calculated evaluation values is equal to or greater than the specified convergence threshold value J5, “i” is sequentially incremented by 1 to calculate two evaluation values that are different from each other by one, and the two evaluation values are calculated. The absolute value difference and the convergence threshold J5 are compared to determine whether to output a composite image. In other words, the composite image output determination is not performed by only two combinations of N and N + 1, or N + 1 and N + 2, but the composite image number N may be an arbitrary number, and two evaluation values that are different from each other are obtained. It is only necessary to calculate and compare the absolute difference between the two evaluation values and the convergence threshold J5 to determine whether to output a composite image.

In the above example, the difference process is calculated based on the absolute difference value, but it may be a difference process with a sign and is not particularly limited. In this case, when the difference result is positive and smaller than the threshold value, the composite image [N + i] or the composite image [N + i + 1] is output from the image composition unit, and when the difference result is a negative value, the evaluation value [N + i + 1] _ev is the evaluation value. [N + i] minimum evaluation value it means that has become larger than _ev the [N + i] corresponding composite image _ev [N + i] is output from the image synthesizing unit.
Moreover, the above example is an example for realizing as an apparatus, and it is not always efficient to converge to the theoretical limit in order to shorten the processing time and reduce the size of the H / W to be configured while maintaining the performance. In such a case, it is more efficient to set the discontinuation reference as described above to interrupt the processing.

  As described above, according to the seventh embodiment, in the image composition unit of the sixth embodiment, the evaluation value of the first composite image using N images and the second using the N + 1 images. When the difference from the evaluation value of the combined image is larger than the specified convergence threshold J5, the number of images is increased by one and a third combined image is generated using N + 2 images, and the third image is compared with the reference image. The evaluation value of the composite image is calculated, the difference between the evaluation value of the second composite image and the third composite image is obtained, and if the difference between the evaluation values is equal to or less than the convergence threshold J5, the second composite image or the third composite image is obtained. On the other hand, when the difference between the evaluation values of the second and third synthesized images is larger than the convergence threshold J5, the same synthesized image is generated by increasing the number of images by one and generating the next synthesized image. The difference between the evaluation values of the composite image is equal to the convergence threshold value J5. Since the greater was repeated sequentially same process, while maintaining the resolution can be suppressed random noise, or it is possible to improve the resolution by the noise suppression.

Embodiment 8 FIG.
In the first to seventh embodiments, the sum of absolute differences is used as correlation data and evaluation values. However, these correlation data and evaluation values are sums of squares of differences, or sums of absolute differences, or Any calculated value may be used as long as it is a value that quantitatively represents the difference between the two images, and is not particularly limited. This is because the degree of difference between two images can be expressed quantitatively, and the degree of similarity and correlation need only be determined based on the magnitude of the value.
As described above, according to the eighth embodiment, the evaluation value of the composite image is the sum of absolute differences, the sum of squared differences, or a value that quantitatively represents the difference between the two images. The degree of difference can be expressed quantitatively, and the degree of similarity can be determined by the magnitude of the value. By applying this to the first to seventh embodiments, the resolution can be maintained and random noise can be maintained. It can be suppressed, or the resolution can be improved by noise suppression.

Embodiment 9 FIG.
In the first to eighth embodiments, when N images are used for synthesis, N pixels at the same position are synthesized as they are, or a moving target area is a reference image. Except for pixels in the target area other than the above, image synthesis is performed for pixels corresponding to the same position. An image taken in a bad shooting condition such as low illuminance contains a lot of noise. In order to reduce or eliminate this noise, image synthesis is performed by the method of the first to eighth embodiments.

In the ninth embodiment, the image synthesizing unit further selects pixels to be synthesized. That is, when synthesizing pixels at the same position in three or more images, the synthesizing process is performed excluding either the maximum value or the minimum pixel value of the pixels at the same position in each image. In the case of this combining process, since the pixels that are extremely far from the true value of the pixels are excluded, the influence of noise can be suppressed. For example, when the pixel level value is concentrated at low illuminance, the pixel value of the composite image can be prevented from shifting from the true value to the higher value by excluding the maximum value pixel.
In this example, the composition processing is performed by excluding either the maximum value or the minimum pixel value of the pixel at the same position in each image, but both the maximum value and the minimum value of the pixel at the same position are both performed. You may exclude and perform a synthetic | combination process.

  As described above, according to the ninth embodiment, the maximum value and the minimum value of the pixels at the same position in each image are combined when the image composition unit performs the composition processing of the pixels at the same position in three or more images. Since one of the pixel values, or both the maximum value and the minimum value of the pixel at the same position are excluded and the combining process is performed, the amount of deviation from the true value of the pixel value of the combined image is suppressed, Random noise can be suppressed while maintaining the resolution, or the resolution can be improved by suppressing noise.

Embodiment 10 FIG.
In the ninth embodiment, in three or more images, either the maximum value or the minimum pixel value of the pixel at the same position of each image or both the maximum value and the minimum value of the pixel at the same position are excluded. In the tenth embodiment, pixels to be combined are selected by the following method.
In the image composition unit, a first weighted average and standard deviation of pixels at the same position are calculated for N images to be synthesized, and a first weight is calculated from the pixels at the same position of the N images. The synthesis process is performed using the remaining pixels excluding pixels whose average is different from the value obtained by multiplying the standard deviation by the coefficient. In the processing by this method, as in the ninth embodiment, since the pixels that are extremely far from the true value of the pixels are excluded, the influence of noise can be suppressed.

In N synthesis target images, when P is a pixel at the same position, AVEw is a first weighted average, SD is a standard deviation, and w ₁ and w ₂ are coefficients to be multiplied by the standard deviation, for example, N images Pixels that meet the following conditions are excluded from pixels at the same position.
1) Pixel P <AVEw− (SD × w ₁ ) (pixel P <AVEw) at the same position
2) Pixel P> AVEw + (SD × w ₂ ) (Pixel P> AVEw)
Here, for example, if both “AVEw− (SD × w ₁ )” and “AVEw + (SD × w ₂ )” are within the pixel value range, w ₁ = w ₂ can be used to move the object up and down from AVEw. However, when the pixel level value is concentrated in a low illuminance and low pixel level value, particularly, “AVEw− (SD × w ₁ )” is the pixel value range. When the value is below the lower limit, the coefficient multiplied by the standard deviation is set to a different coefficient when the value is below the average value and above, and the coefficient when the value is above the average value is made smaller (w ₁ > w ₂ ). The pixel value of the image is prevented from shifting from the true value to the higher value. At this time, if w ₂ = (AVEw−pixel value level lower limit value), the range above and below AVEw is just the target.
In this example, the values of the coefficients w ₁ and w ₂ are set so that the range above and below the AVEw is exactly the target. However, in order to suppress the influence of noise even when the range above and below the AVEw is not exactly the target. It is only necessary to exclude pixels that are extremely far from the true value of the pixels, and it is only necessary to prevent the pixel value of the composite image from deviating from the true value.

  As described above, according to the tenth embodiment, the image composition unit calculates the first weighted average and standard deviation of pixels at the same position for the N images to be processed, and the N images Since pixels having a value that is different from the value obtained by multiplying the standard deviation by the coefficient of the standard deviation are excluded from the pixels at the same position, the synthesis process is performed using the remaining pixels. The amount of deviation from the true value can be suppressed, the resolution can be maintained and random noise can be suppressed, or the resolution can be improved by noise suppression.

  As described above, each embodiment has been described with the image processing apparatus as a target. However, the processing functions of each unit performed by these image processing apparatuses are configured by replacing the processing of a program (image processing program) executed on a computer. It can be done. Therefore, according to the present invention, by executing the same processing as the image processing apparatus of each of the above embodiments as an image processing program, the resolution is maintained and random noise is suppressed, or the resolution is improved by noise suppression. Make it possible.

  20 image storage unit, 30 image correlation calculation unit, 40 image synthesis unit, 50, 90 region search unit, 60 image synthesis unit, 70 image correlation calculation unit, 80 image conversion unit, 100 image synthesis unit.

Claims (14)

An image storage unit for inputting and storing a plurality of images taken in frames at different times;
An image correlation calculation unit that calculates one of a plurality of images at different times in the vicinity read from the image storage unit as a reference image, performs correlation processing between the reference image and an image other than the reference image, and calculates image correlation data;
An image synthesizer that compares the calculated image correlation data with a predetermined correlation threshold and synthesizes the reference image and an image other than the reference image only when the image correlation data is equal to or less than the correlation threshold; An image processing apparatus comprising: An image storage unit for inputting and storing a plurality of images taken in frames at different times;
One of a plurality of images at different times in the vicinity read from the image storage unit is set as a reference image, and an area having the highest correlation with a target area on which the moving object is reflected on the reference image is searched from images other than the reference image. An area search unit that outputs image correlation data by correlation processing between the reference image and an image other than the reference image, and outputs the area information data.
When the calculated image correlation data is lower than a correlation threshold designated in advance, an area having the highest correlation on an image other than the reference image indicated by the area information data is included in the target area of the reference image. On the other hand, when the calculated image correlation data is higher than the correlation threshold value, the region having the highest correlation is excluded from the image other than the reference image in the image obtained by removing the target region from the reference image. An image processing apparatus comprising: an image synthesis unit that synthesizes and outputs an image. An image storage unit for inputting and storing a plurality of images taken in frames at different times;
Image correlation calculation for calculating first image correlation data by using one of a plurality of neighboring images read from the image storage unit at different times as a reference image and performing correlation processing between the reference image and an image other than the reference image And
A predetermined image when a correlation between a designated area in the reference image and an image area of an image other than the reference image corresponding to the designated area is different depending on a position in the screen as determined from the first image correlation data An image conversion unit that converts an image area corresponding to an image other than the reference image based on a conversion parameter so as to match a designated area in the reference image;
Area information indicating whether or not the image area having the highest correlation with the movement target area in the reference image is searched from the converted image after the image conversion and converted image data is a movement target area for each search unit. An area search unit for calculating data and second image correlation data by correlation processing between the converted image and the reference image;
An image processing apparatus comprising: an image synthesis unit that synthesizes the converted image data with a corresponding region of the reference image only when the second image correlation data is smaller than a correlation threshold specified in advance.   The image processing according to any one of claims 1 to 3, wherein the image composition unit performs image composition by addition processing, weighted addition processing, addition averaging processing, weighted average processing, or filter processing. apparatus.   The image processing apparatus according to claim 3, wherein the image conversion unit performs image conversion in any one of enlargement, reduction, rotation, up / down / left / right oblique movement, or a combination thereof.   The image composition unit calculates the evaluation value of the composite image generated using N images and the reference image, and outputs the composite image only when the evaluation value is equal to or less than the specified pixel shift threshold. 6. The image processing apparatus according to claim 1, wherein the image processing apparatus is characterized in that:   The image composition unit generates a first composite image using N images, and generates a second composite image using N + 1 images. About the first composite image and the second composite image The evaluation value of the composite image is calculated with respect to each reference image, the difference between the evaluation value of the first composite image and the evaluation value of the second composite image is obtained, and the convergence threshold specified by the difference between the evaluation values 6. The image processing apparatus according to claim 1, wherein the first composite image or the second composite image is output when the following is true.   When the difference between the evaluation value of the first composite image and the evaluation value of the second composite image is larger than the designated convergence threshold, the image composition unit increases the number of images by one and uses the N + 2 images A composite image is generated, an evaluation value of the third composite image is calculated from the reference image, a difference between the evaluation value of the second composite image and the third composite image is obtained, and the difference between the evaluation values If the difference is less than the convergence threshold, the second composite image or the third composite image is output. On the other hand, if the difference between the evaluation values of the second and third composite images is larger than the convergence threshold, the image is increased by one. The same process is sequentially repeated when the difference between the evaluation values of the composite images is larger than the convergence threshold, such as determining whether to generate a next composite image and output a similar composite image. Item 8. The image processing apparatus according to Item 7.   9. The image correlation data and the evaluation value are sums of absolute differences, sums of squares of differences, or values that quantitatively represent a difference between two images. The image processing apparatus according to claim 1.   When performing the composition processing of the pixel at the same position in three or more images, the image composition unit is either the maximum value or the minimum value of the pixel at the same position in each image, or the pixel at the same position. The image processing apparatus according to claim 1, wherein the composition processing is performed by excluding both the maximum value and the minimum value.   The image compositing unit calculates a first weighted average and standard deviation of pixels at the same position for N images to be combined, and the first weighted average is calculated from the pixels at the same position of the N images. The image according to any one of claims 1 to 9, wherein a pixel having a value far from a value obtained by multiplying the standard deviation by a coefficient is excluded, and the synthesis process is performed using the remaining pixels. Processing equipment. A plurality of frames taken at different times are stored in the image storage unit,
A plurality of neighboring images at different times are read from the image storage unit, one of the plurality of images is used as a reference image, and correlation processing between the reference image and an image other than the reference image is performed to calculate image correlation data. ,
A process of comparing the calculated image correlation data with a pre-specified correlation threshold and synthesizing the reference image and an image other than the reference image only when the image correlation data is equal to or less than the correlation threshold. An image processing program executed by A plurality of frames taken at different times are stored in the image storage unit,
A plurality of images at different times in the vicinity are read from the image storage unit, and one of the plurality of images is used as a reference image, and an area having the highest correlation with a target area on which a moving object is displayed on the reference image is an image other than the reference image. Further searching and outputting as region information data, calculating image correlation data by correlation processing between the reference image and an image other than the reference image,
When the calculated image correlation data is lower than a correlation threshold designated in advance, an area having the highest correlation on an image other than the reference image indicated by the area information data is included in the target area of the reference image. On the other hand, when the calculated image correlation data is higher than the correlation threshold value, the region having the highest correlation is excluded from the image other than the reference image in the image obtained by removing the target region from the reference image. An image processing program for executing processing for synthesizing images on a computer. Input and memorize multiple frames taken at different times.
A plurality of images at different times in the vicinity are read from the image storage unit, one of the plurality of images is used as a reference image, and correlation processing between the reference image and an image other than the reference image is performed to obtain first image correlation data. Calculate
A predetermined image when a correlation between a designated area in the reference image and an image area of an image other than the reference image corresponding to the designated area is different depending on a position in the screen as determined from the first image correlation data Based on the conversion parameter, the corresponding image area of the image other than the reference image is image-converted so as to fit the designated area in the reference image,
Area information indicating whether or not the image area having the highest correlation with the movement target area in the reference image is searched from the converted image after the image conversion and converted image data is a movement target area for each search unit. Data, and second image correlation data by correlation processing between the converted image and the reference image,
Only when the second image correlation data is smaller than a correlation threshold designated in advance, the image processing program for executing the process of combining the converted image data with the corresponding region of the reference image on a computer.

Images