JP2011221680A - Image synthesizing method, image synthesis program and image synthesizer - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent strong reflection of background on a dynamic body part when continuous shooting images of a dynamic body with less move to a two-dimensional direction are synthesized.SOLUTION: A reference image is generated based on image data of a plurality of frames, on the other hand, change characteristics of pixel values of respective pixels of the frames are calculated, and it is determined whether the respective pixels are in a dynamic body main area with high frequency of existence of a dynamic body image or in a background main area with high frequency of existence of a background image based on the pixel value change characteristics. Then, when the pixels are considered as the ones in the dynamic body main area, a pixel value with large difference with a pixel value of the reference image is weakly blended, and on the other hand, when the pixels are considered as the ones in the background main area, the pixel value with large difference with the pixel value of the reference image is strongly blended.

Description

  The present invention relates to an image composition method, an image composition program, and an image composition apparatus that create a single still image by superimposing moving images and continuous shot images.

  Conventionally, there is a method of creating a single still image by superimposing moving images and continuous shot images. In the prior document 1 and the prior document 2, for a pixel at the same position on a plurality of image frames, a background image as a reference image is generated from the average value or median of the pixel values, and in the case of a pixel value close to the background image A blend rate is set low, pixel values that are far apart are set high, and a composite image is generated by adding pixel values multiplied by the respective blend rates.

  As a result, an image having a high appearance frequency is regarded as a part of the background and synthesized at a low blend rate throughout the entire frame, while a moving object having a low appearance frequency is pasted at a high blend rate.

  However, in the above conventional method, there is a case where an image with a small amount of movement of the subject in the two-dimensional direction cannot be combined well. For example, when the subject 100 is rotating like the baseball swing of FIG. 11, since the subject 100 overlaps most of the frame, taking the average value or the median value, the subject that is originally a moving object 100 is recognized as a background image. Then, for example, when the subject 100 is displaced by one frame or two frames and the real background image 101 is captured in the area in the case of five consecutive shots, the background image 101 is regarded as a moving object. As a result, since the background image 101 is strongly blended, there is a problem that the background image 101 is seen through the subject 100.

Registration No. 311469 Registration No. 4415198

  It is an object of the present invention to prevent a strong background from being reflected in a moving part when a continuously shot image of a moving object that moves little in a two-dimensional direction is synthesized.

  The present invention relates to a reference image generation step of generating a reference image based on image data of a plurality of frames, in an image composition method for generating a single image by combining a plurality of imaging frames, and each of the frames Based on the pixel value change characteristic calculation step for calculating the pixel value change characteristic of the pixel, and the pixel value change characteristic, the frequency of the moving object main region where the moving image exists frequently or the background image exists. A region determining step for determining whether the pixel is a high background main region, and if it is regarded as a pixel of the moving object main region, a pixel value having a large difference from the pixel value of the reference image is weakly blended, while the background main region is An image blending step for strongly blending pixel values having a large difference from the pixel value of the reference image when it is regarded as a pixel in the region.

  The present invention also provides a reference image generation step of generating a reference image based on the image data of the plurality of frames, in an image composition program for generating a single image by combining a plurality of imaging frames, A pixel value change characteristic calculating step for calculating a pixel value change characteristic of each pixel, and a frequency of existence of a background image or a moving object main region where each of the pixels is frequently present based on the pixel value change characteristic. A region determination step for determining whether the background main region is high and, if the pixel is considered to be a pixel of the moving body main region, a pixel value having a large difference from the pixel value of the reference image is weakly blended, while the background When it is regarded as a pixel in the main area, an image blending step for strongly blending a pixel value having a large difference from the pixel value of the reference image is executed on the computer. And characterized in that.

  Further, according to the present invention, in an image composition device that synthesizes a plurality of imaging frames to generate one image, a reference image generation unit that generates a reference image based on image data of the plurality of frames, Pixel value change characteristic calculating means for calculating a pixel value change characteristic of each pixel, and based on the pixel value change characteristic, a frequency of a moving object main region where a moving image exists frequently or a background image of each pixel A region determination means for determining whether the background main region is high, and if the pixel of the moving body main region is regarded as a pixel, the pixel value having a large difference from the pixel value of the reference image is weakly blended, while the background And an image blending unit that strongly blends a pixel value having a large difference from the pixel value of the reference image when the pixel is regarded as a pixel in the main area.

  According to the present invention, while generating a reference image based on image data of a plurality of frames, a change characteristic of a pixel value of each pixel of a frame is calculated, and each pixel is a moving object image based on the pixel value change characteristic. It is determined whether the main area of the moving object is frequently present or the main area of the background where the background image is frequently present. And when it is regarded as a pixel of the moving body main area, a pixel value having a large difference from the pixel value of the reference image is blended weakly, whereas when it is regarded as a pixel of the background main area, A pixel value having a large difference from the pixel value is strongly blended.

  Usually, even in an image in which moving objects overlap almost at the same position, the shape of the subject and its clothing changes due to the movement, so that the pixel value always changes throughout the entire frame. On the other hand, when a background with no motion is reflected in the majority of frames, the change in the pixel value of that portion is very small. By capturing the characteristics of these pixel value changes, it is possible to accurately determine a moving object image that was conventionally regarded as a background image. For pixels determined to be a moving object main region, for example, even if a background image is shown in only one frame, if the pixel value of the background image is significantly different from the pixel value of the moving object, it is blended weakly. It is possible to prevent the background image from being reflected in the moving body part. If there is no significant difference from the pixel value of the moving object, the blending is performed at the same rate as that of the moving object, so that there is no sense of incongruity.

  On the other hand, in the case of a pixel regarded as a background main region, for example, if a moving object is reflected in only one frame in all frames, the moving object is strongly blended, so that the moving object portion can be reliably combined on the background. Therefore, a more accurate continuous shot composite photograph can be generated.

It is a flowchart which shows the whole procedure of the image composition method of 1st Example of this invention. It is a block diagram which shows the structure of the digital camera in this embodiment. It is a figure which shows an example of an image synthesis when the to-be-photographed object which moves within a flame | frame is taken continuously. Example 1 It is a figure which shows an example of difference value calculation between area | regions, and an area | region discrimination | determination result. Example 1 It is an area | region discrimination precision improvement flowchart. Example 1 It is a figure explaining judgment of the proximity of a pixel. Example 1 It is a figure which shows a blend rate calculation table. Example 1 It is a figure which shows the example of an ideal continuous shooting synthetic | combination image. Example 1 It is a flowchart which shows the whole procedure of the image composition method of 2nd Example of this invention. It is a figure which shows an example of the frequency characteristic of a small division area | region. (Example 2) It is a figure which shows the example of the conventional continuous shooting synthetic | combination image.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings, taking a digital camera as an example of an image composition apparatus having an image composition program. However, the present invention is not limited to a digital camera, and can be applied to other image composition apparatuses such as a video camera, a mobile phone, and a personal computer.

  FIG. 2 shows the configuration of the digital camera in this embodiment. The photographing optical system 1 includes a photographing lens, a diaphragm, and a mechanical shutter in detail. The photographing lens further includes a focus lens, a zoom lens, and an iris. Each lens and shutter of the photographic optical system 1 are driven by a motor or driver in accordance with a command from the AE / AF control unit 9. The imaging device 2 converts the subject image formed by the photographing optical system 1 into an electrical signal and outputs it as R, G, and B image signals. The image pickup device 2 is composed of a CCD sensor or a CMOS sensor, and accumulated charges are sequentially read according to a timing signal from a timing generator (TG).

  The analog front end (AFE) 3 is an analog signal processing unit, which removes fixed pattern noise by taking the difference between the voltage value at the time of amplification, gain adjustment, reset and the voltage value at the time of signal output for the analog image signal. A correlated double sampling (CDS) process is performed. The AD conversion circuit 4 converts the image signal supplied from the analog front end (AFE) 3 into a digital signal.

  The controller 5 includes a CPU (not shown), an image memory, and a non-volatile memory. The controller 5 executes a shooting program stored in the non-volatile memory, and receives an operation signal from the operation unit 6 to perform a shooting operation. Further, the controller 5 includes blocks of known gain correction, gamma correction, synchronization processing, RGB-YC separation, noise reduction processing, edge enhancement processing, and JPEG compression processing (not shown). Image data of Y signal and C signal is generated by executing gamma correction, synchronization processing, and the like. The gain correction is to adjust the gray balance by correcting a different gain for each channel of the input RAW data (spatial incomplete sampling data before RGB imaging). The gamma correction is to match the characteristics of an image sensor such as a CCD with the input / output characteristics of the LCD 8. The synchronization process generates an RGB3 image from a single-layer CCD with a Bayer array. The generated image data is stored in the memory card 7. In addition, a predetermined decompression process is performed on the compressed image data to generate uncompressed image data. The uncompressed image data is displayed on the LCD 8 as a captured image.

  Further, the controller 5 includes a circuit for generating a preview image to be displayed on the LCD 8, a circuit for detecting a face image from the subject image, and an AWB (auto white balance) circuit. Further, the controller 5 stores an image composition program 5a of the present embodiment in a non-illustrated non-volatile memory, and executes image composition processing to be described later according to the image composition program 5a. The controller 5 may be composed of a single processor, or an MPU that controls the entire system, and a DIP (digital image processor) that processes image data at high speed as its sub-processor.

  The AE / AF control unit 9 includes a CPU and calculates a physical quantity necessary for AE (automatic exposure control) and AF (automatic focus control). Specifically, the brightness of the subject is detected from the image signal, and the distance to the subject is detected. Appropriate exposure is determined from the brightness of the subject, and the aperture value, shutter speed, and ISO sensitivity are determined.

  Next, a first embodiment of the present invention will be described. FIG. 1 is a flowchart showing the overall procedure of the image composition method according to the first embodiment of the present invention. First, in step S1, high-speed continuous shooting image data made by a digital camera is acquired. Specifically, high-speed continuous shooting of a large number (for example, 10) may be performed, and a small number (for example, 5) may be automatically selected at regular intervals from among them, or only a composite number may be captured in advance. You may do it. Alternatively, captured image data may be read from the memory card 7.

  In step S2, a reference image is created from the image frame for synthesis. The reference image is a so-called background image excluding a moving object image, and means an image that is a base for image composition. The image frames used to create the reference image may be all frames to be combined, but if the number of combined frames is large, an arbitrary number may be selected. In this embodiment, all frames are used.

  If the pixel value at an arbitrary pixel (i, j) on the reference image frame is B (i, j), B (i, j) is expressed by the following equation (1).

  Here, N is the number of frames to be combined, and L (n, i, j) is the pixel value of an arbitrary pixel (i, j) on the nth frame. For this pixel value, for example, the Y signal component (luminance signal component) of the image signal RGB-YC separated by the controller 5 is used. The Y signal has 8 bits, that is, 255 gradations. Expression (1) indicates that the reference image B (i, j) is an average value of luminance values of all frames.

  Next, in step S3, an absolute value ΔL (m, i, j) of a difference value of each pixel value L (n, i, j) between consecutive frames is obtained by the following equation (2).

  Here, m means the order of difference calculation between the (n + 1) th frame and the nth frame, and m = N−1 is the maximum.

  FIG. 3 is a diagram showing an example of image composition when five subjects 21 moving in the frame 20 are continuously shot. In addition, illustration of a background image is abbreviate | omitted here. As shown in the figure, it is assumed that the subject 21 moves so as to draw a circular arc around the left from a position 21a on the right side of the drawing. That is, the first frame (n = 1) is an image when the subject 21 is at the position 21a. Similarly, the second frame (n = 2) is the subject 21 at the position 21b and the third frame (n = 3). ) Is an image at the position 21c, the fourth frame (n = 4) is at the position 21d, and the fifth frame (n = 5) is at the position 21e.

  In such a continuous image, five points of pixels P1 to P5 will be described as samples. First, a difference value ΔL (m, P1) between frames for the pixel P1 is obtained. Here, for the sake of clarity, the pixel name P1 is described instead of the coordinates i and j. In the pixel P1, the subject 21 is shown only in the first frame among all five frames, and the other frames are all background images. Therefore, the difference value between the second frame and the first frame, that is, the difference value ΔL (1, P1) of m = 1 is the difference between the pixel value (luminance value) of the background image and the pixel value (luminance value) of the subject 21. The absolute value of. On the other hand, the difference values ΔL (2, P1), ΔL (3, P1), and ΔL (4, P1) between the other frames are almost zero because they are the differences between the background images.

  Next, in the pixel P2, the subject 21 is shown at a position 21a and a position 21b in the first frame and the second frame, respectively. On the other hand, only the background image is shown after the third frame. Therefore, the difference value ΔL (1, P2) between the second frame and the first frame is a difference value between the subjects 21. At this time, the direction and position of the subject 21 are changing although the timing is slight between frames. Also, if the subject is a human, wrinkles on the clothes will change. As a result, the reflection of light changes, and the pixel value slightly changes. Therefore, the difference value ΔL (1, P2) is a gradation value having a certain size. On the other hand, the difference value ΔL (2, P2) of m = 2 is a difference value between the background image and the subject 21 at the position 21b, and is generally a large value. Further, the difference values ΔL (3, P2) and ΔL (4, P2) of m = 3, 4 are almost zero because they are the differences between the background images.

  In the pixel P3, the subject 21 is shown at a position 21a, a position 21b, and a position 21c in the first to third frames, respectively. On the other hand, only the background image is shown after the fourth frame. Therefore, the difference value ΔL (1, P3) between the second frame and the first frame is a difference value between subjects. Similarly, the difference value ΔL (2, P3) between the third frame and the second frame is also a difference value between subjects. The difference value ΔL (3, P3) between the fourth frame and the third frame is the difference value between the subject at the position 21c and the background, and the difference value ΔL (4, P3) between the fifth frame and the fourth frame is The difference value between backgrounds.

  Similar processing is performed for the pixel P4 and the pixel P5. Returning to step S4 in FIG. 1, the controller 5 determines whether each pixel is a pixel of a moving body main region where a moving image exists frequently throughout the entire frame or a background image exists frequently from these calculation results. It is determined whether the pixel is in the background main area.

  FIG. 4 is a diagram showing an example of the inter-frame difference value calculation and the region discrimination result. The controller 5 counts the number of times that the absolute value ΔL of the difference value between frames is equal to or greater than a predetermined value Lx for each of the pixels P1 to P5. The predetermined value Lx may theoretically be set to 1 or more, but in this embodiment, the predetermined value Lx is set to “3” in consideration of noise, subtle camera shake during photographing, and the like. The controller 5 determines the background main area for pixels whose difference value ΔL is equal to or greater than the predetermined value “3” for two or less pixels, and the moving object main area for three or more pixels exceeding the half of the total number m = 4. Is determined. As a result, the pixels P1 and P2 are determined as the background main area, and the pixels P3, P4, and P5 are determined as the moving body main area.

  In the above description, an example in which only five pixels of the points P1 to P5 are determined as samples is shown, but it goes without saying that the same processing is performed for all the pixels on the frame. As a result, in the example of FIG. 3, a region indicated by hatching in the drawing is determined as the moving body main region, and the other regions are determined as the background main region.

  By the way, in the case where the luminance value is hardly changed even though it is a moving object, the region may not be accurately identified by the above method. For example, when the subject is wearing plain dark clothes, the pixel value may hardly change during continuous shooting depending on the pixel. Although such a pixel is originally a moving object main region, it is highly likely that the pixel is determined as a background main region. As a result, for example, some pixels in the shaded moving main area in FIG. 1 are determined as background main area pixels, and the opposite blending process is performed despite the same type of area. For this reason, there exists a possibility that a patch-like composite image may be made.

  Therefore, in the present embodiment, after the above-described determination procedure is completed, the area determination accuracy increasing flowchart of FIG. 5 is executed. First, in step S11, a pixel (referred to as a primary candidate pixel) whose pixel value approximates the representative value of the moving object main region is extracted for each frame from the pixels determined to be the background main region. The representative value of the moving body main region is preferably an average value or a median value of all frames. Further, as an approximation determination, it is preferable to use whether or not the difference value between the representative value and the pixel value is about several gradations or less. As a specific example of the extraction, for example, if the pixel P3 in FIG. 3 is determined to be the background main region even though it is originally a moving object main region, the pixel values in the first to fifth frames of the pixel P3 are all set. Compare with the representative value of the moving body main area. If there is at least one moving body main region that approximates any frame, the pixel of that frame is determined as the primary candidate pixel.

  Next, out of the pixels determined to be the primary candidate pixels, pixels that are close to the moving object main region are extracted as secondary candidate pixels (step S12). This proximity determination will be described with reference to FIG. For example, if the primary candidate pixels Pb and Pc are adjacent to the pixel Pa determined to be a moving body main region pixel as shown in the figure, the pixel Pb directly adjacent to the pixel Pa is naturally a secondary candidate. It is determined as a pixel. Further, the primary candidate pixel Pc adjacent to the pixel Pb determined as the secondary candidate pixel is also determined as the secondary candidate pixel. Further, if there is a primary candidate pixel adjacent to the pixel Pc, the pixel is also determined as a secondary candidate pixel.

  Returning to FIG. 5, in step S <b> 13, a pixel with a high frequency in which secondary candidate pixels exist (for example, a pixel in which secondary candidate pixels exist in 3 frames out of all 5 frames) is newly extracted as pixels of the moving object main region. Thereby, the moving body main region pixels that could not be picked up only by the determination of FIG. 4 can be reliably extracted.

  In the above description, the primary candidate pixel and the secondary candidate pixel are obtained for each pixel in all frames. However, an average of all pixel values of each pixel value is obtained in advance, and the average value is compared with a representative value of the moving object main region. If they are similar, they may be newly extracted as pixels of the moving object main region.

  Returning to FIG. 1, in step S <b> 5, the following image composition processing is performed based on the results of FIGS. 4 and 5. First, for each of the pixels P1 to P5, an absolute value ΔLB (n, i) of a difference between the reference image B (i, j) obtained by Expression (1) and each pixel value L (n, i, j) for each frame. , J) is calculated by equation (3).

  Next, the blend rate for each frame of each pixel is calculated based on the blend rate calculation table of FIG. Here, assuming that the blend ratio of pixels in the moving body main region is αm and the blend ratio of pixels in the background main region is αb, the difference value ΔLB (n, i, j) from the reference image B (i, j) is predetermined. Specific values are determined depending on whether the value is greater than or equal to σ or less than σ. Note that the value of σ may be determined experimentally as an optimal value. Preferably, it is set to about several tens of gradations.

  In the figure, h is the number of pixels in all frames that satisfy condition 1 (ΔLB is less than a predetermined value σ) in the moving object main region, and k is a pixel that satisfies condition 2 (ΔLB is greater than or equal to the predetermined value σ) in the background main region. The number in all frames. The blend ratio αm of each pixel in the moving object main area is 1 / h when the condition 1 is considered, that is, when the moving object image is considered as a pixel, while when the condition 2 is satisfied, that is, the pixel where the background image is reflected Zero when considered.

  On the other hand, the blend ratio αb of each pixel in the background main region is determined when the number k of pixels satisfying the condition 2 (pixels in which a moving image is captured) is zero in all frames, that is, when all frames are only background images. In this case, αb of each pixel is 1 / N (N is the total number of frames). On the other hand, when k is not zero, that is, when there is at least one moving body image in all frames, αb of a pixel satisfying condition 1 is set to zero and αb of a pixel satisfying condition 2 is set to 1 / k. .

  As a specific example, consider the pixel P1, which is the background main region in FIG. Since the moving object is shown in the first frame of P1, the difference value ΔLB (1, P1) from the reference image is equal to or greater than the predetermined value σ. Therefore, the blend rate αb regarding the pixel of the first frame of P1 (hereinafter referred to as P11) is 1 / k. Here, in the pixel P1, since only the first frame satisfies the condition that satisfies the predetermined value σ or more, k = 1, that is, the blend rate αb of the pixel P11 is 1/1 = 1. On the other hand, since the pixel P12 (meaning the pixel of the second frame, hereinafter the same), P13, P14, and P15 of the second and subsequent frames of P1 are all background images, their difference values ΔLB (2, P12), ΔLB (3, P13), ΔLB (4, P14), and ΔLB (5, P15) are all less than the predetermined value σ, and the blend ratio αb is all zero.

  When the blend ratio for all the frames of the pixel P1 is obtained in this way, the combined pixel value for all the frames of each pixel is calculated by the following equation (4).

  Specifically, P1 will be calculated based on equation (4). In the background main region such as P1, the reference image has a value close to the pixel value of the background image, so the number of frames corresponding to condition 1 in P1 is 4, and the number of frames corresponding to condition 2 is 1. Therefore, the composite pixel value of P1 is expressed by the following equation (5).

  That is, for the pixel P1, only the pixel value of the pixel P11 of the first frame is synthesized. Similarly, this time, it calculates about P3 of a moving body main area | region pixel. In the moving object main region such as P3, the reference image has a value close to the pixel value of the moving object, so the number of frames corresponding to condition 1 in P3 is 3, and the number of frames corresponding to condition 2 is 2. Therefore, the composite pixel value of P3 is expressed by the following equation (6).

  Similarly, a blended image can be generated by obtaining the blend ratios for all the pixels on the frame 20 and adding the blend ratios according to the equation (4).

  As described above, according to the present embodiment, the image in the frame is divided into the moving body main area and the background main area, and the images are synthesized with the blend ratios corresponding to each of them. As shown in FIG. 8, an ideal continuous-shot composite image can be obtained without a background image appearing on the subject, as shown in FIG.

  In the above description, the blend rate is given only to the image that is the subject of each area, and the blend rate is zero for the other images. However, the present invention is not limited to this, and a certain blend rate may be given to images that are not the subject. . Thereby, a smooth image is obtained.

  In this embodiment, the luminance value (Y signal) of each pixel is used as the pixel value. However, calculation may be performed for each color of RGB. Furthermore, although the example which calculates | requires a reference | standard image with the average value by Formula (1) was shown in a present Example, you may use a median.

  Next, a second embodiment of the present invention will be described. FIG. 9 is a flowchart showing the overall procedure of the image composition method of the second embodiment. First, in step S21, image data is acquired by the same method as in the first embodiment of FIG. In step S22, a reference image is created from the image frame for synthesis. For this creation, it is assumed that all frames are used as in the first embodiment.

  In step S23, all the image frames for synthesis are once synthesized (referred to as a preliminary synthesized image). In step S24, the preliminary composite image is divided into small divided areas, for example, areas of 16 × 16 pixels. Then, a Fourier transform is performed on each small segment area to calculate a frequency characteristic.

  FIG. 10 is a diagram illustrating an example of the frequency characteristics of the small segment area. In step S25, a region where the peak frequency fa of each small segment region is equal to or lower than the predetermined frequency fx is determined as the moving object main region. This is because, in a region where moving objects overlap, when the target frame is synthesized, an overall blurred image is obtained, and the frequency characteristics are lowered. On the other hand, an area where the peak frequency fa is larger than fx is determined as the background main area. This is because it is considered that the frequency characteristic is improved in the region where the background image is mainly because the edge is clear even if the background image is synthesized.

  However, there is a possibility that the area determined as the moving body main area includes a non-shaded background image such as a blue sky or a plain wall. Therefore, in step S26, an exceptional image is extracted from those determined as the moving object main region. Specifically, for each pixel in the target subdivision area, the difference in pixel value between frames is calculated by the same calculation formula as Expression (2) described in the first embodiment. If the absolute value of the difference is equal to or less than a predetermined value (for example, gradation 3 or less) and is more than half of the total number of operations m, the pixel is regarded as a background main region. In this case, the determination may be made on a pixel basis, or for all of the subsections in which most of the pixels regarded as background main area pixels occupy, all of the pixels may be regarded as background main area pixels.

  In step S27, a final composite image is generated in the same manner as in step S5 of FIG. As a result, as in the first embodiment, an ideal continuous shooting composite image can be generated.

  It can be used for a method of synthesizing digital data of continuously shot images on a digital camera, another portable terminal device, or a personal computer.

DESCRIPTION OF SYMBOLS 1 Image | photographing optical system 2 Image sensor 5 Controller 5a Image composition program 7 Memory card 8 LCD
21 Subject P1, P2, P3, P4, P5 pixels

Claims (9)

In an image composition method for generating a single image by combining a plurality of imaging frames,
A reference image generation step of generating a reference image based on the image data of the plurality of frames;
A pixel value change characteristic calculating step of calculating a pixel value change characteristic of each pixel of the frame;
An area determination step for determining whether each pixel is a moving main area where a moving image exists frequently or a background main area where a background image exists frequently based on the pixel value change characteristics;
When it is regarded as a pixel of the moving body main region, a pixel value having a large difference from the pixel value of the reference image is blended weakly, while when it is regarded as a pixel of the background main region, the reference An image blending step for strongly blending pixel values having a large difference from the pixel values of the image;
An image synthesizing method characterized by comprising: The pixel value change characteristic is difference value data of pixel values between successive frames of each pixel,
The region determination step determines whether the pixel is the moving body main region or the background main region according to a ratio when the absolute value of the difference value is equal to or greater than a predetermined value.
The image synthesizing method according to claim 1. The region determining step determines that the pixel is the moving body main region when the ratio when the absolute value of the difference value is a predetermined value or more is half or more.
The image synthesizing method according to claim 2. In the region determining step, among the pixels determined to be the background main region, a pixel that is close to the pixel of the moving body main region and has a pixel value close to the pixel is determined to be the moving body main region.
The image synthesizing method according to claim 2. The pixel value change characteristic calculating step includes a preliminary synthesized image generating step of generating a preliminary synthesized image by synthesizing a predetermined number or more of the frames to be synthesized, and dividing the preliminary synthesized image into small regions, A frequency characteristic calculating step for calculating a frequency characteristic for each region,
The region determining step determines whether each small region is the moving body main region pixel or the background main region according to the peak frequency of the frequency characteristic.
The image synthesizing method according to claim 1. In the region determining step, when the peak frequency is equal to or lower than a predetermined frequency, the small region is determined as the moving body main region.
The image composition method according to claim 5. The area determination step calculates a difference value between successive frames for each pixel of the small area determined to be the moving body main area, and when the ratio at which the absolute value of the difference value is equal to or less than a predetermined value is high. Determines the pixel as the background main region,
The image composition method according to claim 5. In an image composition program that generates a single image by combining a plurality of imaging frames,
A reference image generation step of generating a reference image based on the image data of the plurality of frames;
A pixel value change characteristic calculating step of calculating a pixel value change characteristic of each pixel of the frame;
An area determination step for determining whether each pixel is a moving main area where a moving image exists frequently or a background main area where a background image exists frequently based on the pixel value change characteristics;
When it is regarded as a pixel of the moving body main region, a pixel value having a large difference from the pixel value of the reference image is blended weakly, while when it is regarded as a pixel of the background main region, the reference An image blending step for strongly blending pixel values having a large difference from the pixel values of the image;
An image composition program for causing a computer to execute the above. In an image synthesis device that generates a single image by combining a plurality of imaging frames,
Reference image generation means for generating a reference image based on the image data of the plurality of frames;
Pixel value change characteristic calculating means for calculating a pixel value change characteristic of each pixel of the frame;
Based on the pixel value change characteristics, an area determination unit that determines whether each pixel is a moving main area where a moving image exists frequently or a background main area where a background image exists frequently;
When it is regarded as a pixel of the moving body main region, a pixel value having a large difference from the pixel value of the reference image is blended weakly, while when it is regarded as a pixel of the background main region, the reference Image blending means for strongly blending pixel values having a large difference from the image pixel values;
An image synthesizing apparatus comprising:

Images