JP2006180382A - Image generating apparatus and image generating method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image generating apparatus generating an image sequences without motion blurring even if a plurality of objects under different motions are present within one image. <P>SOLUTION: The image generating apparatus comprises: an image input unit 101; a generation pixel designation unit 102; a motion domain analysis unit 103; an integrated space-time domain determination unit 104; a pixel judgement unit 105; a pixel value integration unit 106; an integration condition judgement unit 107; an integration condition designation unit 108; and an image output unit 109. The image input unit 101 inputs an image stream, a motion domain analysis unit 103 extracts motions of objects in the image stream, and the integration space-time domain determination unit 104 determines a space-time domain to be integrated on the basis of integration conditions determined by the integration condition designation unit 108. The pixel value integration unit 106 integrates pixel value of the space-time domain for each pixel, and the image output unit 109 outputs a result of the integration as a new image sequences. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image generation device that generates a new image sequence based on an input image sequence, and more particularly to an image generation device that generates an image sequence without motion blur.

  2. Description of the Related Art Conventionally, as an image generation apparatus that generates a new image sequence from an input image sequence, an apparatus that uses movement of the entire screen has been proposed (for example, see Patent Document 1). FIG. 14 is a flowchart illustrating a schematic process of the image generation process performed by the image generation apparatus disclosed in Patent Document 1.

As shown in FIG. 14, the image generation apparatus obtains information (motion vector information) indicating a relative positional shift between two images obtained at different exposure times (step A1). Thereafter, based on the obtained relative motion vector of the two images, the shooting frame of one image is shifted and the pixel value of the two images is added and averaged to generate a final captured image (step). A2).
Japanese Patent Laying-Open No. 2003-319240 (FIG. 5)

  However, the conventional image generation apparatus considers that the motion of the subject in the image is the same, shifts the image frame, and performs an averaging process of pixel values. For this reason, when individual subjects move differently in the image, there is a problem that even if the image frame is moved and addition averaging is performed, motion blur occurs for each subject.

  In addition, when the subject is dark, the S / N ratio of the image is bad, and it is necessary to lengthen the exposure time of the two images to be added in order to solve this. However, if the exposure time is increased, if the subject moves quickly, motion blur occurs in one image. For this reason, there is a problem that motion blur cannot be removed even if the entire image is aligned and averaged.

  The present invention has been made to solve the above-described problems, and can generate an image sequence free from motion blur even when a plurality of subjects having different motions exist in one image. It is a first object of the present invention to provide an image generating apparatus that can be used.

  A second object of the present invention is to provide an image generation apparatus capable of generating an image sequence having a high S / N ratio and no motion blur even when the subject is dark.

  An image generation apparatus according to an aspect of the present invention includes a motion calculation unit that calculates the motion of each of a plurality of pixels included in an image of interest among the plurality of images based on a plurality of temporally continuous images. The pixel position specifying means for specifying the pixel position corresponding to each pixel for each of the plurality of images based on the movement of each pixel for each pixel included in the target image, and included in the target image A pixel value correction unit that corrects the pixel value of each pixel of the image of interest by adding pixel values at pixel positions corresponding to the pixels in the plurality of images for each pixel. .

  In this way, the sum of the pixel values of a plurality of corresponding images is obtained based on the motion, and the pixel values are corrected. For this reason, even when a plurality of subjects having different motions exist in one image, an image sequence free from motion blur can be generated. Further, by integrating the pixel values, the same effect as that obtained by extending the exposure time can be obtained. Therefore, even when the subject is dark, S / N can be improved and motion blur can be suppressed. .

  Preferably, the pixel value correcting unit compares the pixel value of each pixel with the pixel value at the pixel position corresponding to each pixel for each pixel included in the image of interest, and determines the pixel value at the pixel position. It has a pixel determination part which determines whether it makes addition object, and an addition part which adds the pixel value in the pixel position made into the addition object in the said pixel determination part, It is characterized by the above-mentioned.

  For example, even if there is a case where the illumination intensity is increased and the image is completely white due to the effect of imaging a headlight of an automobile, the image at that time can be excluded from the addition target pixels. Even if the pixel of the subject of interest is hidden by the pixel of another subject, the image at that time can be excluded from the addition target pixels.

  More preferably, the pixel value correction means further includes a condition determination unit that determines whether or not a predetermined condition is satisfied, and the addition unit performs an addition process until the predetermined condition is satisfied. To do. In addition, the condition determination unit determines whether a ratio between a value obtained by adding in the addition unit and a predetermined noise value for each pixel included in the target image is larger than a predetermined threshold value. And the addition unit performs an addition process until a ratio between the value obtained by the addition and the predetermined noise value becomes larger than the predetermined threshold value.

When the S / N ratio is used as a condition, there is an advantage that the image quality of the resulting moving image can be more intuitively specified. Further, the condition determining unit is configured to add the adding unit for each pixel included in the target image. It may be determined whether or not the addition processing time is greater than a predetermined threshold, and the addition unit may perform the addition processing until the addition processing time exceeds a predetermined threshold.

  By limiting the addition processing time, forward real-time processing with a certain time delay becomes possible.

  According to the image generating apparatus of the present invention, when shooting an object including a subject with different motion, an image with an increased S / N ratio can be obtained while suppressing motion blur.

  Hereinafter, an image generation system according to an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing a configuration of an image generation system according to an embodiment of the present invention.
The image generation system 200 is a system that generates an image sequence free from motion blur and having an increased S / N ratio from a plurality of image sequences, and includes an imaging device 210, an image storage device 220, an image generation device 230, And a display device 240.

  The imaging device 210 is a device for capturing a subject and obtaining an image sequence composed of a plurality of images.

  The image storage device 220 is a storage device that stores an image sequence captured by the imaging device 210.

  The image generation device 230 is a device that receives an image sequence from the image storage device 220 and generates an image sequence with no motion blur and an increased S / N ratio from the input image sequence.

  The display device 240 is a display device that displays the image sequence generated by the image generation device 230. Note that the image generation device 230 may be realized as a dedicated circuit or as an image processing program in a general-purpose computer.

  FIG. 2 is a block diagram showing an internal configuration of the image generation apparatus 230 in the image generation system 200 shown in FIG.

  The image generation device 230 is a device that generates an image sequence with no motion blur and an increased S / N ratio from the input image sequence as described above, and includes the image input unit 101 and the generated pixel designation unit 102. A motion region analysis unit 103, an integration spatio-temporal region determination unit 104, a pixel determination unit 105, a pixel value integration unit 106, an integration condition determination unit 107, an integration condition designation unit 108, and an image output unit 109. It has.

  The image input unit 101 is a processing unit for inputting an image sequence obtained by the imaging device 210.

  The generated pixel designation unit 102 is a processing unit that designates a pixel position to be subjected to processing for suppressing motion blur and increasing the S / N ratio. The pixel position is specified by the frame number in the image sequence and the position in the frame. That is, the pixel position in the image of interest is automatically designated sequentially.

  The motion region analysis unit 103 is a processing unit that obtains, for each predetermined position, the motion of the image in each frame in the image in the image sequence input by the image input unit 101. The predetermined position may be all the pixels in the image or a block of a predetermined size.

  The integration condition designating unit 108 is a processing unit that designates a condition that defines a range of other pixels to be integrated in correspondence with the pixel designated by the generated pixel designation unit 102. As a condition designation method, the number of pixels to be integrated (the number of frames) may be specified, or the S / N ratio of the result obtained as an integration result may be specified.

  The integrated spatio-temporal region determination unit 104 is a processing unit that determines a spatio-temporal region of pixels to be integrated in order to satisfy the conditions specified by the integration condition specifying unit 108 for the pixels specified by the generated pixel specifying unit 102. .

  The pixel determination unit 105 is a processing unit that determines whether the pixel determined by the integration spatiotemporal region determination unit 104 is valid as an integration target.

  The pixel value integration unit 106 is a processing unit that integrates pixel values for the pixels that the pixel determination unit 105 determines to be valid.

  The integration condition determination unit 107 determines whether or not the integration result of the pixel value integration unit 106 satisfies the integration condition specified by the integration condition specifying unit 108, and if the condition is satisfied, the image generated in the image output unit 109 is determined. Is a processing unit for outputting.

  The image output unit 109 is a processing unit that outputs the image input from the integration condition determination unit 107 as a generated image.

Here, the motion region analysis unit 103 corresponds to “motion calculation means”.
The integrated spatio-temporal region determination unit 104 corresponds to “pixel position specifying means”.

  The pixel determination unit 105, the pixel value integration unit 106, the integration condition determination unit 107, the integration condition designation unit 108, and the image output unit 109 correspond to “pixel value correction means”.

The pixel value integrating unit 106 corresponds to an “adding unit”.
The integration condition determination unit 107 corresponds to a “condition determination unit”.

  Next, processing executed by the image generation device 230 will be described. FIG. 3 is a flowchart of processing executed by the image generation device 230.

  The image input unit 101 reads image sequences that are temporally continuous from the image storage device 220. An example of a continuous image sequence is shown in FIGS. 4 (a) to 4 (e). In the image sequence of FIGS. 4A to 4E, a scene in which a circular object (region B) is moving from left to right is photographed. In this scene, a horizontally long rectangular (lattice pattern) object (region C) exists behind the moving circular object and is stationary. Furthermore, there is a white background area A that is stationary behind it.

  Conventionally, when a dark subject is photographed with a sufficient exposure time (for example, 30 ms), motion blur has occurred around a circular moving object in each frame of a moving image, as in a region D in FIG. .

  Here, as shown in FIGS. 4A to 4E, consider an image sequence in which the exposure time of each frame is sufficiently shortened to reduce motion blur due to movement of an object. At this time, an image having a small S / N ratio can be obtained due to a shortage of light amount although an image blur is hardly caused due to a short exposure time.

  After the image sequence is input, the generated pixel specifying unit 102 sequentially specifies the frames to be generated (S501). Here, it is assumed that the generated pixel designation unit 102 designates the frame shown in FIG.

  Next, the integration condition designating unit 108 designates the integration condition (S502). Here, the integration time is specified as 30 ms as the integration condition. In addition, it is assumed that the imaging device 210 images a subject with a frame interval of 6 ms and an exposure time of 2 ms.

  The integrated spatio-temporal region determination unit 104 determines the number of integrated frames F that satisfies the integration condition (S503). That is, according to the above conditions, the integrated frame number F is calculated as 30 ms / 2 ms = 15 frames.

  Next, the motion region analysis unit 103 obtains an image motion for each position in the image of the frame of interest within each frame of the front and rear F frames centering on the frame of interest (FIG. 4A) (S504). The position for obtaining the motion may be a pixel unit or a region unit.

  Here, the range of the frame for which the motion is obtained is the front and rear F frame centered on the frame of interest (FIG. 4A). This is because the motion of only the F frame is obtained in the processing of S509 described later. This is because an effective pixel may not be included in the addition. As described above, in the frame having the number of frames F or more, the movement in each frame is obtained so that the pixels effective for addition can be used as much as possible.

  The movement at each position on the image is, for example, the method proposed in P.ANANDAN, “A Computational Framework and an Algorithm for the Measurement of Visual Motion”, IJCV, 2, 283-310 (1989) A motion detection method or the like generally used in moving image coding can be used. For this reason, detailed description of the motion detection method will not be repeated here.

  FIG. 5 is a diagram showing temporal changes in pixel values on the horizontal line indicated by the broken line 701 in the image of FIG. That is, FIG. 5 is an image in which pixel values of broken lines 702 to 705 on the same horizontal line as the broken line 701 are extracted from FIGS. 4B to 4E and arranged in the time axis t direction. The position of the horizontal broken line 701 in FIG. 5 corresponds to the time of the frame in FIG. In FIG. 5, the horizontal axis indicates the position on the horizontal line indicated by the broken line 701 in FIG. 4A, and the vertical axis indicates the time transition downward. In FIG. 5, pixel values other than those on the broken lines 701 to 705 are also displayed. However, in actuality, the image is displayed at a frame rate denser than the images shown in FIGS. 4 (a) to 4 (e). , And an image as shown in FIG. 5 is obtained.

  Hereinafter, the operation of the present invention will be described with a focus on the space-time shown in FIG. 5 which is a part of the image in FIG. The same processing is performed for the other regions in FIG. Here, the motion results obtained by the motion region analysis unit 103 are indicated by arrows in FIG. Since the area A is stationary, the arrow points directly below. Similarly, in the region C, the arrow points directly below. On the other hand, since the region B moves from left to right, the arrow points downward to the right.

  Next, the generated pixel specifying unit 102 specifies a pixel position used for generating a pixel value in the frame of interest (here, the frame in FIG. 4A) (S505). FIG. 6 shows a change in motion at each pixel position in the same image as FIG. For example, the generation pixel specifying unit 102 specifies the position where the pixel value is generated as Pa0 in the drawing.

  The integrated spatiotemporal region determination unit 104 determines another frame position corresponding to Pa0 based on the result of the motion region analysis process (S504) (arrow in FIG. 5) and the integrated frame number F (Pa1 to Pa14). (S506). Here, the result of the motion region analysis is mainly used to determine the direction in which the point corresponding to Pa0 exists, and the integrated frame number F is used to determine to what time range the corresponding point is to be obtained. In the same procedure, Pb1 to Pb14 are obtained for Pb0, and Pc1 to Pc14 are obtained for Pc0.

  Next, for each pixel (Pa0 to Pa14, Pb0 to Pb14, Pc0 to Pc14, etc.) obtained in the previous step, it is determined whether or not the pixel determination unit 105 is effective for use in integration (S507). The determination of the effectiveness is performed based on whether it does not deviate from the range of the photographed image, is not concealed by another object, or has a large change in pixel value. Here, in the case of the point Pc0 on the region C, concealment occurs halfway through the movement of the region B. For this reason, the pixels Pc9 to Pc14 in the concealed portion are determined to be invalid and are excluded from integration targets.

  Although Pa1 to Pa14 and Pb1 to Pb14 do not deviate from or conceal from the image, if there is a point where the pixel value changes greatly, the point is determined as an invalid pixel. As a method for determining whether or not the pixel value has changed greatly, for example, an average m and a variance σ of pixel values Pa0 to Pa14 are calculated, and the value of each pixel is set to i, and the following equation (1) is obtained: Judgment is effective when it satisfies.

| I−m | <α · σ (1)
(Α is a real constant greater than 0)
Possible causes of pixel value changes include temporal changes in illumination intensity and color in the image, the effects of camera noise during imaging, changes in the surface reflection characteristics of the subject, and reflections due to changes in the orientation of the subject surface A change in light, an error in the motion region analysis process (S504), a limit of analysis accuracy, and the like are conceivable. FIG. 7 is an example of an image showing temporal changes in pixel values when the illumination intensity temporarily changes. In FIG. 7, at the time indicated by the arrow E, the illumination intensity increases in the entire image and becomes pure white due to the influence of imaging a headlight of an automobile or the like. In such a case, the image at this time is excluded from the integration target pixels.

  If it is determined that the pixel satisfies the validity (y in S508), the process of S509 described later is performed. If it is determined that the pixel does not satisfy the validity (n in S508), the process proceeds to S506. . At this time, in S506, a new corresponding pixel position is determined. For example, when it is determined that Pc5 is invalid as a pixel for Pc0, Pc6 that has not yet been evaluated for effectiveness is designated.

  The pixel value integration unit 106 performs integration for pixels determined to be valid (S509). For example, when the pixel of interest is Pa0, the pixel values Pa0 to Pa14 are integrated. At this time, a predetermined gain may be multiplied and integrated as necessary. For example, if the number of effective pixels does not reach the predetermined 15 and becomes 10 due to the pixel invalidity determination, the added number of other peripheral pixels is insufficient, so 15/10 = 1.5 is set as the integrated value. Do the multiplication.

  Next, the integration condition determination unit 107 determines whether the integration condition is satisfied (S510). In this case, it is determined whether or not the integrated number has reached 15. If the condition is satisfied (y in S510), the process proceeds to S511. If the condition is not satisfied (n in S510), the process returns to S506, and a new corresponding pixel position is determined to compensate for the insufficient pixel. For example, Pc-1, Pc-2, Pc-3, etc. in FIG. 6 are selected as new pixels corresponding to Pc0. As described above, it is not always necessary to select only pixels in one direction in terms of time with respect to the frame of interest, and pixels located in front of and behind in time may be selected.

  If the integration condition is satisfied (y in S510), the pixel value obtained by the integration is output as the pixel value of the target pixel of the target frame (S511).

  The integration condition determination unit 107 determines whether or not an integration value has been obtained for all pixels of the frame of interest (S512). If there is a pixel that has not obtained an integration value, the process proceeds to S505, where all pixel values are obtained. If it is obtained, the process proceeds to S513.

  The integration condition determination unit 107 determines whether or not all consecutive frames have been processed (S513). If processing of all frames has not been completed, the unprocessed new frame is processed. The process proceeds to S501, and when all the frames have been processed, the entire process ends.

  For the frame shown in FIG. 4A, an image with an improved S / N ratio as shown in FIG. 8 is finally obtained.

  Here, in the case where the above-described processing is performed for successive frames, there is a case where the temporal addition ranges partially overlap for corresponding pixels. For this reason, when the integration time range is longer than the time interval between frames, it is possible to reduce the calculation time required for integration by temporarily storing overlapping integration results. This point will be described with reference to FIG. In order to obtain the integrated pixel value of Pb0 in FIG. 9, pixels existing in the range of the arrow 801 indicated by the broken line are integrated. Here, in order to obtain the integrated pixel value of Pb1 next by temporarily storing the integrated value S0 of the portion below Pb1 in the broken-line arrow area, a new solid-line arrow By calculating the partial integrated value and adding it to the integrated value S0, the integrated pixel value of Pb1 can be obtained. For this reason, the processing time concerning calculation of integrated value S0 is shortened.

  As described above, according to the present embodiment, a corresponding pixel value is selected from a plurality of frames based on motion region analysis, and further, effective pixels are selectively added to be included in the luminance signal of each pixel. It is possible to cancel the noise component that is generated, and it is possible to increase the ratio (S / N ratio) between the true luminance value and the noise of an effective pixel.

  In addition, even when the subject moves in a dark place, it is possible to obtain a bright image without motion blur by increasing the integration number. Conventionally, in order to suppress motion blur of a subject including different motions, it has been necessary to photograph the subject with a short exposure time. However, when the exposure time is shortened, in a dark place, as shown in FIG. 10A, the noise component generated by the camera or the like becomes relatively large with respect to the luminance (signal component) of the object. For this reason, the image quality has been lowered due to a decrease in the S / N ratio. On the other hand, when the present invention is used, it is possible to obtain an image as shown in FIG. 10B for each successive frame, and it is possible to obtain a video with a high S / N ratio by suppressing motion blur of a subject even in a dark place. It becomes possible.

  In this embodiment, the description has been made on the assumption that there is one moving object. However, even when there are a plurality of subjects having different motions, the motion is obtained for each subject and the pixel values are accumulated. Is doing. For this reason, even in such a case, an image having no motion blur and a high S / N ratio can be obtained.

  The image generation system according to the present embodiment has been described based on the embodiment, but the present invention is not limited to this embodiment.

  For example, in the above-described embodiment, the integration condition specifying unit 108 specifies the exposure time as the integration condition, but it is also possible to specify the S / N ratio to be obtained as a result. When the S / N ratio is designated, there is an advantage that the image quality of the resulting moving image can be designated more intuitively. In this case, for the noise component generated from the imaging device 210, for example, the pixel value of the imaging device 210 in a state where light is blocked is measured in advance, and this is set as the noise component N. On the other hand, the maximum value or the average value of the pixel values obtained by the integration process (S509) is used as the signal component S, and in the integration condition determination process (S510), the S / N ratio is calculated each time, and the S / N ratio is a desired value. Is reached, it is determined that the integration condition is satisfied, and the integration is terminated.

  Further, as another integration condition, the maximum time T required for the integration process may be used. In this case, when the processing time from the start of the processing shown in FIG. 3 (from the processing of S501) reaches the time T, it is determined in the integration condition determination processing (S510), and the time when the time T is exceeded. Thus, it may be determined that the integration condition is forcibly satisfied, and the addition of the pixel values may be terminated. By using such conditions, near real-time processing with a certain time delay is possible, and the image generation device 230 can be easily connected to an existing image transmission / recording / display device.

  Furthermore, as the integration condition, a temporary capacity of an image storage device (not shown) necessary for the addition process may be designated. This is because it is necessary to hold the previous and next frame images in order to perform the integration process on the frame of interest, but the larger the integration number, the greater the storage capacity required. This is because it is necessary to provide. Here, the image storage device corresponds to “temporary storage means”.

  Furthermore, combining the plurality of integration conditions described above may end the addition of pixel values when at least one of the plurality of conditions is satisfied.

  Note that it is desirable to use a camera with a high frame rate and shutter speed as the imaging device 210. By using such a camera, motion blur in individual frames is less likely to occur. This makes it easier to analyze the motion between frames.

  In general, high-speed camera images have a large amount of information per unit time, and it is difficult to transmit and record images. By outputting, existing devices and systems can be used for image transmission and image recording, and an image with a high S / N ratio can be obtained.

  Further, the effect of reducing the noise of the camera, that is, the improvement of the S / N ratio can be expected more when the noise has a temporally or spatially random property. This is because the influence of noise can be reduced by accumulating pixel values over several frames due to the random nature of noise.

  Note that part or all of the image generation processing performed by the image generation apparatus of the present invention may be performed by a dedicated device, a computer terminal device, a communication device arranged in a base station, or a stand-alone device. The CPU built in the computer may execute the image generation processing program to execute the image generation processing.

  Further, the present invention may be realized as a single device as the camera 300 as shown in FIG. 11 using the configuration excluding the display device 240 in the configuration of the image generation system 200 shown in FIG.

  Furthermore, a display device 240 may be added to the camera 300 shown in FIG. 11 to realize a camera with a moving image recording / playback function. In these cases, the frame rate of the finally output / displayed moving image may be about 30 fps (frame / second), but the frame rate of the image sequence used for the internal image processing is set to a high frame if necessary. By setting the rate, it is possible to facilitate analysis of the motion region and the like.

  Further, the present invention may be realized as a display device 400 such as a television as shown in FIG. 12 using a configuration excluding the imaging device 210 in the configuration of the image generation system 200 shown in FIG. In this case, the S / N ratio of an image recorded in advance can be improved and displayed.

  In the above-described embodiment, the image generation process has been described on the assumption that the moving object (region B in FIG. 4) in the image moves in the horizontal direction. It moves not only in the direction but also in the vertical direction and other free directions. Therefore, the time change of the moving object as shown in FIG. 5 is expressed by a three-dimensional image as shown in FIG. That is, in FIG. 13, the time change of the moving object when the area B moves in the upper right direction and further in the lower right direction is shown as a three-dimensional image. In such a case, the point P0 on the region B moves to P1 after several frames, and further moves to P2 after several frames. Therefore, in order to obtain the pixel value of P0, integration is performed using the pixel values of P1 and P2, which are positions corresponding to P0 in the three-dimensional image.

  INDUSTRIAL APPLICABILITY The present invention can be applied to an image generation apparatus that can obtain an image with an increased S / N ratio while suppressing motion blur of a subject when shooting a plurality of subjects that move differently. The present invention can be applied to a video photographing device such as a digital camera, a device that records and reproduces video, a monitor device that displays accumulated video, and the like.

It is a block diagram which shows the structure of the image generation system which concerns on embodiment of this invention. It is a block diagram which shows the internal structure of the image generation apparatus in the image generation system shown in FIG. It is a flowchart of the process which an image generation apparatus performs. (A)-(e) is a figure which shows an example of the image containing the moving object imaged with the imaging device which concerns on this Embodiment, (f) shows the moving object imaged with the conventional imaging device. It is a figure which shows an example of the image containing. It is a figure which shows an example of the time change of the moving image containing a moving object. It is a figure which shows an example of the time change of the moving image containing a moving object. It is a figure which shows an example of the time change of the moving image containing a moving object. It is a figure which shows the processing result by an image generation apparatus. It is a figure which shows an example of the time change of the moving image containing a moving object. (A) is a figure which shows an example of the image of a certain frame obtained by exposure for a short time, (b) is a figure which shows an example of the frame image obtained by embodiment of this invention. It is a figure which shows the structure of the camera which concerns on embodiment of this invention. It is a figure which shows the structure of the display apparatus which concerns on embodiment of this invention. It is the figure which showed the time change of the moving image containing a moving object three-dimensionally. It is a flowchart which shows the conventional image addition method.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Image input part 102 Generation | occurrence | production pixel designation | designated part 103 Motion region analysis part 104 Integration space-time area | region determination part 105 Pixel determination part 106 Pixel value integration part 107 Integration condition determination part 108 Integration condition designation part 109 Image output part 200 Image generation system 210 Imaging Device 220 Image storage device 230 Image generation device 240 Display device 300 Camera 400 Display device 701 to 705 Broken line

Claims (16)

Based on a plurality of temporally continuous images, motion calculating means for calculating the motion of each of the plurality of pixels included in the image of interest among the plurality of images;
For each pixel included in the image of interest, pixel position specifying means for specifying a pixel position corresponding to each pixel for each of the plurality of images based on the movement of each pixel;
Pixel value correction means for correcting the pixel value of each pixel of the image of interest by adding pixel values at pixel positions corresponding to the pixels of the plurality of images for each pixel included in the image of interest; An image generation apparatus comprising: The pixel value correcting means includes
For each pixel included in the image of interest, the pixel value of each pixel is compared with the pixel value at the pixel position corresponding to each pixel to determine whether the pixel value at the pixel position is to be added. A pixel determination unit;
The image generation apparatus according to claim 1, further comprising: an addition unit that adds pixel values at pixel positions that are added by the pixel determination unit. The pixel value correcting unit further includes a condition determining unit that determines whether or not a predetermined condition is satisfied,
The image generation apparatus according to claim 2, wherein the addition unit performs addition processing until the predetermined condition is satisfied. The condition determining unit determines whether or not the number of the pixel values added in the adding unit has reached a predetermined number of frames for each pixel included in the image of interest,
The image generation apparatus according to claim 3, wherein the addition unit performs addition processing until the number of pixel values reaches a predetermined number of frames. The image generation apparatus according to claim 4, wherein the adding unit selects and adds a pixel value at a corresponding pixel position from an image positioned temporally before the target image. The condition determining unit determines whether a ratio between a value obtained by adding in the adding unit and a predetermined noise value is greater than a predetermined threshold for each pixel included in the target image. And
The said addition part performs an addition process until the ratio of the value calculated | required by the said addition and the said predetermined noise value becomes larger than the said predetermined threshold value. Image generation device. The condition determination unit determines whether a ratio between an average value of pixel values added by the addition unit and the predetermined noise value is greater than the predetermined threshold value for each pixel included in the target image. The image generation apparatus according to claim 6, wherein the image generation apparatus determines whether or not the image is generated. The condition determining unit has a ratio between a maximum value of pixel values added by the adding unit and the predetermined noise value for each pixel included in the image of interest greater than the predetermined threshold value. It is determined whether it is. The image generation apparatus of Claim 6 characterized by the above-mentioned. The image generation apparatus according to claim 6, further comprising a noise value storage unit that stores the predetermined noise value in advance. The condition determination unit determines whether or not the addition processing time in the addition unit is greater than a predetermined threshold for each pixel included in the image of interest,
The image generation apparatus according to claim 3, wherein the addition unit performs addition processing until the addition processing time exceeds a predetermined threshold value. Furthermore, it comprises temporary storage means for storing the plurality of images used for the addition processing in the addition unit,
The condition determination unit determines whether or not the capacity of the plurality of images stored in the temporary storage unit is larger than a predetermined threshold value,
The image generation apparatus according to claim 3, wherein the addition unit performs an addition process until a capacity of the plurality of images stored in the temporary storage unit exceeds the predetermined threshold value. The pixel value correcting unit further includes a condition determining unit that determines whether or not the number of the pixel values added in the adding unit reaches a predetermined number of frames for each pixel included in the image of interest,
The adding unit corrects the addition result based on a ratio between the predetermined number of frames and the number of pixel values when the number of added pixel values does not reach the predetermined number of frames. The image generating apparatus according to claim 2. The pixel determination unit
An average value calculating unit that calculates an average value of the pixel value of each pixel and the pixel value at the pixel position corresponding to each pixel;
A variance calculating unit that calculates a variance between a pixel value of each pixel and a pixel value at a pixel position corresponding to each pixel;
Addition target determination that determines a pixel value at the pixel position as an addition target when an absolute value of a difference between the pixel value at the corresponding pixel position and the average value is less than a multiplication value of the variance and a predetermined constant The image generating apparatus according to claim 2, wherein the image generating apparatus includes: The said addition part performs an addition process using the image of both the image ahead and the back image temporally rather than the said attention image. Image generation device. Based on a plurality of temporally continuous images, a motion calculation step of calculating the motion of each of the plurality of pixels included in the image of interest among the plurality of images;
For each pixel included in the image of interest, based on the movement of each pixel, a pixel position specifying step for specifying a pixel position corresponding to each pixel for each of the plurality of images;
For each pixel included in the image of interest, a pixel value correction step of correcting the pixel value of each pixel of the image of interest by adding pixel values at pixel positions corresponding to the pixels of the plurality of images. An image generation method comprising: Based on a plurality of temporally continuous images, a motion calculation step of calculating the motion of each of the plurality of pixels included in the image of interest among the plurality of images;
For each pixel included in the image of interest, based on the movement of each pixel, a pixel position specifying step for specifying a pixel position corresponding to each pixel for each of the plurality of images;
For each pixel included in the image of interest, a pixel value correction step of correcting the pixel value of each pixel of the image of interest by adding pixel values at pixel positions corresponding to the pixels of the plurality of images. A program characterized by being executed by a computer.

Images