JP2015204488A - Motion detection apparatus and motion detection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique capable of improving the accuracy of motion detection.SOLUTION: A motion detection apparatus 1A comprises: a short-exposure image motion detection unit 41 which detects the motion of a short-exposure image on the basis of a past short-exposure image and a current short-exposure image, and generates short-exposure image motion detection information; a long-exposure image motion detection unit 42 which detects the motion of a long-exposure image on the basis of a past long-exposure image and a current long-exposure image, and generates long-exposure image motion detection information; and an integration unit 43 which integrates the long exposure image motion detection information and the short exposure image motion detection information and outputs integrated motion detection information.

Description

  The present invention relates to a motion detection device and a motion detection method.

  In recent years, a short-exposure image (hereinafter also simply referred to as “short-exposure image”) and a long-exposure image (hereinafter also simply referred to as “long-exposure image”) are continuously photographed and combined. An imaging function called WDR (wide dynamic range) or HDR (high dynamic range) that obtains an image that captures a dynamic range that exceeds the dynamic range that the sensor can shoot is increasing. Such a photographing function is particularly effective in a scene with a very large contrast ratio such as a composition of backlight.

  However, due to the mechanism of synthesizing the short-exposure image and the long-exposure image, there is a problem that if the subject moves, a shift occurs during the synthesis and artifacts such as doubled contours occur. In order to reduce such artifacts, a technique is used in which motion information is generated from a captured image, motion information is generated, and adaptive synthesis processing is performed using the motion information. Various techniques have been disclosed as techniques for reducing artifacts using motion information.

  For example, Patent Document 1 focuses on the fact that blur due to subject movement (hereinafter also referred to as “blur”) occurs due to the long exposure time of a long exposure image. A technique is disclosed in which an exposure image is mainly used to perform a combining process to obtain a combined result without blur and artifacts.

  Here, in the technique described in Patent Literature 1, motion detection is performed by normalizing a short exposure image and a long exposure image with an exposure ratio and calculating a difference between the two. In addition, the technique described in Patent Document 1 employs a method of estimating a noise component and subtracting the noise component from an image and calculating a difference in order to eliminate the influence of noise as much as possible.

  In addition, Patent Document 2 discloses a technique for reducing artifacts in a motion region by correcting and correcting a positional deviation of a moving object. Further, Patent Document 3 discloses a technique for reducing artifacts in a motion region by outputting a result of multi-sheet composite WDR when a motion amount is small and outputting a gradation correction result of a single image when the motion amount is large. Is disclosed.

JP 2013-066142 A JP 2013-240031 A JP 2013-093786 A

  However, the technique described in Patent Document 1 has the following problems. First, the problem is that motion cannot be detected in a saturated region in a long exposure image. Since there is no information in the saturated region in the long exposure image, it is difficult to detect motion by comparing the saturated region in the long exposure image with the short exposure image. In particular, when a saturated region in a long exposure image is large and there is a motion in the vicinity thereof, the accuracy of motion detection tends to be lowered.

  Secondly, in a region where the signal amplitude of a short-exposure image is very small, the SN ratio is lowered and motion detection becomes difficult. In the technique described in Patent Document 1, the noise component is estimated to eliminate the influence. However, it is difficult to accurately restore the signal once the SN ratio has been lowered, and the noise component estimation accuracy is also low. There is a limit. Furthermore, if the dynamic range of the WDR process is to be expanded, the exposure time of the short exposure image will be shortened very much, the entire short exposure image will be photographed darkly and the noise amplitude will be increased, and the short exposure image and the long exposure image will be increased. The accuracy of motion detection obtained by comparing with an image tends to decrease.

  Thirdly, when normalizing the pixel values of the long exposure image and the short exposure image with the exposure ratio, it is difficult to ensure normalization accuracy. Basically, the exposure ratio is determined based on the difference in exposure time, but the exposure ratio calculated from the actually captured pixel value may not match the exposure ratio assumed from the set exposure time. In some cases, the exposure ratio calculated from the actually captured pixel value is obtained with decimal precision instead of integer precision. Therefore, it is not easy to accurately match the pixel value levels of the long exposure image and the short exposure image. If the accuracy of normalization is low, it is impossible to distinguish between motion and insufficient accuracy of normalization, so that the accuracy of motion detection decreases.

  Fourthly, when the subject includes a flicker light source, the problem is that the accuracy of motion detection decreases. Since a long exposure image has a long exposure time, it is unlikely to be affected by flicker. However, a short exposure image may be affected by flicker, and some pixel values on the screen may fluctuate. In this case, flicker in the short-exposure image is detected as a motion, which adversely affects the WDR process or flicker remains in the WDR output image. Therefore, generally, when the subject includes flicker, motion detection is disabled.

  In addition, since the techniques described in Patent Documents 2 and 3 perform motion detection by comparing a short exposure image and a long exposure image, they have the same problems as the problems described in Patent Document 1. doing.

  Therefore, the present invention provides a technique capable of improving the accuracy of motion detection.

  According to an embodiment of the present invention, a short-exposure image motion detection unit that detects short-exposure image motion based on a past short-exposure image and a current short-exposure image and generates short-exposure image motion detection information A long exposure image motion detection unit that detects long exposure image motion based on a past long exposure image and a current long exposure image and generates long exposure image motion detection information; and the long exposure image motion detection And an integration unit that integrates the information and the short-exposure image motion detection information and outputs the integrated motion detection information.

  According to this configuration, short exposure image motion detection information is generated by comparing short exposure images, long exposure image motion detection information is generated by comparing long exposure images, and short exposure image motion detection information and long exposure image are generated. The motion detection information is integrated. Therefore, since motion detection information generated by comparing images having the same exposure condition is integrated, the accuracy of motion detection can be improved.

  The short exposure image motion detection unit generates the short exposure image motion detection information based on a relationship between the motion of the short exposure image and a first threshold, and the long exposure image motion detection unit The long-exposure image motion detection information may be generated based on a relationship between a second motion and a second threshold different from the first threshold. The first threshold value and the second threshold value may be set in advance or may be set by the user.

  When the setting is made so that the long exposure image motion detection information and the short exposure image motion detection information are not integrated, the integration unit includes the long exposure image motion detection information and the short exposure image motion detection information. Any one of them may be output. For example, when it is set not to integrate the long exposure image motion detection information and the short exposure image motion detection information, only the long exposure image motion detection information may be output without outputting the short exposure image motion detection information. Then, in the subsequent stage of the integration unit, the short exposure image motion detection information generated from the short exposure image including flicker is not used, and the long exposure image generated from the long exposure image not including flicker is used. Only motion detection information is used.

  The motion detection device includes a use image selection unit that generates a use image selection information by selecting which of a long exposure image and a short exposure image to use for each region, and the motion detection information based on the integrated motion detection information. An update unit that updates use image selection information; and a combining unit that generates a composite image by combining the long exposure image and the short exposure image based on the updated use image selection information. Good. According to this configuration, a more stable composite image is generated based on the generated use image selection information and the integrated motion detection information.

  The motion detection apparatus includes a reduction processing unit that reduces the past long exposure image, the current long exposure image, the past short exposure image, and the current short exposure image, and the short exposure image motion detection The short exposure image motion detection information is generated based on the reduced past short exposure image and the reduced short exposure image at the current time, and the long exposure image motion detection unit is configured to reduce the past long exposure image. The long exposure image motion detection information may be generated based on the exposure image and the reduced long exposure image at the current time.

  According to such a configuration, the reduced past short-exposure image and the reduced short-time exposure image at the current time are written in the memory, and the reduced past short-exposure image and the reduced short-time exposure image at the current time move. It is transferred to the detection unit. Further, the reduced past long exposure image and the reduced current time long exposure image are written in the memory, and the reduced past long exposure image and the reduced current time long exposure image are transferred to the motion detection unit. Is done. Therefore, it is possible to suppress an increase in memory area and an increase in data transfer amount.

  Further, according to another embodiment of the present invention, the step of detecting the movement of the long exposure image based on the past long exposure image and the current long exposure image to generate long exposure image motion detection information; Detecting the motion of the short exposure image based on the past short exposure image and the short exposure image at the current time to generate short exposure image motion detection information; and the long exposure image motion detection information and the short exposure image motion Integrating the detection information and outputting the integrated motion detection information.

  According to this method, the short exposure image motion detection information is generated by comparing the short exposure images, the long exposure image motion detection information is generated by comparing the long exposure images, and the short exposure image motion detection information and the long exposure image are compared. The motion detection information is integrated. Therefore, since motion detection information generated by comparing images having the same exposure condition is integrated, the accuracy of motion detection can be improved.

  As described above, according to the present invention, it is possible to improve the accuracy of motion detection.

It is a figure which shows the function structural example of the motion detection apparatus which concerns on the 1st Embodiment of this invention. It is a figure showing a mode that the pair of a long exposure image and a short exposure image was image | photographed continuously. It is a block diagram which shows the detailed functional structural example of a motion detection part. It is a flowchart which shows the example of operation | movement of the use image selection which concerns on the 1st Embodiment of this invention. It is a figure for demonstrating the 1st example of the effect which the same embodiment has. It is a figure for demonstrating the 2nd example of the effect which the same embodiment has. It is a figure which shows the function structural example of the motion detection apparatus which concerns on the 2nd Embodiment of this invention.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

  In the present specification and drawings, a plurality of constituent elements having substantially the same functional configuration may be distinguished by attaching different alphabets after the same reference numeral. However, when it is not necessary to particularly distinguish each of a plurality of constituent elements having substantially the same functional configuration, only the same reference numerals are given.

(First embodiment)
First, a first embodiment of the present invention will be described. First, the functional configuration of the motion detection device 1A according to the first embodiment of the present invention will be described. FIG. 1 is a diagram showing a functional configuration of a motion detection device 1A according to the first embodiment of the present invention. As illustrated in FIG. 1, the motion detection device 1A includes a sensor 10, a frame memory 20A, a use image selection unit 30, a motion detection unit 40, an update unit 50, a WDR synthesis unit 70, and a gradation compression unit 80. Hereinafter, the function of each functional block included in the motion detection device 1A will be sequentially described in detail.

  The motion detection apparatus 1A changes the exposure setting of the sensor 10 and continuously shoots two images. Here, it is assumed that short exposure shooting is performed first, and then long exposure shooting is performed. However, long exposure photography may be performed first and then short exposure photography. The short-exposure image and the long-exposure image photographed in this way are written as a pair in the frame memory 20A. Shooting of the long exposure image and the short exposure image and writing of the captured long exposure image and the short exposure image to the frame memory 20A are continuously performed.

  FIG. 2 is a diagram illustrating a state in which a pair of a long exposure image and a short exposure image is continuously captured. Referring to FIG. 2, a short exposure image (t−1), a long exposure image (t−1), a short exposure image (t), a long exposure image (t), a short exposure image (t + 1), and a long exposure image (t + 1). ) Is shown in time series. Referring to FIG. 2, the shooting time of each image is shown in parentheses, but in the present specification and this drawing, the shooting time of each image may also be shown in parentheses.

  Here, in the present specification, at least the past long exposure image, the current long exposure image, the past short exposure image, and the current short exposure image need only remain in the frame memory 20A. Therefore, the interval at which the long exposure image and the short exposure image are taken is not particularly limited. Further, each frame of the long exposure image and the short exposure image may be written in the frame memory 20A, or may be written once in a plurality of frames in the frame memory 20A.

  For example, as in the example shown in FIG. 1, when there are two sets of areas in which pairs of long exposure images and short exposure images are written, the motion detection device 1A sets two pairs of long exposure images and short exposure images. What is necessary is just to write alternately with respect to the area | region of a group. For example, the short exposure image (t−1) and the long exposure image (t−1) written in the frame memory 20A may be overwritten by the short exposure image (t + 1) and the long exposure image (t + 1).

  In the example shown in FIG. 1, the motion detection apparatus 1A has one common system for outputting the long exposure image and the short exposure image, and the sensor 10 sometimes outputs the long exposure image and the short exposure image. Although the output is divided, the long exposure image and the short exposure image may be output simultaneously. In such a case, the motion detection apparatus 1A only needs to have two systems: a system for outputting a long exposure image from the sensor 10 and a system for outputting a short exposure image. Each shutter time is determined by, for example, a dynamic range of a subject to be imaged, a sensor specification, and the like.

  In the embodiments of the present invention, the terms short exposure image and long exposure image are used, but these terms do not limit the absolute exposure time of each of the two captured images. Therefore, when two images with different exposure times are taken, an image with a relatively short exposure time corresponds to a short exposure image and an image with a relatively long exposure time is a long exposure. Corresponds to an image.

  The sensor 10 is configured by an image sensor that forms an image of light from the outside on the light receiving plane of the image sensor, photoelectrically converts the imaged light into a charge amount, and converts the charge amount into an electric signal. The type of the image sensor is not particularly limited, and may be, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor).

  The use image selection unit 30 refers to the short exposure image (t) and the long exposure image (t) read from the frame memory 20A, and saturates and moves each of the long exposure image (t) and the short exposure image (t). Are used to generate use image selection information for selecting either the short exposure image (t) or the long exposure image (t) as the use image. As an algorithm for selecting either the short exposure image or the long exposure image, various algorithms are assumed.

  For example, since a region that has been saturated in the long exposure image is highly likely not to be saturated in the short exposure image, the short exposure image may be selected as the use image of the region. However, with this process alone, artifacts such as a double outline may occur in an area where there is a large movement. For this reason, a process of detecting a motion and reducing a phenomenon that a contour becomes double based on the motion may be performed. An algorithm for selecting either a short exposure image or a long exposure image including such processing is not particularly limited.

  As described above, the use image selection information may be a set of binary data indicating which of the short exposure image and the long exposure image is to be selected. It may be a set of mixing ratios indicating whether to mix at a ratio. For example, the use image selection unit 30 may increase the mixing ratio of the short-exposure images as the degree of saturation of the long-exposure images increases. In addition, the use image selection unit 30 may increase the mixing ratio of the short exposure image as the movement of the short exposure image or the long exposure image increases. The algorithm for calculating the mixing ratio of the short exposure image and the long exposure image is not particularly limited.

  The motion detector 40 detects motion with reference to the short exposure image (t-1), the long exposure image (t-1), the short exposure image (t), and the long exposure image (t). Details of the functions of the motion detection unit 40 will be described with reference to FIG. FIG. 3 is a block diagram illustrating a detailed functional configuration example of the motion detection unit. Referring to FIG. 3, the motion detection unit 40 includes a short exposure image motion detection unit 41, a long exposure image motion detection unit 42, and an integration unit 43. Hereinafter, functions of the short exposure image motion detection unit 41, the long exposure image motion detection unit 42, and the integration unit 43 will be described.

  The short exposure image motion detection unit 41 detects the motion of the short exposure image based on the short exposure image (t-1) and the short exposure image (t), and detects the short exposure image motion based on the motion of the short exposure image. Generate information. The generation method of the short exposure image motion detection information is not particularly limited. For example, the short exposure image motion detection unit 41 detects the short exposure image motion based on the relationship between the detected short exposure image motion and a first threshold (hereinafter also referred to as “short exposure image motion detection threshold”). Information only needs to be generated.

  Specifically, the short-exposure image motion detection unit 41 detects a difference between pixel values or gradients of corresponding regions in the short-exposure image (t−1) and the short-exposure image (t), and the difference is short-exposure. A region larger than the image motion detection threshold may be detected as a motion region. On the other hand, the short exposure image motion detection unit 41 may detect a region where the difference is smaller than the short exposure image motion detection threshold as a non-motion region. An area having the same difference as the short-exposure image motion detection threshold value may be detected as any area. The short exposure image motion detection unit 41 may generate such a detection result as short exposure image motion detection information.

  The long exposure image motion detection unit 42 detects the motion of the long exposure image based on the long exposure image (t-1) and the long exposure image (t), and detects the long exposure image motion based on the motion of the long exposure image. Generate information. The generation method of the long exposure image motion detection information is not particularly limited. For example, the long exposure image motion detection unit 42 detects the long exposure image motion based on the relationship between the detected long exposure image motion and a second threshold (hereinafter also referred to as “long exposure image motion detection threshold”). Information only needs to be generated.

  Specifically, the long exposure image motion detection unit 42 detects a difference in pixel value or gradient in a corresponding region between the long exposure image (t−1) and the long exposure image (t), and the difference is long exposure. A region larger than the image motion detection threshold may be detected as a motion region. On the other hand, the long exposure image motion detection unit 42 may detect a region where the difference is smaller than the long exposure image motion detection threshold as a non-motion region. An area having the same difference as the long-exposure image motion detection threshold may be detected as any area. The long exposure image motion detection unit 42 may generate such a detection result as long exposure image motion detection information.

  The integration unit 43 integrates the long exposure image motion detection information and the short exposure image motion detection information and outputs the integrated motion detection information. There is no particular limitation on how long exposure image motion detection information and short exposure image motion detection information are integrated. For example, the integration unit 43 may set a region that is a motion region in at least one of the long exposure image motion detection information and the short exposure image motion detection information as the motion region. On the other hand, the integration unit 43 may set a non-motion area as a non-motion area in both the long exposure image motion detection information and the short exposure image motion detection information.

  In this way, it is expected that the accuracy of motion detection is improved by integrating the long exposure image motion detection information and the short exposure image motion detection information. The short exposure image motion detection threshold and the long exposure image motion detection threshold described above may be set in advance or may be set by the user. Also, different values may be set for the short exposure image motion detection threshold and the long exposure image motion detection threshold. Here, either the short exposure image motion detection threshold or the long exposure image motion detection threshold may be a large value.

  For example, when motion detection information is generated based on the relationship between the pixel value difference and the threshold value, the short exposure image motion detection threshold value may be larger than the long exposure image motion detection threshold value. . Since a long exposure image contains less noise, if it is determined that a region in which a minute pixel value difference occurs is also a motion region, the accuracy of motion detection increases, while a short exposure image has a large amplitude. This is because noise tends to be included, and it is expected to prevent erroneous determination that the area where the noise is generated is a motion area.

  The integration unit 43 may always integrate the long exposure image motion detection information and the short exposure image motion detection information, but may not integrate the long exposure image motion detection information and the short exposure image motion detection information. Good. For example, when the setting is made so that the long exposure image motion detection information and the short exposure image motion detection information are not integrated, the integration unit 43 selects either the long exposure image motion detection information or the short exposure image motion detection information. Either of them may be output. The setting that the long exposure image motion detection information and the short exposure image motion detection information are not integrated may be made by the user.

  Which of the long exposure image motion detection information and the short exposure image motion detection information is output is not particularly limited. Here, for example, when the long exposure image motion detection information and the short exposure image motion detection information are not integrated, for example, when the subject includes flicker. In general, since a long exposure image has a long exposure time, it does not include flicker, and only a short exposure image includes flicker. Further, flicker often moves, and flicker may be detected as movement even though the object is not actually moving at all. As a result, using the short exposure image motion detection information may adversely affect the WDR synthesis result.

  In order to prevent such an adverse effect, the integration unit 43 outputs the long exposure image motion detection information without outputting the short exposure image motion detection information when it is set not to integrate the long exposure image motion detection information and the short exposure image motion detection information. Only the exposure image motion detection information may be output. Then, in the subsequent stage of the integration unit 43, the short exposure image motion detection information generated from the short exposure image including flicker is not used, and the long exposure generated from the long exposure image not including flicker is used. Only image motion detection information is used.

  The update unit 50 updates the use image selection information generated by the use image selection unit 30 based on the motion detection information integrated by the integration unit 43 of the motion detection unit 40. Specifically, the update unit 50 may refer to the integrated motion detection information and update the use image selection information so that the short exposure image is selected as the use image in the motion region. By doing so, artifacts such as a double outline of the composite image can be reduced. On the other hand, the update unit 50 may refer to the integrated motion detection information so as not to update the use image selection information generated by the use image selection unit 30 in the non-motion region.

  Here, an example of the operation of selecting a used image will be described in more detail. FIG. 4 is a flowchart showing an example of the operation of selecting used images according to the first embodiment of the present invention. Such an operation can be performed by the use image selection unit 30 and the update unit 50. Note that the use image selection operation illustrated in FIG. 4 is merely an example, and the use image selection operation is not limited to the example illustrated in FIG. 4.

  As shown in FIG. 4, the integrated motion detection information is referred to, and if the target pixel is a motion region (“Yes” in step S11), a short exposure image is selected (step S13). On the other hand, if the target pixel is a non-motion region (“No” in step S11), if the long exposure image is saturated (“Yes” in step S12), the short exposure image is selected (step S13). ) If the long exposure image is not saturated (“No” in step S12), the long exposure image is selected (step S14).

  Here, when the operation of selecting used images for all the pixels has not been completed (“No” in step S15), the operation is shifted to step S11, and the operation of selecting used images for the next pixel is continued. Done. On the other hand, when the operation of selecting used images for all pixels is completed (“Yes” in step S15), the operation of selecting used images is ended.

  The WDR synthesis unit 70 receives the updated use image selection information from the update unit 50, and generates a WDR image by synthesizing the short exposure image and the long exposure image based on the updated use image selection information. . Specifically, the WDR synthesis unit 70 refers to the updated use image selection information, uses the short exposure image for the short exposure image use region, and uses the long exposure image for the long exposure image use region. To generate a composite image.

  The combining method by the WDR combining unit 70 is not particularly limited. For example, it is assumed that a value indicating that a long exposure image is selected is “0” and a value indicating that a short exposure image is selected is “1”. In such a case, the WDR synthesizing unit 70 sets α as the mixing ratio constituting the used image selection information, and α × (pixel value of the short exposure image) + (pixel value) corresponding to the long exposure image and the short exposure image. 1−α) × (pixel value of the long exposure image) can be calculated, and the calculation result can be used as a combined image (WDR image).

  The gradation compression unit 80 performs compression processing on the WDR image generated by the WDR synthesis unit 70 so that the bit range of the image signal having a wide dynamic range falls within a predetermined bit range. As such compression processing, tone mapping according to a look-up table (LUT) may be used, but any method may be used.

  The subsequent stage of the gradation compression unit 80 is connected to an image processing engine including, for example, a demosaic unit that generates an RGB plane from Bayer data, a contour enhancement unit, and color management. For this reason, it is preferable that the data amount of the output signal from the gradation compression unit 80 is adjusted so as to match the size of the input data to the image processing engine (for example, about 12 bits). If the data size is simply reduced, the image is converted into a dark image. Therefore, it is preferable that the high luminance side is strongly compressed so as to approximate human visual characteristics.

  In the example described above, an example in which two types of images (short exposure image and long exposure image) having different exposure amounts are captured and combined has been described. However, the type of image used for composition is not particularly limited. For example, the present embodiment can be applied to an example in which three or more types of images having different exposure amounts are captured and combined.

  For example, when using a short exposure image, a medium exposure image, and a long exposure image, a motion detection unit corresponding to each of the three images is provided, and the three motion detection information is integrated. At this time, if the setting is made so that the motion detection information is not integrated, it is sufficient to use only the motion detection information of an image that does not include flicker. An image that does not include flicker may change depending on the flicker frequency and the like, and is not particularly limited, but may be a long exposure image.

  Then, the example of the effect which the 1st Embodiment of this invention has is demonstrated. FIG. 5 is a diagram for explaining a first example of the effect exhibited by the first embodiment of the present invention. Referring to FIG. 5, a short exposure image (t-1), a long exposure image (t-1), a short exposure image (t), and a long exposure image (t) are shown. Each image shows a scene of a bright outdoors shot from the room. The short-exposure image shows the movement of clouds outside the window, but the window region is saturated in the long-exposure image.

  In such a scene, the movement of the clouds cannot be detected when the conventional technique for comparing the short exposure image and the long exposure image is used. Specifically, when using the prior art, there is no information in the saturated area of the long exposure image, so distinguish whether the clouds appearing in the short exposure image are moving or stationary. I can't. Referring to FIG. 5, an example of motion detection information obtained when the conventional technology is used is shown as “motion detection information of the prior art”.

  On the other hand, in the first embodiment of the present invention, motion is detected from past and current time images taken with the same exposure amount. Therefore, the movement of the cloud can be detected by comparing the short exposure images of the past and the current time. Referring to FIG. 5, an example of motion detection information obtained when the embodiment of the present invention is used is shown as “motion detection information of the present embodiment”. In addition, according to the first embodiment of the present invention, highly accurate motion detection information is obtained by integrating the short exposure image motion detection information and the long exposure image motion detection information.

  Taking this as an example, the advantage of the first embodiment of the present invention is that “motion detection is performed between images taken under the same exposure conditions”, and this advantage is also effective for interference other than saturation. For example, even when blur is present in a long exposure image, if motion detection is performed by comparing short exposure images, motion can be detected by comparing blurless images. can get.

  Even when the short-exposure image includes noise with a large amplitude, by performing motion detection by comparing long-exposure images with little noise, it is possible to detect an accurate motion that does not consider the influence of noise. Further, by performing motion detection by comparing short-exposure images including noise, it is possible to detect a clear motion that does not include erroneous detection due to noise. Then, by integrating the motion detection information of both, highly accurate motion detection information can be obtained.

  Furthermore, in the prior art, it is necessary to normalize the short-exposure image and the long-exposure image after comparing them according to the exposure ratio. On the other hand, in the first embodiment of the present invention, it is only necessary to perform motion detection between images taken under the same exposure conditions, the accuracy of normalization is not a problem, and the motion is always accurate regardless of the exposure ratio. Detection information is obtained.

  FIG. 6 is a diagram for explaining a second example of the effect exhibited by the first embodiment of the present invention. Referring to FIG. 6, a short exposure image (t-1), a long exposure image (t-1), a short exposure image (t), and a long exposure image (t) are shown. Each image shows a scene of a bright outdoors shot from the room. A flicker band appears in the wall area of the short exposure image (t-1) with the room light as a flicker light source, and the position of the flicker band moves in the vertical direction in the short exposure image (t). Yes. On the other hand, the flicker band is not photographed in the long exposure image.

  In such a scene, when a conventional technique for comparing a short exposure image and a long exposure image is used, a flicker band is detected as a motion, and thus erroneous detection occurs in a wide range. Referring to FIG. 6, an example of motion detection information obtained when the conventional technology is used is shown as “motion detection information of the prior art”. Referring to this “prior art motion detection information”, the flicker band has been detected as motion.

  On the other hand, when the first embodiment of the present invention is used, motion detection is performed by comparing images taken under the same exposure conditions. Therefore, if the setting is made not to integrate the long exposure image motion detection information and the short exposure image motion detection information and only the motion detection information of the long exposure image not affected by flicker is used, the flicker band is determined as the motion. And an accurate motion detection result can be obtained. Referring to FIG. 6, an example of motion detection information obtained when the embodiment of the present invention is used is shown as “motion detection information of the present embodiment”. Referring to the “motion detection information of this embodiment”, the flicker band is not detected as a motion.

(Second Embodiment)
Subsequently, a second embodiment of the present invention will be described. In the first embodiment of the present invention, it is necessary to store a past short-exposure image and a long-exposure image in a memory as compared with the prior art, and an increase in a memory area and an increase in data transfer amount are realized in terms of hardware. Can become a bottleneck. In the second embodiment of the present invention, a description will be given of a configuration that can achieve the same effects as the first embodiment of the present invention and can suppress an increase in memory area and an increase in data transfer amount.

  FIG. 7 is a diagram showing a functional configuration of a motion detection device 1B according to the second embodiment of the present invention. As shown in FIG. 7, the motion detection device 1B according to the second embodiment of the present invention is compared with the motion detection device 1A according to the first embodiment of the present invention. 15 is further provided. Hereinafter, the reduction processing unit 15 will be mainly described in detail.

  The reduction processing unit 15 reduces the past long exposure image, the current long exposure image, the past short exposure image, and the current short exposure image. The method for generating the reduced image is not particularly limited, but a set of average values for each rectangular area constituting the image may be generated as the reduced image. Referring to FIG. 7, each reduced image includes a frame memory as a reduced long exposure image (t−1), a reduced long exposure image (t), a reduced short exposure image (t−1), and a reduced short exposure image (t). 20B.

  The long exposure image motion detection unit 42 generates long exposure image motion detection information based on the reduced long exposure image (t−1) and the reduced long exposure image (t). The short exposure image motion detection unit 41 generates short exposure image motion detection information based on the reduced short exposure image (t−1) and the reduced short exposure image (t). According to such a configuration, it is possible to suppress an increase in memory area and an increase in data transfer amount. Note that the short exposure image and the long exposure image referred to by the use image selection unit 30 do not have to be reduced. Referring to FIG. 7, the short-exposure image and the long-exposure image that have not been reduced are written in the frame memory 20B as a full-size short-exposure image (t) and a full-length-exposure image (t).

  The effect of the second embodiment of the present invention will be described with a specific example. For example, if the reduction ratio is set to 1/10 both vertically and horizontally, the data amount of the reduced image is 1/100 of the data amount of the original image. Therefore, the data amount of 4 frames of reduced images (the past long exposure image, the current long exposure image, the past short exposure image, and the current short exposure image) is also 1/25 (4 Therefore, in the second embodiment of the present invention, an increase in the data amount can be remarkably suppressed as compared with the prior art.

  Of course, the spatial resolution of the motion detection information is also considered to be 1/100. However, if motion detection is performed with reference to a reduced image, there is an advantage that noise can be reduced during the reduction process and motion detection can be performed using a high quality image. In addition, performing motion detection with reference to a reduced image is equivalent to performing motion detection with reference to a certain region, and thus has an advantage that a spatially robust detection result can be obtained. In addition, even if the spatial resolution of motion detection information is somewhat reduced, it can be said that motion detection information with sufficient accuracy to apply motion adaptation processing to a moving object having a certain size is obtained. It can be said that there is almost no substantial demerit of this configuration using images.

  Heretofore, the first embodiment and the second embodiment of the present invention have been described. According to the first embodiment of the present invention, the short exposure image motion that detects the motion of the short exposure image based on the past short exposure image and the short exposure image at the current time and generates the short exposure image motion detection information. A long exposure image motion detection unit 42 that detects the motion of the long exposure image based on the past long exposure image and the long exposure image at the current time to generate long exposure image motion detection information; There is provided a motion detection device 1A including an integration unit 43 that integrates image motion detection information and short exposure image motion detection information and outputs integrated motion detection information.

  According to this configuration, short exposure image motion detection information is generated by comparing short exposure images, long exposure image motion detection information is generated by comparing long exposure images, and short exposure image motion detection information and long exposure image are generated. The motion detection information is integrated. Therefore, since motion detection information generated by comparing images having the same exposure condition is integrated, the accuracy of motion detection can be improved.

  According to the second embodiment of the present invention, the reduction processing unit 15 that reduces the past long exposure image, the current long exposure image, the past short exposure image, and the current short exposure image is provided. The exposure image motion detection unit 41 generates short exposure image motion detection information based on the reduced past short exposure image and the reduced short exposure image at the current time, and the long exposure image motion detection unit 42 is reduced. A motion detection apparatus 1B is provided that generates long exposure image motion detection information based on the past long exposure image and the reduced long exposure image at the current time.

  According to such a configuration, the reduced past short-exposure image and the reduced short-time exposure image at the current time are written in the memory, and the reduced past short-exposure image and the reduced short-time exposure image at the current time move. It is transferred to the detection unit. Further, the reduced past long exposure image and the reduced current time long exposure image are written in the memory, and the reduced past long exposure image and the reduced current time long exposure image are transferred to the motion detection unit. Is done. Therefore, it is possible to suppress an increase in memory area and an increase in data transfer amount.

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

  Further, in the technical field, artifacts at the time of moving object synthesis are a very big problem, and the accuracy of the motion detection information used for reducing the artifacts is important. According to the embodiment of the present invention, stable motion detection is possible even when the captured image includes various disturbances, and the embodiment of the present invention can realize it with a realistic circuit scale. The point is excellent.

1 (1A, 1B) motion detection device 10 sensor 15 reduction processing unit 20 (20A, 20B) frame memory 30 used image selection unit 40 motion detection unit 41 short exposure image motion detection unit 42 long exposure image motion detection unit 43 integration unit 50 Update unit 70 WDR synthesis unit 80 Gradation compression unit

Claims (6)

A short exposure image motion detection unit that detects the motion of the short exposure image based on the past short exposure image and the short exposure image at the current time and generates short exposure image motion detection information;
A long exposure image motion detection unit that detects the motion of the long exposure image based on the past long exposure image and the long exposure image at the current time and generates long exposure image motion detection information;
An integration unit that outputs the motion detection information integrated by integrating the long exposure image motion detection information and the short exposure image motion detection information;
A motion detection device comprising: The short exposure image motion detection unit generates the short exposure image motion detection information based on a relationship between the motion of the short exposure image and a first threshold;
The long exposure image motion detection unit generates the long exposure image motion detection information based on a relationship between the motion of the long exposure image and a second threshold different from the first threshold;
The motion detection device according to claim 1. When the setting is made so that the long exposure image motion detection information and the short exposure image motion detection information are not integrated, the integration unit includes the long exposure image motion detection information and the short exposure image motion detection information. Output one of them,
The motion detection device according to claim 1. The motion detection device includes:
A use image selection unit that selects, for each region, whether to use a long exposure image or a short exposure image, and generates use image selection information;
An update unit that updates the use image selection information based on the integrated motion detection information;
A combining unit that generates a combined image by combining the long exposure image and the short exposure image based on the updated use image selection information;
The motion detection device according to claim 1, comprising: The motion detection device includes:
A reduction processing unit for reducing the past long exposure image, the current long exposure image, the past short exposure image, and the current short exposure image;
The short exposure image motion detection unit generates the short exposure image motion detection information based on the reduced past short exposure image and the reduced short exposure image at the current time,
The long exposure image motion detection unit generates the long exposure image motion detection information based on the reduced past long exposure image and the reduced current exposure long time image.
The motion detection device according to claim 1. Detecting the motion of the long exposure image based on the past long exposure image and the current long exposure image to generate long exposure image motion detection information;
Detecting the motion of the short exposure image based on the past short exposure image and the short exposure image at the current time to generate short exposure image motion detection information;
Integrating the long exposure image motion detection information and the short exposure image motion detection information to output integrated motion detection information;
Including a motion detection method.