JP2679778B2 - Image motion detection device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION INDUSTRIAL APPLICABILITY The present invention is directed to shake and panning of an image pickup device.
The present invention relates to an image motion detection device configured to be able to detect only an object having a motion different from the background even when the entire captured image moves.

2. Description of the Related Art Conventionally, as a device for detecting a moving object in an image, there is a device as shown in FIG. This is because the image signal of the previous frame is stored in the frame memory 3, and when the image signal 34 of the current frame is input, the difference between the signal of each pixel of the previous frame and the current frame is calculated, and this difference value is a predetermined value. When the value is exceeded, the pixel is regarded as a moving area. For example, as shown in FIG. 11 (a), when an object moves while the background is stationary, this device calculates the difference between two frames to generate an image in the image as shown in FIG. 11 (b). Only moving objects can be detected as moving objects.

However, in the conventional image motion detection device, as shown in FIG. 12 (a), when the background is not stationary due to the motion of the imaging device such as panning, this device is As shown in FIG. 12 (b), the background is also detected as a moving area by taking the difference between the frames, and therefore only the moving object cannot be detected in the image.

The present invention solves the above-mentioned conventional problems, and an object thereof is to provide an image motion detection device capable of detecting only an object having a motion different from the background even when the background is not stationary. To do.

Means for Solving the Problems The present invention according to claim 1 is a detection means for detecting the overall movement of an image signal, and a correction for correcting the overall movement of the image signal based on the movement detected by the detection means. Means, a moving area detecting means for detecting a changing area of a signal of a corrected image corrected by the correcting means as a moving area, and the movement of the detected moving area due to the incomplete correction. The movement of the image provided with a filter means for removing the area erroneously detected by the area detecting means and adaptively changing the reference of the size of the area to be removed according to the motion vector of the entire screen. It is a detection device.

According to a second aspect of the present invention, there is provided detection means for detecting the entire movement of the image signal, correction means for correcting the entire movement of the image signal based on the movement detected by the detection means, and the correction means. Among the detected moving areas, the moving area detecting unit that compares at least two frames of the corrected images and detects an area having a different signal as a moving area from the comparison result, and the correction is incomplete. In order to eliminate the region erroneously detected by the motion region detecting means, the size standard of the region to be removed is adaptively changed according to the amplitude or variance of the motion vector of the entire screen. It is an image motion detection device provided with a filter means.

According to a third aspect of the present invention, there is provided detection means for detecting the entire movement of the image signal, correction means for correcting the entire movement of the image signal based on the movement detected by the detection means, and the correction means. Means for detecting regions having different signal values using the corrected image, and means for detecting the different regions because the correction is incomplete according to the detected magnitude of movement of the entire screen. The image motion detecting device is provided with a removing unit that predicts a region erroneously detected by the above and adaptively removes it.

According to a fourth aspect of the present invention, a detection unit that detects the entire motion of the image signal as a motion vector, a correction unit that corrects the entire motion of the image signal based on the detected motion vector, and the correction unit. According to the amplitude amount or the variance of the detected motion vector, comparison detection means for comparing images of at least two frames including the corrected image corrected by the above, and detecting regions having different signal values from the comparison result. An image motion detection apparatus comprising: a removal unit that predicts a region erroneously detected by the comparison detection unit because the correction is incomplete and adaptively removes the region in which the correction is incomplete. .

According to a fifth aspect of the present invention, output means for detecting an image pickup state by using an acceleration sensor, converting the detection result into a signal, and outputting the signal, and detecting the entire movement of the image signal by using the signal representing the image pickup state. The detection means, the correction means for correcting the entire movement of the image signal based on the movement detected by the detection means, and the image of at least two frames including the correction image corrected by the correction means are compared. An image motion detection apparatus is provided with a comparison detection unit that detects regions having different signal values from the comparison result.

According to a sixth aspect of the present invention, output means for outputting a signal indicating the image pickup state, detection means for detecting the entire movement of the image signal using the signal indicating the image pickup state, and the detection means detect the movement. A correction unit that corrects the entire motion of the image signal based on the motion and an image of at least two frames including the corrected image corrected by the correction unit are compared, and from the comparison result, a region having a different signal value is compared. And a comparison detection means for predicting the size of an area where the correction is incomplete according to the entire movement of the detected image signal, and the comparison detection means because the correction is incomplete. The image motion detecting apparatus is provided with a removing unit that adaptively removes the region erroneously detected by the above.

Action In the image motion detecting device of the present invention, the motion of the entire screen of the image due to the change of the imaging direction of the imaging device is corrected by the means for correcting the entire motion. By performing motion detection using the corrected output image signal, it is possible to detect only a moving object having a motion different from the overall motion. Further, since the incompletely corrected area is removed by the removing means in the areas where the detected signal values are different from each other, it is possible to eliminate an error in the detection of the motion vector or an edge of the background portion due to image distortion or the like. No false detection of motion area.

Embodiments Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing an image motion detecting apparatus according to a first embodiment of the present invention. In FIG. 1, 1 is a motion vector detecting means, 2 is a representative motion vector determining means, 3 and 3a are frame memories, 4 is a reading control means, 5 is a difference, threshold processing means, 6 is a median filter, 7 Is a motion amplitude / dispersion calculating means, and 34 is an input terminal.

The operation of the image motion detection device will be described with reference to FIG.

First, in FIG. 2, 8 is the image of the previous frame, 9 is the image of the current frame, 10 is the motion vector of the moving object in the image, 11 is the motion vector of the background in the image, 12 is the moving object in the image, 13 is each motion vector detection area on the screen, 14 is a 0 vector, 15 is an inverse vector of the motion vector of the background in the image, 16 is a moving object detected by performing difference processing and threshold processing on the images of two frames. Area, 17 is the edge of the background detected as a motion area when the image is subjected to difference and threshold processing because the motion correction of 2 frames is not perfect, 18 is the screen position of the previous frame, 19 is the screen position of the current frame Is.

First, as shown in FIG. 2 (a), an image 9 of the current frame having a general motion including the background and the moving object 12 is input to the image 8 of the previous frame, to the input terminal 34 of the present apparatus. .
In the motion vector detecting means 1, as shown in FIG. 2 (b), each motion vector detecting area 13 is obtained by dividing the screen into 16 parts.
Then, the motion vector of the image is detected respectively. Moving object 12
The motion vector 10 of the moving object 12 is detected mainly in the area where the background motion vector 11 exists, and the background motion vector 11 is detected mainly in the area where the background pattern exists. Further, the 0 vector 14 is detected in an area where no pattern exists. The representative motion vector determination means 2 determines and outputs, as a background motion vector 11, a vector in which the statistically largest number of 16 vectors detected from each motion vector detection area 13 coincides. The method of calculating the vector in which the statistically largest number of vectors match is to take a two-dimensional histogram for the vector values in the horizontal and vertical directions of the vector, and obtain the vector value that maximizes the two-dimensional histogram value. Is. This operation will be briefly described with reference to FIGS. 2 (f) and 2 (g). FIG. 2 (f) is a plot of the points of the motion vector input to the representative motion vector determination means 2 within the detection range. FIG. 2 (g) shows this histogram viewed only in the horizontal direction. The background motion vector 11 is output by detecting this maximum point. The frame memory 3 stores the image signal of the current frame while the background motion vector 11 is detected. Based on the motion vector 11, the read-out control means 4 uses the output position 18 of the previous frame as a reference and the output position of the current frame in the direction of the inverse vector 15 of the motion vector 11 as shown in FIG. 2 (c). The image signal is read from the frame memory 3 by shifting 19 and output. The frame memory 3a stores the output of the previous frame of the read control means 4. The difference / threshold processing means 5 takes the difference between the image signal of the previous frame from the frame memory 3a and the image signal of the current frame of the reading control means 4, and the pixel whose absolute value exceeds the threshold value. Is output as a motion area.
The output is the shaded portion such as 16 and 17 in FIG. 2 (d). However, a slight error in motion vector detection,
Or, if the movement of the background cannot be completely corrected by parallel movement due to image distortion such as shading, or rotation of the screen, it may be detected as a movement area at the edge portion of the background picture as shown in 17. . Since this generally occurs only in the edge portion in the form of dots or lines, it can be removed by the median filter 6, and only the moving region 16 by the moving object is output as shown in FIG. 2 (e). The size of the portion detected as the motion area generated in the edge portion of the background can be predicted from the size of the motion vector of the background when the cause is image distortion such as shading. When the detection error or the rotation of the screen is the cause, the degree of coincidence of the motion vectors of the respective detection areas detected by the motion vector detecting means 1, that is, the motion gathering in the background motion vector 11 shown in FIG. 2 (g). It can be predicted from the magnitude of the vector variance 29. The motion amplitude / dispersion calculating means 7 obtains the amplitude and variance 29 of the background motion vector 11 and outputs it to the median filter 6. The median filter 6 changes the size of the matrix of the median filter according to the background motion vector 11 and the variance 29 input from the motion amplitude / variance calculating means 7. This makes it possible to remove even if the motion is large and the size of the motion area generated at the edge portion of the background becomes large.

Next, a second embodiment of the present invention will be described with reference to the drawings. FIG. 3 is a block diagram showing an image motion detecting apparatus according to a second embodiment of the present invention. In FIG. 3, the same elements as those in FIG. 1 are given the same numbers. The operation of the motion detection device for this image
This will be described with reference to the drawings. In FIG. 4, as shown in FIG.
The same elements are given the same numbers.

First, in the input terminal 34 of this device, as in the first embodiment,
As shown in FIG. 4 (a), an image 9 of the current frame having general motion including the background and the moving object 12 is input to the image 8 of the previous frame. The operations of the motion vector detection 1 and the representative motion vector determination means 2 are the same as in the first embodiment. The frame memory 3 stores the image signal of the current frame while the background motion vector 11 is detected. At the same time, the frame memory 3a stores the image signal of the previous frame. The read control means 4 is a motion vector
As shown in FIG. 4 (c), the output position 18 of the previous frame is shifted in the direction of the motion vector 11 on the basis of the position 19 of the current frame on the basis of 11 to obtain the image signal from the frame memory 3a. Read and output. Difference and threshold processing means 5
Is the image signal of the current frame from the frame memory 3,
The difference between the image signals of the previous frame output from the reading control means 4 is calculated, and the pixel whose absolute value exceeds the threshold value is output as the motion area. The output is shown in Fig. 4 (d).
It is a portion shown by diagonal lines such as 16 and 17. Similarly to the first embodiment, when the background motion cannot be completely corrected due to a slight error in motion vector detection or image distortion such as shading, as shown in 17, the edge portion of the background pattern is It may be detected as a moving area.
The amplitude calculating means 7 and the median filter 6 operate in the same manner as in the first embodiment, and are removed to output only the moving area 16 by the moving object as shown in FIG. 4 (e). In the first embodiment, the position of the current frame is shifted with reference to the corrected output screen of the previous frame, so that the movement of the moving object is based on the corrected coordinates of the screen as shown in FIG. 2 (e). The area 16 is output. In the second embodiment, since the position of the current frame is shifted based on the current frame, the moving object is moved based on the coordinates of the screen of the current field as shown in FIG. 4 (e). The motion area 16 by is output.

Hereinafter, a third embodiment of the present invention will be described with reference to the drawings. FIG. 5 is a block diagram of an image motion detecting apparatus according to the third embodiment of the present invention. In FIG. 5, the same elements as in FIG. 2 are given the same numbers. In FIG. 5, 1a is a motion vector detecting means. The operation of this image motion detecting device will be described with reference to FIG. In FIG. 6 (a), the same elements as those in FIG. 2 are given the same numbers.

In FIG. 6 (b), 26 is a correlation function, 27 is a correlation function obtained in the background portion, and 28 is a correlation function obtained in the region of the moving object.

First, as shown in FIG. 6A, an image 9 of the current frame having a general motion including the background and the moving object 12 is input to the image 8 of the previous frame, as shown in FIG. In the motion vector detecting means 1a, first, as shown in FIG. 6B for the data of the entire screen, the absolute value of the difference in the deviation within the motion vector detection range is obtained as the correlation function 26. This correlation function 26 is the correlation function 27 obtained in the background part.
And the correlation function 28 obtained in the moving object area. Here, if the area of the moving object is sufficiently small with respect to the background, the average amplitude of the correlation function 27 becomes sufficiently larger than that of the correlation function 28. Therefore, by obtaining the minimum point of the correlation function 26, the background motion vector 11 Can be asked. In this way, the motion vector detection means 1a finds the minimum point of the correlation function 26, detects the background motion vector 11, and outputs it. The frame memory 3 stores the image signal of the current frame while the background motion vector 11 is detected. At the same time, the frame memory 3a stores the image signal of the previous frame. After that, the operations of the read control means 4, the difference, the threshold value processing means 5, and the median filter 6 are the same as those in the second embodiment.

Hereinafter, a fourth embodiment of the present invention will be described with reference to the drawings. FIG. 7 is a block diagram of an image motion detecting device according to the fourth embodiment of the present invention. In FIG. 7, the same elements as those in FIG. 1 are given the same numbers. In FIG. 7, 20 is an acceleration sensor attached to the image pickup apparatus, 21 is a zoom magnification sensor, 22 is a motion calculation means, 26 is a read-out control, and data interpolation means. The operation of this image motion detecting device will be described with reference to FIG. In FIG. 8, the same elements as in FIG. 2 are given the same numbers. In FIG. 8, 23 is a motion vector of the screen center part of the current frame with respect to the previous frame, 24 is a rotation angle of motion of a peripheral part around the screen center of the current frame with respect to the previous frame, and 25 is a current frame with respect to the previous frame. This is the rate of change in zoom magnification.

First, as shown in FIG. 8 (a), an image 9 of the current frame having a general motion including the background and the moving object 12 is input to the image 8 of the previous frame to the input terminal 34 of the present apparatus. .
The frame memory 3a stores the image signal of the previous frame.

At this time, the acceleration sensor 20 attached to the imaging device
As a result, the change in the angle of the image pickup device in the shooting direction and the rotation angle about the shooting direction are input to the movement calculation means 22. At the same time, the zoom magnification is input from the zoom magnification sensor 21 to the movement calculation means 22. The motion calculation means 22 is the eighth
The motion vector 23 of the screen center portion of the current frame with respect to the previous frame shown in FIG. 10A, the rotation angle 2 of the motion of the peripheral portion around the screen center of the current frame with respect to the previous frame 2
4. Calculate the rate of change 25 in the zoom factor of the current frame with respect to the previous frame. Read control, data interpolation means 26
As shown in FIG. 8 (b), the image signal of the previous frame is read from the frame memory 3a with reference to the position 19 of the current frame, data interpolation is performed, and the result is output. The difference / threshold processing means 5 calculates the difference between the image signal of the current frame from the frame memory 3 and the image signal of the previous frame output from the reading control means 4, and the absolute value thereof exceeds the threshold. The pixels of are output as a motion area. The output is the shaded portion such as 16 and 17 in FIG. 8 (c). Similarly to the first embodiment, when the background motion cannot be completely corrected due to a slight error in motion vector detection or image distortion such as shading, as shown in 17, the edge portion of the background pattern is It may be detected as a moving area. The amplitude calculating means 7 and the median filter 6 operate in the same manner as in the first embodiment, and are removed to output only the moving area 16 by the moving object as shown in FIG. 8 (d).

In the first and second embodiments, the movement of the parallel movement of the entire image between the two frames is corrected. However, not only the movement of the parallel movement but also the movement of the entire rotation and enlargement is corrected by, for example, the first movement. 7 (a) and 7 (b) are detected, and one of the two frames is rotated and enlarged as shown in FIGS. 8 (a) and 8 (b) to correct the overall movement between the two frames. However, it is naturally conceivable that the device detects a motion area different from the overall motion.

Further, in the fourth embodiment, the changes in the image pickup direction and the zoom magnification are calculated by using the outputs of the acceleration sensor and the zoom magnification sensor, but this may be any signal as long as it can signal the image pickup state.

For example, as a fifth embodiment, an image motion detecting device shown in FIG. 9 can be considered. The difference from the fourth embodiment is that it is provided with means 30 for controlling the image pickup direction of the image pickup apparatus, means 31 for controlling the zoom magnification, and the like.
A point for outputting a zoom magnification control signal, and also a motion calculation means
In 22, the control signal in the image pickup direction is used instead of the output of the acceleration sensor of the fourth embodiment, and the zoom magnification control signal is used instead of the zoom magnification sensor.

EFFECTS OF THE INVENTION As described above, the present invention has means for correcting the entire movement of the image signal and means for comparing two frame images output from the correction means and detecting areas having different output values. It is possible to realize an image motion detection device capable of detecting only a moving object having a motion different from the motion of the entire image even when the entire image is moving due to shaking or panning of the imaging device.

[Brief description of the drawings]

FIG. 1 is a block diagram of an image motion detecting apparatus according to the first embodiment of the present invention, and FIG. 2 is an image diagram for explaining the operation of the image motion detecting apparatus according to the first embodiment of the present invention. Third
FIG. 4 is a block diagram of an image motion detecting apparatus according to the second embodiment of the present invention, and FIG. 4 is an image diagram for explaining the operation of the image motion detecting apparatus according to the second embodiment of the present invention. FIG. 7 is a block diagram of an image motion detecting apparatus according to the third embodiment of the present invention. FIG. 6 is an image diagram for explaining the operation of the image motion detecting apparatus according to the third embodiment of the present invention. The figure is a block diagram of an image motion detecting apparatus according to a fourth embodiment of the present invention.
FIG. 8 is an image diagram for explaining the operation of the image motion detecting device according to the fourth embodiment of the present invention, and FIG. 9 is a block diagram of the image motion detecting device according to the fifth embodiment of the present invention. 10th
FIG. 11 is a block diagram of a conventional image motion detection device,
FIG. 12 is an image diagram for explaining the operation of the conventional image motion detection device. 1, 1a ... Motion vector detection means, 2 ... Representative motion vector determination means, 3, 3a ... Frame memory, 4 ... Readout control means, 5 ... Difference, threshold value processing means, 6 ...
Median filter, 7 ... Motion amplitude / dispersion calculation means,
8: previous frame image, 9: current frame image, 10
...... Motion vector of moving object in image, 11 …… Motion vector of background in image, 12 …… Moveable object in image, 13 …… Each motion vector detection area on screen, 14 …… 0 vector, 15 …
… Inverse vector of the background motion vector in the image, 16 …… Motion area due to moving object, 17 …… Background edge detected as motion area, 18 …… Screen position of previous frame, 19…
… Screen position of current frame, 20… Accelerometer, 21…
Zoom magnification sensor, 22 ... Motion calculation means, 23 ... Background motion vector at the screen center, 24 ... Rotation angle of background motion at the screen center, 25 ... Zoom magnification change, 26 ...
... Read-out control, data interpolation means, 29 ... Motion vector dispersion, 30 ... Imaging direction control means, 31 ... Zoom magnification control means, 34 ... Image input terminal.

Claims (6)

(57) [Claims] 1. A detection means for detecting the overall movement of an image signal, a correction means for correcting the overall movement of the image signal based on the movement detected by the detection means, and a correction corrected by the correction means. A moving area detecting unit that detects a changing area of an image signal as a moving area, and removes an area of the detected moving areas that is erroneously detected by the moving area detecting unit because the correction is incomplete. In addition, the image motion detection apparatus is provided with a filter unit that adaptively changes the size reference of the area to be removed according to the motion vector of the entire screen. 2. A detecting means for detecting the entire movement of the image signal, a correcting means for correcting the entire movement of the image signal based on the movement detected by the detecting means, and a correction corrected by the correcting means. At least two frames including images are compared with each other, and a moving area detecting unit for detecting an area having different signals as a moving area from the comparison result, and the correction is incomplete among the detected moving areas. Filter means for removing the area erroneously detected by the motion area detecting means, and adaptively changing the size criterion of the area to be removed according to the amplitude or variance of the motion vector of the entire screen. An image motion detecting device characterized by being provided. 3. A detection means for detecting the overall movement of the image signal, a correction means for correcting the overall movement of the image signal based on the movement detected by the detection means, and a correction corrected by the correction means. It is erroneously detected by a unit that detects an area having a different signal value using an image and a unit that detects the different area because the correction is incomplete depending on the detected magnitude of movement of the entire screen. A motion detecting device for an image, comprising: a removing unit that predicts a region and adaptively removes the region. 4. Detection means for detecting the entire movement of the image signal as a motion vector, and correction means for correcting the entire movement of the image signal based on the detected movement vector.
Comparison detection means for comparing images of at least two frames including the correction image corrected by the correction means, and detecting areas having different signal values from the comparison result, and the amplitude amount or variance of the detected motion vector. According to the above, the image forming apparatus further comprises: a removing unit that predicts a region erroneously detected by the comparison detecting unit because the correction is incomplete and adaptively removes the region in which the correction is incomplete. Image motion detection device. 5. An output means for detecting an image pickup state by using an acceleration sensor, converting the detection result into a signal and outputting the signal, and a detecting means for detecting the entire movement of the image signal by using the signal indicating the image pickup state. Based on the movement detected by the detection means, a correction means for correcting the entire movement of the image signal and an image of at least two frames including the corrected image corrected by the correction means are compared, and the comparison result An image motion detecting apparatus, comprising: a comparison detecting unit for detecting regions having different signal values from the image detecting unit. 6. An output means for outputting a signal indicating an image pickup state, a detecting means for detecting the entire movement of an image signal using the signal showing the image pickup state, and a movement detected by the detecting means. In addition, comparison is performed by comparing a correction unit that corrects the entire movement of the image signal with an image of at least two frames including the correction image that has been corrected by the correction unit, and detect a region having a different signal value from the comparison result. Depending on the detection means and the overall movement of the detected image signal,
And a removing unit that adaptively removes the region erroneously detected by the comparison detecting unit because the size of the region where the correction is incomplete is predicted. Image motion detection device.