JP2012221164A - Motion vector detection device, motion vector detection method and motion vector detection program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a motion vector detection device for accurately detecting a motion vector on a traveling object.SOLUTION: The motion vector detection device includes: means for inputting at least two frame images, calculating the motion vectors of feature points from the input two frame images, and outputting the calculated motion vectors and the latest time frame image; means for detecting the latest time frame image and the motion vectors estimated as erroneous detection vectors from the motion vectors by using template matching, and outputting the motion vectors by removing the detected erroneous detection vectors from the input motion vectors; means for calculating angle parameters on the image about the motion vectors from which the erroneous detection vectors have been removed, creating a featured value list in which the calculated angle parameters and motion vector ID uniquely applied to each of the motion vectors are associated with each other, and outputting the created featured value list; and means for determining the motion vectors by clustering the motion vectors in the featured value list, and outputting them.

Description

  The present invention detects a motion vector from at least two frame images input by a camera that captures a scene including an object moving in real space, and detects a motion vector on the moving object from the motion vector. The present invention relates to a vector detection device, a motion vector detection method, and a motion vector detection program.

  A moving object motion vector that detects a motion vector from at least two frame images input by a camera that captures a scene including an object moving in real space, and detects a motion vector on the moving object from the detected motion vector The detection device can detect, for example, the reverse direction of a pedestrian or a car using the detected motion vector.

  As a conventional moving object motion vector detecting device, a moving object is calculated from one or more frame images using template matching or spatiotemporal differentiation, and clustered according to the similarity of the calculated motion vectors. There is known a method of detecting (see, for example, Non-Patent Document 1).

"Pattern information processing", Yoshiaki Shirai and Masahiko Taniuchi, published by Ohmsha pp. 139-pp. 142, 1998)

  However, in the conventional method, in order to detect a moving object on the assumption that the motion vector calculated by template matching or spatiotemporal differentiation is correct, the detected motion vector has a repetitive texture in the real space to be imaged. Or if there is an object with little texture information, occlusion where the object is occluded by another object, or a false detection vector due to an opening problem caused by an edge part on the captured image, etc. There is a problem that it cannot be detected.

  The present invention has been made in view of such circumstances, and when detecting a motion vector on a moving object, the motion vector on the moving object is accurately detected even if the calculated motion vector includes a false detection vector. An object of the present invention is to provide a motion vector detection device, a motion vector detection method, and a motion vector detection program.

  The present invention inputs at least two frame images, calculates a motion vector of a feature point from the two input frame images, and outputs the calculated motion vector and the latest frame image A motion vector obtained by detecting, using template matching, a motion vector that is estimated as a false detection vector from the detection means, the latest time frame image, and the motion vector, and excluding the detected false detection vector from the input motion vector And an angle parameter on the image for the motion vector excluding the false detection vector, and the calculated angle parameter is associated with a motion vector ID uniquely assigned to each motion vector. A feature quantity calculating means for creating a feature quantity list and outputting the created feature quantity list; By clustering the motion vector in the feature list, characterized by comprising a motion vector clustering means determines and outputs the motion vector.

  The present invention inputs at least two frame images, calculates a motion vector of a feature point from the two input frame images, and outputs the calculated motion vector and the latest frame image A motion vector obtained by detecting a motion vector presumed to be a false detection vector from the latest time frame image and the motion vector using template matching, and removing the detected false detection vector from the input motion vector; And a step of removing the false detection vector, calculating an angle parameter on the image for the motion vector excluding the false detection vector, and associating the calculated angle parameter with a motion vector ID uniquely assigned to each motion vector Create a feature list and output the created feature list. And-up, by clustering the motion vector in the feature list, and having a motion vector clustering step determines and outputs the motion vector.

  The present invention is a motion vector detection program for causing a computer on a motion vector detection device to detect a motion vector, which inputs at least two frame images and obtains a motion vector of a feature point from the two input frame images. A motion vector detection step for calculating and outputting the calculated motion vector and the latest time frame image; and using the template matching of the latest time frame image and a motion vector estimated as a false detection vector from the motion vector A false detection vector removing step for outputting a motion vector obtained by removing the detected false detection vector from the input motion vector, and calculating an angle parameter on the image for the motion vector excluding the false detection vector The calculated angle parameter is uniquely assigned to each motion vector. Generating a feature quantity list in which the motion vector IDs are associated with each other, and a feature quantity calculating step for outputting the created feature quantity list; and clustering the motion vectors in the feature quantity list to determine the motion vector. And causing the computer to perform a motion vector clustering step to be output.

  According to the present invention, after calculating the motion vector, the false detection vector is detected and removed from the calculated motion vector, and the remaining false detection vector and the motion vector on the moving object are separated as a Gaussian mixture distribution. In addition, it is possible to detect the motion vector on the moving object.

It is a block diagram which shows the structure of one Embodiment of this invention. It is a flowchart which shows operation | movement of the false detection vector removal part 11 shown in FIG. It is a flowchart which shows operation | movement of the feature-value calculation part 13 shown in FIG. It is a flowchart which shows operation | movement of the motion vector clustering part 14 shown in FIG. It is explanatory drawing which shows an example of the frame image input into the motion vector detection apparatus. It is explanatory drawing which shows an example of the frame image input into the motion vector detection apparatus. It is explanatory drawing which shows an example of a motion vector calculation result. It is explanatory drawing which shows each area | region in a false detection vector removal process. It is explanatory drawing which shows a feature-value calculation process. It is explanatory drawing which shows an example of a feature-value list | wrist. It is explanatory drawing which shows a motion vector clustering process. It is explanatory drawing which shows a motion vector clustering process. It is explanatory drawing which shows a motion vector clustering process.

  Hereinafter, a motion vector detection apparatus according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the embodiment. In this figure, reference numeral 1 denotes a motion vector detection device constituted by a computer device, which inputs two frame images and outputs a motion vector on a moving object. Reference numeral 2 denotes a storage unit that stores data used for each process of the motion vector detection device 1. The “moving object” here includes an arbitrary moving subject such as a person or an object.

  The motion vector detection device 1 includes a motion vector detection unit 11, an erroneous detection vector removal unit 12, a feature amount calculation unit 13, and a motion vector clustering unit 14. The motion vector detection unit 11 inputs at least two or more frame images F (t−1) and F (t), extracts feature points that are corners or intersections on the image from the frame image at the latest time, and features A motion vector for a point is calculated from at least two frame images using template matching or a spatiotemporal differential method, and the calculated motion vector and a frame image at the latest time t are output.

  The false detection vector removal unit 12 receives the latest frame image F (t) and a motion vector, detects a motion vector estimated as a false detection vector from the motion vector by using template matching, and receives the input motion. A motion vector obtained by removing a false detection vector detected from the vector is output.

  The feature amount calculation unit 13 receives the motion vector output from the false detection vector removal unit 12, calculates the angle parameter on the image over all the motion vectors, and the calculated angle parameter is unique to each motion vector. A feature quantity list in which the given motion vector ID is associated is created, and the created feature quantity list is output.

  The motion vector clustering unit 14 receives the feature amount list output from the feature amount calculation unit 13, clusters the input feature amount list using the EM algorithm, and outputs the list in association with a plurality of output clusters. A motion vector on a moving object is determined and output by threshold processing for a Gaussian distribution parameter.

  Next, processing operations of the motion vector detection device 1 shown in FIG. 1 will be described with reference to the drawings. First, the processing operation of the motion vector detection unit 11 shown in FIG. 1 will be described with reference to FIGS. 5 and 6 are examples of frame images input to the motion vector detection device 1. FIG. 6 shows a frame image F (t) obtained by imaging the real space at the current time with a camera, and FIG. 5 shows a frame image F (t−1) obtained by imaging the same real space one hour before the current time. . The motion vector detection unit 11 extracts feature points that are corners or intersections on the image from the frame image F (t) at the latest time, and the motion vectors for the extracted feature points are used as the frame image F (t) and the frame image F ( Calculated using t-1). Here, the motion vector calculation process is performed by, for example, a template matching process for the frame image F (t−1). That is, by comparing the image data in the vicinity of the feature points already extracted in the frame image F (t), for example, the image data in the vicinity of 8 of the feature points with the image data of the frame image F (t−1), In F (t−1), the image coordinates of a point corresponding to the feature point to be processed on the frame image F (t) are determined. The determined image coordinates and the image coordinates of the feature point to be processed on the frame image F (t) are determined as motion vectors. This process is performed over all feature points extracted in the frame image F (t), and a motion vector and the frame image F (t) are output. The following description will be made on the assumption that the motion vector shown in FIG. 7 has been calculated by this processing.

  Next, the processing operation of the false detection vector removing unit 12 shown in FIG. 1 will be described with reference to FIG. If the end point of the target motion vector is a point near the own point excluding the vicinity of the own point and there is a point having a pattern very similar to the own point, the false detection vector removing unit 12 repeats the presence of texture or texture information. Since there is a high possibility of erroneous detection due to a small amount of data, processing to be excluded from the processing target in the subsequent stage is performed. First, the false detection vector removal unit 12 receives the frame image F (t) and the motion vector output from the motion vector detection unit 11 (step S1). Then, the false detection vector removal unit 12 determines whether or not all the motion vectors have been processed (step S2), and if not, selects one motion vector from the input motion vectors, The end point of the selected motion vector is set as the processing target point.

  Next, the false detection vector removal unit 12 sets a search window of a predetermined size near the processing target point, and stores the image data in the set search window as a template in the storage unit (step S3). Subsequently, the false detection vector removing unit 12 sets a search region indicating the scanning range of the template in the template matching process in the vicinity of the processing target point (step S4). The size of the search area is predetermined and is larger than the set search window size.

  Next, the false detection vector removing unit 12 sets a mask region indicating a range in which template matching processing is not performed in the vicinity of the processing target point (step S5). The size of the mask area is determined in advance, and the size is larger than 0 and smaller than the search area. Subsequently, the false detection vector removing unit 12 performs a template matching process on the image data in the search area excluding the mask area, thereby determining a point similar to the processing target point from the search area (step S6). This similarity determination process can be realized by selecting the point with the highest similarity among the similarities of the points calculated by template matching. As the similarity value, for example, a sum of squares (SSD: Sum of Squared Differences) of pixel values in a window that includes a processing target point in the search window and a search area having the same size as the window. Can be used. Further, not only SSD but also any normalized correlation value and absolute sum of pixel value differences (SAD: Sum of Absolute Differences) may be used as long as they are used for narrowing down corresponding points in an image. For example, when an SSD value is used as the similarity value, a similar point can be determined by selecting the smallest point among the SSD values of the respective points.

  Next, the false detection vector removing unit 12 determines whether or not the determined similarity exceeds a predetermined threshold (step S7), and the similarity of a point similar to the determined processing target point is set to a predetermined threshold. If so, the selected motion vector is removed from the list of input motion vectors (step S8). For example, when an SSD value is used as the similarity value, when the similarity is below a predetermined threshold, that is, when the similarity is similar to the processing target point, the selected motion vector is a list of input motion vectors. Remove from. The false detection vector removal unit 12 performs this process over all the input motion vectors, and then outputs a motion vector (step S9), and ends the process.

  In this way, a motion vector that is presumed to be a false detection vector due to the presence of an object having repetitive texture in the real space or having little texture information, or an opening problem caused by an edge portion on the captured image. By using the template matching, it is possible to remove the erroneously detected motion vector.

  Next, the processing operation of the feature quantity calculation unit 13 shown in FIG. 1 will be described with reference to FIG. First, the feature quantity calculation unit 13 receives the motion vector output from the false detection vector removal unit 12 (step S11). Then, the feature quantity calculation unit 13 determines whether or not all the motion vectors have been processed (step S12). If the result of this determination is that processing has not been completed, the feature quantity calculation unit 13 calculates an angle parameter on the image of the moving vector (step S13), and the motion vector uniquely assigned to each motion vector is the calculated angle parameter. Feature quantity list information associated with the ID is created and stored in the storage unit 2 (step S14). Then, after performing the same processing for all the motion vectors, the feature amount calculation unit 13 outputs the feature amount list stored in the storage unit 2 (step S15), and ends the processing.

The processing performed by the feature amount calculation unit 13 will be described in detail with reference to FIGS. First, the feature amount calculation unit 13 selects one motion vector as a processing target vector from the input motion vectors. Here, description will be made assuming that the start point coordinates of the processing target vector are (Xs, Ys) and the end point coordinates are (Xe, Ye). First, the angle α of the processing target vector is calculated by the equation (1).
α = tan ⁻¹ ((Ys−Ye) / (Xs−Xe)) (1)

  The unit of the angle calculated by equation (1) is radians, but the unit may be converted into an angle. The angle is described using a coordinate system based on the i-axis of the image shown in FIG. Subsequently, the calculated angle is stored in association with the motion vector ID of the processing target vector. Here, the motion vector ID is an ID uniquely assigned to each motion vector. After performing this process for all the input motion vectors, the angle recorded in association with the motion vector ID is output as a feature quantity list (see FIG. 10), and the process is terminated.

In the above description, an example in which only one α that calculates the angle of the processing target vector using Equation (1) is used, but from α calculated using Equation (1), Equation (2) and Equation (3) respectively. _Two calculated α ₁ and α ₂ may be used.
α ₁ = sin α (2)
α ₂ = cos α (3)

  Next, the processing operation of the motion vector clustering unit 14 shown in FIG. 1 will be described with reference to FIG. The motion vector clustering unit 14 receives the feature amount list output from the feature amount calculating unit 13 (step S21). In addition to the motion vector on the moving object, the motion vector represented by this feature amount list includes an occlusion in which the object is concealed by another object and a motion vector detected in error caused by another cause. . FIG. 11 shows a histogram distribution of the angles of the motion vectors expressed by the feature quantity list. Here, the histogram distribution of the angle of the motion vector is modeled and expressed by a Gaussian mixture distribution that is a combination of multiple Gaussian distributions, and the parameters of each peak are determined by performing a known process such as an EM algorithm (Expectation-maximization algorithm). Thus, clustering is performed on each of the motion vector on the moving object, the erroneous motion vector due to occlusion, and the erroneous motion vector due to other causes (step S22).

  Since the EM algorithm is a known algorithm, a detailed description thereof is omitted here. However, the EM algorithm is an algorithm that estimates a parameter of a probability model (normal distribution curve in the present embodiment) in a numerical analysis by an iterative method. Used when the probabilistic model depends on hidden parameters that cannot be observed. The EM algorithm is effective because it easily converges to a good solution compared to other estimation methods, is often simple to implement, and the processing speed is basically high. FIG. 12 shows an example in which a normal distribution curve is estimated by applying the EM algorithm to the angular distribution of FIG.

  In this embodiment, the EM algorithm is used when estimating the normal distribution curve, but other algorithms can naturally be used. Clustering is performed by executing the EM algorithm using the feature list as an input, and a cluster number is assigned in association with the motion vector ID. In addition, Gaussian distribution parameters are output for the number of clusters associated with each cluster. Here, the Gaussian distribution parameter includes at least an average value indicating the center position of the Gaussian distribution, a variance value indicating the spread of the Gaussian distribution, and a weight value indicating a weight for the mixed distribution of the Gaussian distribution. good.

  Next, the motion vector clustering unit 14 determines and outputs the motion vector on the moving object by threshold processing (step S23) for the Gaussian distribution parameters output in association with the plurality of clusters output by the EM algorithm. . This threshold processing can be realized by comparing the variance value of the Gaussian distribution parameter with a predetermined value, for example. It can also be realized by comparing the weight value of the mixed distribution with a predetermined value.

  Next, the motion vector clustering unit 14 outputs the motion vector ID of the feature list belonging to the cluster number determined as the moving object motion vector as the moving object motion vector (step S24), and ends the process. As the output, the start point and end point of the motion vector are recorded together with the angle of the motion vector in association with the motion vector ID when the feature amount list is generated by the feature amount calculation unit 13, and is stored in the motion vector clustering unit 14. The start point and end point of the motion vector associated with the motion vector ID determined as the motion vector motion vector may be output. As a result of the above processing, the vector on the moving object shown in FIG. 13 is detected.

  Although the present invention has been specifically described above based on the embodiments, the description of the above embodiments is for explaining the present invention, and limits the invention described in the claims. Should not be construed as reducing the range. Moreover, each means structure of this invention is not restricted to the said embodiment, Of course, a various deformation | transformation is possible within the technical scope as described in a claim.

  As described above, the error vector whose motion vector group angular distribution does not follow the Gaussian mixture distribution is removed, so that the remaining motion vector group can be approximated by the Gaussian mixture distribution, and then the EM algorithm is performed by motion vector clustering. By clustering the motion vector group by the above, it becomes possible to specify the motion vector caused by the moving object from the motion vector group.

  Further, in order to avoid that the angle of the motion vector is treated as a discontinuous function as an input of the EM algorithm (0) and 360 ° are treated as being different from each other in the feature quantity calculation unit 13 (2) By introducing the equation (3), it is possible to avoid a decrease in clustering accuracy.

  Also, by applying the template matching process to the frame image F (t) as a target, a simple template matching process (for example, a search window and a target window in the search area) that does not require consideration of illumination fluctuations required for comparison with other images. The sum of the absolute values of the pixel value differences between the first and second pixels) can be applied.

  A motion vector detection process is performed by recording a program for realizing the function of the processing unit in FIG. 1 on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium. May be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  Applications where it is essential to detect a motion vector from at least two frame images input by a camera that captures a scene including an object moving in real space, and to detect a motion vector on the moving object from the motion vector Applicable to.

  DESCRIPTION OF SYMBOLS 1 ... Motion vector detection apparatus, 11 ... Motion vector detection part, 12 ... False detection vector removal part, 13 ... Feature-value calculation part, 14 ... Motion vector clustering part, 2 ... Storage

Claims (3)

Motion vector detection means for inputting at least two frame images, calculating a motion vector of a feature point from the two input frame images, and outputting the calculated motion vector and the latest frame image;
A motion vector that is estimated to be a false detection vector from the latest time frame image and the motion vector is detected using template matching, and an error that outputs a motion vector obtained by removing the detected false detection vector from the input motion vector is output. Detection vector removing means;
For the motion vector excluding the false detection vector, an angle parameter on the image is calculated, a feature amount list in which the calculated angle parameter is associated with a motion vector ID uniquely assigned to each motion vector is created, and the created feature A feature quantity calculating means for outputting a quantity list;
And a motion vector clustering means for determining and outputting the motion vector by clustering the motion vectors in the feature quantity list. A motion vector detection step of inputting at least two frame images, calculating a motion vector of a feature point from the two input frame images, and outputting the calculated motion vector and the latest frame image;
A motion vector that is estimated to be a false detection vector from the latest time frame image and the motion vector is detected using template matching, and an error that outputs a motion vector obtained by removing the detected false detection vector from the input motion vector is output. A detection vector removal step;
For the motion vector excluding the false detection vector, an angle parameter on the image is calculated, a feature amount list in which the calculated angle parameter is associated with a motion vector ID uniquely assigned to each motion vector is created, and the created feature A feature quantity calculating step for outputting a quantity list;
A motion vector clustering step of determining and outputting the motion vector by clustering the motion vectors in the feature quantity list. A motion vector detection program for causing a computer on a motion vector detection device to detect a motion vector,
A motion vector detection step of inputting at least two frame images, calculating a motion vector of a feature point from the two input frame images, and outputting the calculated motion vector and the latest frame image;
A motion vector that is estimated to be a false detection vector from the latest time frame image and the motion vector is detected using template matching, and an error that outputs a motion vector obtained by removing the detected false detection vector from the input motion vector is output. A detection vector removal step;
For the motion vector excluding the false detection vector, an angle parameter on the image is calculated, a feature amount list in which the calculated angle parameter is associated with a motion vector ID uniquely assigned to each motion vector is created, and the created feature A feature quantity calculating step for outputting a quantity list;
A motion vector detection program that causes the computer to perform a motion vector clustering step of determining and outputting the motion vector by clustering the motion vectors in the feature quantity list.

Images