CN108133172B

CN108133172B - Method for classifying moving objects in video and method and device for analyzing traffic flow

Info

Publication number: CN108133172B
Application number: CN201711138992.4A
Authority: CN
Inventors: 宋景选; 曹黎俊
Original assignee: Beijing Huadaoxing Technology Co ltd
Current assignee: Beijing Huadaoxing Technology Co ltd
Priority date: 2017-11-16
Filing date: 2017-11-16
Publication date: 2022-04-05
Anticipated expiration: 2037-11-16
Also published as: CN108133172A

Abstract

The invention discloses a method for classifying moving objects in a video, and a method and a device for analyzing traffic flow. The method comprises the following steps: extracting the motion trail of each target object in the video; performing similarity modeling of a space-time relation on the motion tracks of the target objects, and determining the space-time similarity between the motion tracks of the target objects; and clustering the motion tracks of the target objects by utilizing the space-time similarity between the motion tracks of the target objects to obtain a group of the target objects which are similar in time and space. The method and the device can ensure the rapidity and the accuracy of video data monitoring and the effectiveness of abnormal event detection.

Description

Method for classifying moving objects in video and method and device for analyzing traffic flow

Technical Field

The invention relates to the field of pattern recognition, machine learning and computer vision, in particular to a method for classifying moving objects in a video and a method and a device for analyzing traffic flow.

Background

With the rapid development of digital network technology, video images become an important carrier for information transfer. By the end of 2011, only the number of food monitoring cameras in Guangdong province breaks through 110 thousands. Meanwhile, the monitoring data generated by these monitoring cameras is increasing. The large amount of rich motion information contained in video image sequences has attracted a great deal of interest.

The traditional monitoring method relies on manual monitoring of the video, which obviously increases more and more labor cost. Because the number of video paths monitored simultaneously by manpower is limited, for some huge systems, the number of monitoring paths is large, and a plurality of persons are required to monitor simultaneously. The more critical problem is that the monitoring video is monitored for a long time, so that the vision of monitoring personnel becomes tired, the attention is distracted, a lot of visual information in the picture can be 'invisible' by human eyes, the monitoring efficiency and accuracy are not high, and even the phenomena of missing report or false report can occur. In addition, for a large-scale camera network and massive monitoring data, useful data only account for a few parts, and it is almost impossible to search for the abnormality from the massive data by manpower. Therefore, although the human eyes can directly distinguish moving objects from the new sequence of video images and extract moving information, the requirement of social development cannot be met by only relying on the natural intelligence of human beings to acquire and process the moving information.

Based on this, the human cost of video data monitoring is reduced, human vision is replaced by computer vision, motion information is extracted, analyzed and understood from a video image sequence, and the efficiency, accuracy and effectiveness of monitoring are improved, so that the technical problem to be solved urgently is formed.

Disclosure of Invention

In view of the above, the present invention has been made to provide a method and apparatus for classifying moving objects in a video that overcomes or at least partially solves the above problems.

In a first aspect, an embodiment of the present invention provides a method for classifying moving objects in a video, including:

extracting the motion trail of each target object in the video;

performing similarity modeling of a space-time relation on the motion tracks of the target objects, and determining the space-time similarity between the motion tracks of the target objects;

and clustering the motion tracks of the target objects by utilizing the space-time similarity between the motion tracks of the target objects to obtain a group of the target objects which are similar in time and space.

In some optional embodiments, the extracting the motion trajectory of each target object in the video specifically includes:

and detecting each target object from the video sequence, tracking each target object in a time sequence to obtain the spatial position of each target object in the time sequence, and acquiring the motion tracks of all the target objects.

In some optional embodiments, after obtaining the motion trajectories of all the target objects, the method further includes:

and preprocessing the motion trail of the target object to remove noise and the motion trail which does not meet the preset requirement.

In some optional embodiments, the modeling of the similarity of the spatio-temporal relationship of the motion trajectories of the target objects specifically includes:

respectively analyzing the space similarity and the time sequence relation information between every two target objects according to the motion tracks of the target objects;

and fusing the time sequence relationship information between every two motion tracks of each target object into the space similarity between every two motion tracks of each target object, and establishing a similarity model of the space-time relationship between every two motion tracks of each target object.

In some optional embodiments, analyzing the spatial similarity between each two motion trajectories of each target object specifically includes:

for a given track A and a given track B, calculating a spatial distance f (A, B) between the track A and the track B, and normalizing the spatial distance f (A, B) to obtain a final spatial similarity between the track A and the track B:

F(A,B)＝exp(-f(A,B)/σ) (1)

in the above formula (1), σ is a normalized scale parameter.

In some optional embodiments, analyzing the time-series relationship information between each two motion trajectories of each target object specifically includes:

calculate the timing weight W between given track A and track B:

W＝1/(1+exp(-C)) (2)

in the above formula (2), the calculation formula of the parameter C is as follows:

in the above formula (3), Δ d is the time-series coincidence ratio between the track A and the track B, and η is the track A and the track BRatio of short-time traces to long-time traces in trace B, and η_tThen is the timing length ratio threshold between track a and track B,

and

the time sequence lengths of the motion tracks A and B are respectively, and K is an exponential parameter.

In some optional embodiments, establishing a similarity model of a space-time relationship between each two motion trajectories of each target object specifically includes:

weighting the spatial similarity between every two target objects by utilizing the time sequence relation information between every two target objects to obtain a space-time similarity model between two motion tracks as follows:

in the above equation (4), F and w represent the spatial similarity and the timing weight between the trajectories, respectively, and λ is a scale factor.

In a second aspect, an embodiment of the present invention provides a method for analyzing traffic flow, including:

extracting the motion trail of each vehicle object in the video;

carrying out similarity modeling of a space-time relation on the motion tracks of the vehicles, and determining the space-time similarity between the motion tracks of the vehicles;

clustering the motion tracks of the vehicles by using the space-time similarity between the motion tracks of the vehicles to obtain a vehicle group with similar time and space;

and carrying out statistical analysis on the vehicles in each vehicle group to obtain the traffic flow information in the preset area.

In a third aspect, an embodiment of the present invention provides an apparatus for classifying a moving object in a video, including:

the acquisition module is used for extracting the motion trail of each target object in the video;

the modeling module is used for carrying out similarity modeling of a space-time relation on the motion tracks of the target objects and determining the space-time similarity between the motion tracks of the target objects;

and the clustering module is used for clustering the motion tracks of the target objects by utilizing the space-time similarity between the motion tracks of the target objects to obtain a group of the target objects which are similar in time and space.

In some optional embodiments, the obtaining module is specifically configured to:

In some optional embodiments, the obtaining module is further configured to:

In some optional embodiments, the modeling module comprises:

the spatial similarity analysis submodule is used for respectively analyzing the spatial similarity between every two target objects according to the motion tracks of the target objects;

the time sequence relation information analysis submodule is used for respectively analyzing the time sequence relation information between every two target objects for the motion tracks of the target objects;

and the modeling submodule is used for fusing the time sequence relation information between every two motion tracks of each target object into the space similarity between every two motion tracks of each target object and establishing a similarity model of the space-time relation between every two motion tracks of each target object.

In some optional embodiments, the spatial similarity analysis submodule is specifically configured to:

F(A,B)＝exp(-f(A,B)/σ) (1)

in the above formula (1), σ is a normalized scale parameter.

In some optional embodiments, the timing relationship information analysis sub-module is specifically configured to:

calculate the timing weight W between given track A and track B:

W＝1/(1+exp(-C)) (2)

in the above formula (3), Δ d is the time-series coincidence ratio between the trajectory A and the trajectory B, η is the ratio of the time-series short trajectory to the time-series long trajectory in the trajectory A and the trajectory B, and η_tThen is the timing length ratio threshold between track a and track B,

and

In some optional embodiments, the modeling submodule is specifically configured to:

In a fourth aspect, an embodiment of the present invention provides an apparatus for analyzing a traffic flow, including:

the acquisition module is used for extracting the motion trail of each vehicle object in the video;

the modeling module is used for carrying out similarity modeling of the space-time relationship on the motion tracks of the vehicles and determining the space-time similarity between the motion tracks of the vehicles;

the clustering module is used for clustering the motion tracks of the vehicles by utilizing the space-time similarity among the motion tracks of the vehicles to obtain a vehicle group similar in time and space;

and the analysis module is used for carrying out statistical analysis on the vehicles in each vehicle group to obtain the traffic flow information in the preset area.

In a fifth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium having stored thereon computer instructions, which when executed by a processor, implement the steps of:

extracting the motion trail of each target object in the video;

The technical scheme provided by the embodiment of the invention has the beneficial effects that at least:

the spatial-temporal relation between the motion tracks is analyzed to obtain a spatial-temporal similarity model of the motion tracks, the motion tracks are clustered, and the dynamic group is detected and analyzed according to the clustering group, so that the labor cost is saved, and the rapidness, accuracy and effectiveness of dynamic group analysis can be better guaranteed.

The time sequence information between the motion tracks is fused into the space similarity, and the time-space similarity modeling is carried out, so that the motion tracks appearing in the same time sequence section and the motion tracks existing in different time sequence sections can be measured through a designed unified time-space similarity model, the time-space dynamic relation between the motion tracks in a longer time sequence range can be mined, more global dynamic analysis in the longer time sequence range can be better carried out on the dynamic group, and the more robust and higher-precision dynamic group detection performance can be obtained.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:

fig. 1 is a flowchart of a method for classifying moving objects in a video according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a motion trajectory according to an embodiment of the present invention;

FIG. 3 is a flowchart of a similarity modeling method for a motion trajectory spatiotemporal relationship according to an embodiment of the present invention;

FIG. 4 is a diagram illustrating an exemplary timing relationship between motion trajectories according to an embodiment of the present invention;

fig. 5 is a flowchart of a specific implementation flow of the method for analyzing traffic flow according to the second embodiment of the present invention;

FIG. 6 is a flowchart of a method for analyzing spatial similarity of motion trajectories according to a second embodiment of the present invention;

FIG. 7 is a diagram illustrating a process of clustering motion trajectories according to a second embodiment of the present invention;

FIG. 8 is a schematic structural diagram of an apparatus for classifying moving objects in a video according to an embodiment of the present invention;

FIG. 9 is a sub-module structure diagram of a modeling module in an embodiment of the invention;

fig. 10 is a schematic structural diagram of an analysis apparatus for traffic flow according to an embodiment of the present invention.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

In order to solve the problems that manual detection of dynamic groups wastes manpower, the speed is low, the accuracy is low, report omission and report error are more likely to occur, and even some work is almost impossible to be completed by manpower in the prior art, the embodiment of the invention provides a dynamic group analysis method, which can quickly and accurately implement dynamic group monitoring analysis and ensure the high efficiency, reliability and effectiveness of the analysis.

Example one

An embodiment of the present invention provides a method for classifying moving objects in a video, a flow of which is shown in fig. 1, and the method includes the following steps:

step S101: and extracting the motion trail of each target object in the video.

The target object may be an object appearing in the video to be analyzed, such as a vehicle or a crowd, or any other specific object to be analyzed.

In the above step S101, all moving target objects, which are key points, are detected from the video sequence, and the selection of the moving target object detection method takes into consideration the following factors:

1. the motion characteristics of the target object comprise the track characteristics and the speed characteristics of the motion of the target object;

2. the background characteristics comprise the speed of background change, the intensity of background light, the existence of shielding and the like;

3. the requirements of the detection system include real-time detection, detection precision, calculation amount and the like.

The detection method of the moving target object is selected by comprehensively considering the above factors, and commonly used methods for detecting the moving target object include a frame difference method, a background difference method, an optical flow method, and the like. For example, the moving speed of the target object is moderate, the target object does not move too fast or too slow, the background change is fast, the requirement on the real-time performance is high, and a frame difference method with a simple algorithm can be selected; when the background changes, a background difference method can be selected; the optical flow method may be selected when the accuracy is required to be high and a large amount of calculation can be performed.

The embodiment of the present invention may also implement detection on the target object by using other target object detection methods, which are not limited herein.

And selecting a detection method according to the analysis method, detecting each target object from the video sequence to serve as a key point, tracking the key point on a time sequence to obtain the spatial position of the key point on the time sequence, and acquiring the motion tracks of all the target objects.

FIG. 2 is an exemplary diagram of a motion trajectory, looking at trajectory A as a combination of a series of observation points:

wherein

Represents the spatiotemporal position of the ith observation point,

the coordinates of the ith observation point in the spatial dimension,

coordinates of the ith observation point in the time dimension.

Due to illumination change, shielding and the like, it is difficult to obtain a long motion trajectory and certain noise, so after obtaining the motion trajectories of all target objects, the motion trajectories of the target objects can be preferably preprocessed to remove noise and motion trajectories that do not meet preset requirements. The detection result is more accurate.

Step S102: and performing similarity modeling of the space-time relationship on the motion tracks of the target objects, and determining the space-time similarity between the motion tracks of the target objects.

This step S102 can be implemented in the following manner: respectively analyzing the spatial similarity between every two target objects and the time sequence relation information between every two target objects according to the motion tracks of the target objects; and fusing the time sequence relationship information between every two motion tracks of each target object into the space similarity between every two motion tracks of each target object, and establishing a similarity model of the space-time relationship between every two motion tracks of each target object.

The similarity modeling method of the motion trajectory space-time relationship can be shown by referring to fig. 3, and comprises the following steps:

step S1021: and analyzing the spatial similarity between every two motion tracks of each target object.

F(A,B)＝exp(-f(A,B)/σ) (1)

in the above formula (1), σ is a normalized scale parameter.

The spatial similarity between two motion trajectories is measured by their spatial distance, which is commonly used, for example, in the following: euclidean distance (Euclidean distance), Mahalanobis distance (Mahalanobis distance), Minkowski distance (Minkowski distance), Hausdorff distance (Hausdorff distance), and cosine distance, among others. The appropriate similarity measure is selected based on the type and characteristics of the motion trajectory, and the particular needs of the analysis problem.

Step S1022: and analyzing the time sequence relation information between every two motion tracks of each target object.

In step S1022, the timing relationship information may be represented by timing weights between the motion trajectories; further, the timing weight W between a given track A and a given track B may be calculated by:

W＝1/(1+exp(-C)) (2)

and

The timing relationship between the motion trajectories can be divided into two parts: and a time sequence overlapping part and a time sequence non-overlapping part, referring to fig. 4, A, B, C three motion tracks, wherein track a and track B have overlapping and non-overlapping parts in time sequence, track a and track C have non-overlapping parts in time sequence, and track B and track C have overlapping and non-overlapping parts in time sequence.

Wherein deltad is defined as the time sequence coincidence degree between the motion tracks, the motion track which appears at the beginning is defined as A, and the other track which appears later is defined as B during specific calculation, then

Δd＝t_{A_end}-t_{B_start} (4)

In the above formula (4), t_{A_end}And t_{B_start}Respectively representing the end instant of track A and the start instant of track B, said instants being measured in particular by frames, e.g. t _{A_end}27 indicates the number of frames in which track A endsIs 27.

Δ d >0, indicating that two motion trajectories overlap in time sequence, for example, the time sequence overlap ratio Δ d between trajectory a and trajectory B in fig. 4 is 27-9-18 >0, and the time sequence overlap ratio Δ d between trajectory B and trajectory C is 42-34-8 > 0; Δ d <0, indicating that there is no timing overlap between the two motion trajectories, e.g., 27-34-7 <0 timing overlap ratio Δ d between trajectory a and trajectory C in fig. 4.

It should be noted that the steps S1021 and S1022 are independent of each other, and there is no strict chronological order.

Step S1023: and establishing a similarity model of the space-time relationship between every two motion tracks of each target object.

In step S1023, the temporal relationship information between each two motion trajectories of each target object may be used to weight the spatial similarity between each two motion trajectories, so as to obtain a spatial-temporal similarity model between two motion trajectories:

in the above equation (5), F and w represent the spatial similarity and the timing weight between the trajectories, respectively, and λ is a scale factor.

According to the method, a similarity model of the space-time relationship between every two motion tracks of all the target objects is established.

Step S103: and clustering the motion tracks of the target objects by utilizing the space-time similarity between the motion tracks of the target objects to obtain a group of the target objects which are close in time and space.

And clustering the motion tracks of the target objects by utilizing the space-time similarity among the motion tracks of the target objects, wherein the commonly used motion track clustering methods comprise division clustering, hierarchical clustering, density clustering, neural network clustering, statistical clustering and the like, and the suitable clustering method can be selected according to the actual condition of the target object to be analyzed, and other suitable clustering methods can also be selected.

Selecting a clustering method, and dividing the motion tracks with similar behaviors, namely clustering the motion tracks with closer spatial distance and shorter time sequence interval into one class; dividing the motion tracks with different behaviors, namely dividing the motion tracks with longer spatial distance or longer time sequence interval, and obtaining the groups of target objects which are close in time and space so as to be used for group analysis of the motion objects.

Step S104: and performing group analysis on the target object according to the group of the target object.

For example, the target objects may be subjected to global group analysis according to the group of target objects obtained by the above method, such as analyzing the group density, the moving speed, the motion law, and the like of the target objects. Meanwhile, abnormal events including position abnormality, azimuth abnormality, speed abnormality and the like can be detected through analysis.

In the method of the embodiment, the spatiotemporal relationship between the motion tracks is analyzed to obtain the spatiotemporal similarity model of the motion tracks, the motion tracks are clustered, and the dynamic group is detected and analyzed according to the clustering group, so that the labor cost is saved, and the rapidity, the accuracy and the effectiveness of the analysis of the dynamic group can be better ensured.

Example two

The second embodiment of the present invention provides a specific implementation manner that the method for classifying moving objects in a video is applied to an application scenario of traffic flow analysis, where the implementation process is directed to the analysis of traffic flow, and the method for analyzing traffic flow, as shown in fig. 5, specifically includes the following steps:

step S501: an input video sequence is acquired.

And inputting a video sequence of traffic flow information shot by the road camera. The video sequence may be input in real time or may be integrated. The traffic flow analysis device acquires the video sequence of the traffic flow information shot by the input road camera, and further determines the time sequence length of the video sequence analyzed each time, wherein the time sequence length of the video sequence analyzed each time can be determined according to the following modes:

the first method is as follows: according to the requirement of the real-time performance of analysis, if the requirement of the real-time performance of analysis is high, for example, a vehicle close to a violation state, a violation vehicle or an accident vehicle needs to be detected in real time so as to be processed in time, the shorter time sequence length of the video sequence analyzed each time is set on the premise of ensuring that the number of motion tracks has enough representativeness; if only the density characteristic, the moving speed, the motion rule and other characteristics of the vehicle need to be analyzed, but the requirement on the real-time performance of the analysis is not high, the longer time sequence length of the video sequence analyzed each time can be set.

The second method comprises the following steps: determining according to the change speed of an actual video sequence, for example, if the change speed of the video sequence is high, the motion trajectory in the video sequence with a shorter time sequence length can meet the representative requirement, and the shorter time sequence length of the video sequence analyzed each time can be set; if the video sequence changes slowly, the time sequence length of each analyzed video sequence is set to be longer so as to meet the requirement that the motion track of the video sequence is representative.

The third method comprises the following steps: and comprehensively determining the time sequence length of the video sequence analyzed each time according to the real-time requirement and the change speed of the actual video sequence.

In this embodiment, the time sequence length of the traffic flow video analyzed each time can be set to 200 frames, the first 200 frames of video sequences are extracted, the motion tracks of the vehicles are extracted from the video sequences, preprocessing is performed, pairwise temporal-spatial similarity analysis is performed on the preprocessed motion tracks, and then clustering analysis is performed on all the tracks; and then, performing the same operation on the subsequent 200 frames of video sequences, and so on until all the video sequences are analyzed.

Step S502: and extracting the motion trail of each vehicle object in the video.

Selecting a vehicle detection method: since the vehicle may move fast, slowly, or even still, and the background changes due to light, occlusion, etc., the optical flow method may be selected to detect the vehicle in the video sequence.

The change of the gray scale of each pixel of the image is regarded as motion, the change rate of the gray scale is a velocity vector, and all the velocity vectors form a light velocity field. If there are no moving objects in the image, the optical flow vector of the entire image is uniformly varied. If a moving object exists in the image, the optical flow vector of the moving object is necessarily different from the optical flow vector of the neighborhood background, so that the position of the moving object can be detected. The optical flow method can detect vehicles moving slowly or even being stationary, and is less affected by environmental changes.

And detecting the vehicles in the video sequence by an optical flow method, tracking the vehicles on a time sequence to obtain the spatial positions of the vehicles on the time sequence, and acquiring the motion tracks of all the vehicles.

Step S503: and preprocessing the motion trail of each vehicle.

And preprocessing the motion trail of the vehicle to remove noise and the motion trail which does not meet the preset requirement. For example, the wandering trajectory is not in accordance with the actual situation and is preset to be an unsatisfactory movement trajectory.

Step S504: and analyzing the spatial similarity between every two motion tracks of each vehicle.

The modified Hausdorff distance may be used to measure the spatial distance between the vehicle motion profiles due to the unequal length and shape differences between the target vehicle motion profiles. The spatial similarity degree of the motion trail is mainly expressed by the spatial neighbor relation and the speed relation of the motion trail, so that the spatial Euclidean distance related to the spatial neighbor relation and the cosine distance related to the speed are introduced in the calculation process of the corrected Housdov distance.

For a given trajectory a and trajectory B, the method for analyzing the spatial similarity is shown in fig. 6, and comprises the following steps:

step S5041: and searching the nearest point on the track B corresponding to each observation point on the track A.

The trajectory is considered as a combination of a series of observation points:

wherein

Representing the spatiotemporal position of the ith observation point. For a given trajectory

And track

By aiming at each observation point on the motion trail A

Searching for a point epsilon (i) closest to the point on the motion trail B in the following way:

in the above-mentioned formula (6),

and

a coordinate point representing the ith observation point on the motion trajectory a, and j is the jth point on the motion trajectory B.

Step S5042: a modified housdov distance between trajectory a to trajectory B is calculated.

Searching all observation points on the track A on the track B by using the formula (6)

The corrected hausdorff distance between trajectory a and trajectory B after the closest point epsilon (i) is given by:

in the above formula (7), N_AIs the number of observation points in the trajectory A, beta is the equilibrium coefficient of the space Euclidean and cosine distances in the formula, and v_iAnd v_ε(i)Are respectively

And

the moving speed of (2).

Step S5043: and searching the nearest point on the track A corresponding to each observation point on the track B.

Since the corrected housfow distance d (a, B) between the trajectory a and the trajectory B and the corrected housfow distance d (B, a) between the trajectory B and the trajectory a are asymmetric, it is necessary to calculate d (a, B) and d (B, a) separately when calculating the corrected housfow distance between the trajectory a and the trajectory B.

With the method of step S5041, each observation point on the corresponding trajectory B is searched for the closest point on the trajectory a.

Step S5044: a modified housdov distance between trajectory B to trajectory a is calculated.

Using the method of step S5042, a corrected hausdorff distance d (B, a) between trajectory B to trajectory a is calculated.

Steps S5041 to S5042 and steps S5043 to S5044 may be performed first, or steps S5041 to S5042 may be performed first, or steps S5043 to S5044 may be performed first, or steps S5041 to S5042 and steps S5043 to S5044 may be performed simultaneously.

Step S5045: the spatial distance between trajectory a and trajectory B is calculated.

Taking the minimum of the modified hausdorff distance d (a, B) between the trajectory a and the trajectory B and the modified hausdorff distance d (B, a) between the trajectory B and the trajectory a as the spatial distance between the trajectory a and the trajectory B:

f(A,B)＝min(d(A,B)，d(B,A)) (8)

step S5046: and calculating the spatial similarity between the track A and the track B.

And (3) performing exponential normalization processing on the spatial distance F (A, B) between the track A and the track B according to a formula (1) to obtain the spatial similarity F (A, B) between the track A and the track B.

Step S505: and analyzing the time sequence relation information between every two motion tracks of each vehicle.

Step S504 and step S505 are not in sequence, and any one of them may be performed first, or may be performed simultaneously.

And (3) calculating a time sequence weight influence parameter C between the two tracks according to a formula (3), and further calculating a space-time weight between the two tracks by using a formula (2).

Step S506: and performing similarity modeling of the space-time relationship on the motion tracks of the vehicles to determine the space-time similarity between the motion tracks of the vehicles.

And (5) establishing a motion trajectory space-time similarity model according to the formula (5).

Step S507: and clustering the motion tracks of the vehicles by utilizing the space-time similarity between the motion tracks of the vehicles to obtain a vehicle group with similar time and space.

The data of the movement track of the traffic flow is the result of slowly aggregating and evolving from a small group to a large group, so the embodiment adopts a hierarchical clustering method of low-upward to obtain a clustering group of the movement track of the traffic flow.

The embodiment of the invention can also realize clustering of the motion tracks of the vehicles by other clustering methods, which is not limited herein.

The specific clustering process of hierarchical clustering is shown in fig. 7a and 7b, that is, a single motion trajectory is taken as one class at the beginning of clustering, then corresponding trajectories are gradually fused into the same class according to the spatio-temporal compactness degree between the trajectories, and as the clustering hierarchy increases, smaller clusters are fused into larger clusters.

In order to automatically stop clustering in different scenes according to the conditions of the scenes, namely, the clustering number of hierarchical clustering is automatically determined by utilizing the compactness among motion tracks, the minimum intra-class difference and the maximum inter-class difference are introduced, and the clustering effect is optimal when the inter-class difference is maximum and the intra-class difference is minimum. The optimal cluster number is calculated as follows:

in the above formula (9), S_BAnd S_WRespectively representing the inter-class difference and the intra-class difference of the clustering group in the current hierarchical clustering state.

As the clustering hierarchy increases, the inter-class difference and the intra-class difference become larger, and C is calculated when each hierarchy is clustered_N，C_NMaximum temporal clustering effect is optimal, C_NAnd automatically stopping clustering when the maximum value is reached.

Fig. 7a is a clustering process diagram of low-level clustering of motion trajectories, and fig. 7b is a high-level clustering process diagram:

the first level of hierarchical clustering firstly carries out single motion track t₁、t₂、t₃...

And in the second-level hierarchical clustering, two motion tracks with high space-time similarity are fused into the same type according to the closeness degree between every two motion tracks. Classifying two tracks with the space-time similarity less than or equal to a set threshold value of the space-time similarity of the current level into one class, for example, t, according to the space-time similarity model between every two motion tracks obtained in step S506, i.e. the space-time similarity between the two tracks calculated by formula (5)₁And t₂The space-time similarity between them is less than the threshold value, and is classified as C_1,2，t₃Finding tracks t with temporal similarity within a specified range₃Fall into one category individually.

Third level hierarchical clustering, wherein the clusters obtained by the second level clustering are respectively judged between every two clusters, whether all tracks contained in one cluster and the other clusterThe space-time similarity between every two tracks contained in the group is less than or equal to a set space-time similarity threshold value of the current level, and if yes, the two groups are grouped into one class; if the spatial similarity of any two tracks is greater than the set space-time similarity threshold of the current level, the two groups cannot be grouped into one group. For example, group C_1,2Containing the track t₁、t₂Respectively with t₃The space-time similarity between the two is analyzed, and t is found₁And t₃Time-space similarity of (1), t₂And t₃If the space-time similarity is less than the threshold value of the space-time similarity of the current level, the group C_1,2And t₃Poly as one C_{1,2，3……}。

And by analogy, continuing higher-level hierarchical clustering, and simultaneously respectively calculating the ratio of the inter-class difference and the intra-class difference of the clustering classes in the current hierarchical clustering state during clustering of each hierarchy.

The N-2 level hierarchical clustering respectively obtains a group D according to the method of the third level hierarchical clustering₁、D₂And D₃……。

Level N-1 hierarchical clustering, according to the above method, group D was compared₁All traces and groups D contained in₂All the space-time similarity of every two of the tracks contained in the method is less than or equal to the space-time similarity threshold value of the current level, and then D is determined₁And D₂Poly is a type D_1，2(ii) a Group D₃A group D is not found in which the spatiotemporal similarity between all the included tracks and all the included tracks is less than or equal to the spatiotemporal similarity threshold value of the current level₃Fall into one category individually.

The Nth level hierarchical clustering finds a group D_1，2And group D₃All the space-time similarity between every two included tracks is less than or equal to the space-time similarity threshold value of the current level, and D is obtained_1，2And D₃Clustering into one type to obtain the track path of one type of cluster group₁. And calculating the ratio C of the inter-class difference to the intra-class difference of the cluster group of this hierarchy_NThe maximum, namely the clustering effect is optimal at the moment, the clustering is stopped,and obtaining an optimal clustering group.

In the method, the setting of the spatiotemporal similarity threshold of the current level is gradually reduced along with the increase of the clustering level.

Step S508: and carrying out statistical analysis on the vehicles in each vehicle group to obtain the traffic flow information in the preset area.

The traffic flow movement track clustering group obtained by the method carries out group global analysis on the traffic flow, knows traffic information such as traffic flow density, speed, movement direction, queuing degree and the like, and can effectively monitor violation behaviors and prevent traffic accidents by analyzing and detecting abnormal behaviors and giving an alarm in time.

For the traffic flow analysis task, as the urbanization process is accelerated, the traffic problem is increasingly serious, and the ultra-large-scale traffic flow data cannot be dealt with by a mode of only manually watching videos. In the embodiment, space-time similarity modeling is carried out on the vehicles moving on the road according to the space-time relation of the movement tracks, the movement tracks are clustered by using a hierarchical clustering method based on a space-time similarity model, and the clustering number is automatically determined according to the tightness among the movement tracks in each scene, so that intelligent analysis on mass traffic flow data is realized, the global information of the traffic flow can be better obtained, the specific information of the traffic flow of the monitoring video is obtained in a fast and accurate mode, and abnormal events are detected through analysis. The high efficiency, accuracy and effectiveness of the traffic flow detection are ensured.

Based on the same inventive concept, the embodiment of the invention also provides a device for classifying moving objects in videos, which can be applied to various fields of human behavior patterns, traffic logistics, emergency evacuation management, animal habit analysis, marketing, computer geometry, simulation and the like.

The structure of the device is shown in fig. 8, and comprises:

an obtaining module 801, configured to extract a motion trajectory of each target object in a video;

the modeling module 802 is configured to perform similarity modeling of a space-time relationship on the motion trajectories of the target objects, and determine a space-time similarity between the motion trajectories of the target objects;

the clustering module 803 is configured to cluster the motion trajectories of the target objects by using the spatial-temporal similarity between the motion trajectories of the target objects, so as to obtain a group of target objects that are similar in time and space.

Preferably, the obtaining module 801 is specifically configured to detect each target object from the video sequence, track each target object in a time sequence, obtain a spatial position of each target object in the time sequence, and obtain motion trajectories of all target objects.

Preferably, as shown in fig. 9, the modeling module 802 includes:

the spatial similarity analysis submodule 8021 is configured to analyze spatial similarities between each two of the motion trajectories of the target objects;

the time sequence relation information analysis submodule 8022 is configured to analyze time sequence relation information between each two of the motion tracks of the target objects;

the modeling submodule 8023 is configured to fuse the time-sequence relationship information between each two of the motion trajectories of each target object into the spatial similarity between each two of the motion trajectories, and establish a similarity model of the time-space relationship between each two of the motion trajectories of each target object.

Preferably, the spatial similarity analysis submodule 8021 is specifically configured to: for a given track A and a given track B, calculating a spatial distance f (A, B) between the track A and the track B, and normalizing the spatial distance f (A, B) to obtain a final spatial similarity between the track A and the track B:

F(A,B)＝exp(-f(A,B)/σ) (1)

in the above formula (1), σ is a normalized scale parameter.

Preferably, the timing relationship information analysis sub-module 8022 is specifically configured to:

calculate the timing weight W between given track A and track B:

W＝1/(1+exp(-C)) (2)

and

Preferably, the modeling submodule 8023 is specifically configured to:

Based on the same inventive concept, an embodiment of the present invention further provides an apparatus for analyzing a traffic flow, where the apparatus is shown in fig. 10, and includes:

an obtaining module 1001, configured to extract a motion trajectory of each vehicle object in a video;

the modeling module 1002 is configured to perform similarity modeling of a space-time relationship on the motion trajectories of the vehicles, and determine a space-time similarity between the motion trajectories of the vehicles;

the clustering module 1003 is configured to cluster the motion trajectories of the vehicles by using the spatial-temporal similarity between the motion trajectories of the vehicles to obtain a vehicle group with similar time and space;

the analysis module 1004 is configured to perform statistical analysis on vehicles in each vehicle group to obtain traffic flow information in a preset area.

With regard to the apparatus for classifying moving objects in video and the apparatus for analyzing traffic flow in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated herein.

Based on the same inventive concept, an embodiment of the present invention further provides a non-transitory computer-readable storage medium, where instructions in the storage medium, when executed by a processor, enable the processor to perform the above-mentioned evidence obtaining method for electronic evidence, including:

extracting the motion trail of each target object in the video;

Unless specifically stated otherwise, terms such as processing, computing, calculating, determining, displaying, or the like, may refer to an action and/or process of one or more processing or computing systems or similar devices that manipulates and transforms data represented as physical (e.g., electronic) quantities within the processing system's registers and memories into other data similarly represented as physical quantities within the processing system's memories, registers or other such information storage, transmission or display devices. Information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

It should be understood that the specific order or hierarchy of steps in the processes disclosed is an example of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged without departing from the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, and are not intended to be limited to the specific order or hierarchy presented.

In the foregoing detailed description, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments of the subject matter require more features than are expressly recited in each claim. Rather, as the following claims reflect, invention lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. Of course, the processor and the storage medium may reside as discrete components in a user terminal.

For a software implementation, the techniques described herein may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. The software codes may be stored in memory units and executed by processors. The memory unit may be implemented within the processor or external to the processor, in which case it can be communicatively coupled to the processor via various means as is known in the art.

What has been described above includes examples of one or more embodiments. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the aforementioned embodiments, but one of ordinary skill in the art may recognize that many further combinations and permutations of various embodiments are possible. Accordingly, the embodiments described herein are intended to embrace all such alterations, modifications and variations that fall within the scope of the appended claims. Furthermore, to the extent that the term "includes" is used in either the detailed description or the claims, such term is intended to be inclusive in a manner similar to the term "comprising" as "comprising" is interpreted when employed as a transitional word in a claim. Furthermore, any use of the term "or" in the specification of the claims is intended to mean a "non-exclusive or".

Claims

1. A method for classifying moving objects in a video, comprising:

extracting the motion trail of each target object in the video;

performing exponential operation on the space similarity between every two target objects by utilizing the time sequence relation information of the motion tracks of the target objects according to the following formula (1) to obtain a space-time similarity model between the two motion tracks, and determining the space-time similarity between the motion tracks of the target objects:

in the above formula (1), S, F and w represent the space-time similarity, the space similarity and the time sequence weight between two motion tracks, respectively, and λ is a scale factor;

2. The method according to claim 1, wherein the extracting the motion trajectory of each target object in the video specifically comprises:

3. The method according to claim 1, wherein analyzing spatial similarity between each two of the motion trajectories of the target objects specifically comprises:

F(Α,B)＝exp(-f(Α,B)/σ) (2)

in the above formula (2), σ is a normalized scale parameter.

4. The method according to claim 1, wherein analyzing the time-series relationship information between each two of the motion trajectories of the target objects specifically comprises:

calculate the timing weight w between a given track a and a track B:

w＝1/(1+exp(-C)) (3)

in the above formula (3), the calculation formula of the parameter C is as follows:

in the above formula (4), Δ d is the time-series coincidence ratio between the trajectory A and the trajectory B, η is the ratio of the time-series short trajectory to the time-series long trajectory in the trajectory A and the trajectory B, and η_tThen is the timing length ratio threshold between track a and track B,

and

5. A method for analyzing a traffic flow, comprising:

extracting the motion trail of each vehicle object in the video;

the time sequence relationship information between every two of the motion tracks of each target object is utilized to carry out the spatial similarity between every two of the motion tracks according to the following formula (1)

Performing exponential operation to obtain a space-time similarity model between the two motion tracks, and determining the space-time similarity between the motion tracks of each vehicle:

6. An apparatus for classifying moving objects in a video, comprising:

the modeling module is used for respectively analyzing the space similarity and the time sequence relation information between every two target objects according to the motion tracks of the target objects; utilizing the time sequence relation information of the motion trail of each target object to carry out the spatial similarity between the two objects according to a formula

Performing exponential operation to obtain a space-time similarity model between the two motion tracks, and determining the space-time similarity between the motion tracks of each target object, wherein S, F and w in the formula respectively represent the space-time similarity, the space similarity and the time sequence weight between the two motion tracks, and lambda is a scale factor;

7. The apparatus of claim 6, wherein the modeling module comprises:

8. An apparatus for analyzing a traffic flow, comprising:

Performing exponential operation to obtain a space-time similarity model between the two motion tracks, and determining the space-time similarity between the motion tracks of each vehicle, wherein S, F and w in the formula respectively represent the space-time similarity, the space similarity and the time sequence weight between the two motion tracks, and lambda is a scale factor;