KR102035184B1 - Method and Apparatus for detecting abnormal behavior - Google Patents

Abstract

According to an embodiment of the present invention, an abnormal motion detection apparatus may include a feature point tracking unit for extracting feature points of an object moving in an input image and tracking a positional change over time of the extracted feature points; A topic online learning unit classifying the input image into a document unit which is a bundle of the trajectories, and identifying probability distribution states of topics constituting the classified documents; And a motion pattern online learning unit for inferring a spatiotemporal relationship between identified topics and learning a motion pattern. It includes.

Description

Apparatus and method for detecting abnormal behavior

The present invention relates to an apparatus and method for detecting abnormal motion using an online learning method.

As a method of learning the motion flow of an object in an image, there are two methods of learning based on a trajectory and a local feature. These learning methods can be divided into three types.

First, there is a method of learning an image by tracking the moving objects in the image to find the trajectory, and clustering the trajectories into several major patterns. This method defines the distance between the trajectories so that similar trajectories have a close distance, and uses them to classify the similar trajectories into respective patterns.

Second, there is a method of learning an image by using a Gaussian mixture model and kernel density estimation to define the probability of transition from one pixel to the next. This method statistically learns the velocity and magnitude of the object passing through each position of the image, rather than looking for a typical pattern of trajectories. Compared to the first method, this method has more robust performance for images projected at an arbitrary angle, and can also handle broken trajectories more effectively.

Third, there is an approach to extract low-level local features such as optical flow and use them in learning. This method creates a learning model using a Gaussian mixture model or a probabilistic topic model.

However, most of such learning uses a batch learning scheme, and thus there is a drawback that this change cannot be reflected in the learning model when the situation of the image continuously changes.

KR 2011-0133476

According to an exemplary embodiment of the present invention, even when a plurality of normal movement patterns occur at a time difference in an arbitrary local region of the input image, it is possible to implement the learning.

Another preferred embodiment of the present invention provides an abnormal behavior detection method capable of learning the speed and direction information of a feature point in an image and grasping the spatiotemporal relationship between the patterns.

In addition, a preferred embodiment of the present invention provides a method for detecting abnormal behavior that is adaptable to changes in time according to a change in time, and robust in complex images such as crowded images.

In a preferred embodiment of the present invention, the abnormal motion detection apparatus extracts the feature points of the moving object in the input image, and then tracks the positional changes over time of the extracted feature points to identify the trajectories of the extracted feature points part; A topic online learning unit classifying the input image into document units which are the bundles of the trajectories, and determining a probability distribution state of topics constituting the classified documents by applying an online learning method that is a stochastic topic model; And a motion pattern online learning unit learning a motion pattern by learning a region, a speed, and a direction for each identified topic, and inferring a spatiotemporal relationship between the identified topics. Characterized in that it comprises a.

In a preferred embodiment of the present invention, the abnormal motion detection apparatus extracts the feature points of the moving object in the input image, and then tracks the positional changes over time of the extracted feature points to identify the trajectories of the extracted feature points part; The input image is classified into document units representing the bundles of the trajectories, and a polynomial distribution parameter probability vector value indicating a histogram distribution of topics constituting each document is inferred by an online learning method, which is a stochastic topic model. A trajectory classification unit for clustering the positions of the trajectories for each topic within the space; and a spatiotemporal relationship inference unit for infering the spatiotemporal relationship based on the inferred polynomial distribution parameter probability vector value; and the area, velocity, and direction for each identified topic. And a motion pattern online learning unit learning a motion pattern by inferring a spatiotemporal relationship between the identified topics.

Preferably, the apparatus for detecting abnormal motion uses an Gaussian learning result of the learned motion pattern to determine the abnormal motion pattern when the trajectory included in every frame of the input image has a low probability of being included in the learned motion pattern. It characterized in that it further comprises;

Preferably, the trajectory of the word w _ji A set of words and a set of vector differences v _{ji τ} , wherein the set of words represents a set of words indicating grid point positions through which the respective trajectories _pass through the input image, in which case w _ji is the i th trajectory Passing through a grid, wherein the set of vector differences represents a set of difference v _ji τ of a vector value between the position of the actual feature point in the word and the position of the actual feature point before the τ frame.

Preferably, the polynomial distribution parameter probability vector value indicating the probability distribution state of the plurality of topics constituting the document includes a document-topic probability distribution θ _d and a topic-word probability distribution φ _k . do.

Preferably, it is characterized by inferring the spatiotemporal relationship using the K-means clustering method.

In a preferred embodiment of the present invention, the abnormal motion detection method extracts the feature points of the moving object in the input image from the feature point tracking unit, and then tracks the positional changes of the extracted feature points over time to track the traces of the extracted feature points. Identifying; Classifying the input image into a document unit which is a bundle of the trajectories in a topic online learning unit, and determining a probability distribution state of topics constituting a classified document by applying an online learning method that is a stochastic topic model; And learning area, speed, and direction by the identified topics in the motion pattern online learning unit, and inferring the spatiotemporal relationship between the identified topics to learn the motion patterns.

In a preferred embodiment of the present invention, the abnormal motion detection method extracts the feature points of the moving object in the input image from the feature point tracking unit, and then tracks the positional changes of the extracted feature points over time to track the traces of the extracted feature points. Identifying; The trajectory classification unit classifies the input image into document units representing the bundle of trajectories, and infers a polynomial distribution parameter probability vector value indicating a histogram distribution of topics constituting each document by an online learning method, which is a stochastic topic model. Clustering positions of trajectories for each topic in each document; and inferring a spatiotemporal relationship based on the polynomial distribution parameter probability vector value inferred by the spatiotemporal relationship inference unit; And learning the movement pattern by inferring the spatiotemporal relationship between the identified topics by learning regions, speeds, and directions for each of the identified topics in the movement pattern online learning unit.

The present invention has the effect of learning the main movement of the objects in the image, such as cars and people in a complex situation, such as range, and based on this learning model, it is possible to automatically find the abnormal behavior that occurs in the image. In addition, it is possible to perform the function regardless of the traffic situation, such a function can be applied to automatically detect abnormal behavior in various real-time surveillance (surveillance) situation. When applied to the traffic monitoring system, traffic violations such as unauthorized crossings and illegal U-turns can be detected automatically as abnormal behaviors, and in other monitoring situations, it is possible to detect abnormally low-frequency behaviors as abnormal. It can be effective.

In particular, the abnormal motion detection apparatus and method proposed by the present invention has an advantage that learning can be performed even when a plurality of normal motion patterns occur at a time difference in any local region in the image. And, it is possible to grasp the spatiotemporal relationship between speed, direction information, and each movement pattern. In addition, it is possible to adapt to the change of the image according to the change of time, there is an effect that shows the robust performance even in complex images, such as crowd crowded image.

1 is a preferred embodiment of the present invention, and shows an internal configuration of the abnormal behavior detection apparatus 100.
2A and 2B illustrate an example of classifying an input image into document units and identifying probability distribution states of topics constituting the classified documents using the OLDA learning method.
FIG. 3 is a diagram illustrating an example in which trajectory information is obtained by extracting feature points of a moving object in an input image and tracking a change in position of the extracted feature points over time according to an exemplary embodiment of the present invention.
4 shows an example of a method for showing each locus information as a preferred embodiment of the present invention.
FIG. 5 illustrates an example of identifying a spatiotemporal association using a K-means clustering technique as a preferred embodiment of the present invention.
6 illustrates an example of learning a movement pattern by inferring a spatiotemporal relationship between topics in accordance with a preferred embodiment of the present invention.
7 illustrates a graphic type of an online learning method, which is a stochastic topic model, according to a preferred embodiment of the present invention.
8 (a) to 8 (c) show an example of a stepwise reasoning method as a preferred embodiment of the present invention.
9A and 9B illustrate a preferred embodiment of the present invention, which clusters positions of trajectories by topic.

1 is a preferred embodiment of the present invention, and shows an internal configuration of the abnormal behavior detection apparatus 100.

The abnormal behavior detection apparatus 100 includes a feature point tracking unit 110, a topic online learning unit 120, a movement pattern online learning unit 130, and an abnormal detection unit 140.

For example, when the input image is a traffic situation image, the feature point tracking unit 110 extracts feature points of an object moving in the input image, and tracks the positional changes of the extracted feature points over time to grasp the trajectory information.

According to an exemplary embodiment of the present invention, the feature point tracking unit 110 uses the KLT tracking technique to track the trajectory information in the input image for each main topic (for example, straight trajectory information, left turn trajectory information, right turn trajectory information, and U turn trajectory information). Classify and learn the area, velocity, etc. using the classified trajectories. An embodiment thereof refers to FIG. 3.

The topic online learning unit 120 classifies the trajectory information identified by the feature point tracking unit 110 for each main topic such as a left-right straight trajectory, a left turn trajectory, a U-turn trajectory, and a vertical straight trajectory.

To this end, the topic online learning unit 120 classifies an input image into document units representing a bundle of trajectories, and applies words to construct a classified document by applying an online learning method, which is a stochastic topic model. Identify the probability distribution status of the topics to which they belong. For a specific embodiment of this refer to Figures 2a to 2b.

Thereafter, the movement pattern online learning unit 130 learns the region, the speed, and the direction for each topic identified by the topic online learning unit 120, and infers the spatiotemporal relationship between the identified topics. In a preferred embodiment of the present invention, the K-means clustering technique is used to infer the spatiotemporal relationship. For example, after a left turn, a motion pattern is learned by inferring a spatiotemporal relationship such as a case where U-Turn occurs. An embodiment thereof refers to FIG. 5.

The abnormality detecting unit 140 determines that an abnormal behavior has occurred when the abnormality of the movement pattern is learned from the movement pattern online learning unit 130. In the abnormality detection unit 140, the trajectories included in every frame of the input image are included in the learned motion pattern using the Gaussian learning result of the motion pattern trained by the motion pattern online learning unit 130 (see Equations 10 to 12). If the probability is low, it is distinguished by an abnormal movement pattern.

2A and 2B illustrate a preferred embodiment of the present invention. The topic online learning unit (FIGS. 1 and 120) classifies an input image into document units using an OLDA learning method, and probability distribution of topics constituting the classified documents. An example of grasping a state is shown.

In an exemplary embodiment of the present invention, an online latent dirichlet allocation (LDA) learning method using Variational Bayes (VB) is used as an example of an online learning method that is a stochastic topic model.

OLDA is a stochastic topic model mainly used in the field of natural language processing. It is a technique of classifying several documents by subject and finding out which subject the words in the document belong to. In addition to the field of natural language processing, it is currently applied to various fields of computer image processing.

According to an exemplary embodiment of the present invention, even when the situation of the input image is continuously changed by using the online learning method, there is an advantage that the change can be reflected in the learning model. In an exemplary embodiment of the present invention, the output result may be continuously updated according to the input sequentially input using the online LDA learning method.

In addition, according to an exemplary embodiment of the present invention, even if a plurality of normal motion patterns occur at a certain time in a local area of the input image by using the online learning method, each motion pattern may be learned.

Referring to FIG. 2A, in an exemplary embodiment of the present invention, the input image is classified into document units 201, 202, and 203 representing a bundle of trajectories. In this case, each of the documents d1, d2, d3 ... (201, 202, 203) is composed of a bundle of trajectories of m units (m is a natural number).

2B illustrates an example of identifying probability distribution states of topics belonging to words constituting a classified document by applying an online learning method, which is a stochastic topic model, according to an exemplary embodiment of the present invention.

In one embodiment, the document d1 201 is composed of Tr ₁ , Tr ₂ , Tr ₃ ,..., And Tr _m trajectories, and each trajectory 210 is a set of words indicating the position of each trajectory ( 220).

In detail, each frame of the input image is divided into an n * n size grid, and a grid position at which each trajectory passes is represented by a word. For example, if the j th trajectory crosses the i th grid, it is denoted by W _ji . In this way, the j th trajectory is represented by a set of words {W _ji , i = 1,2, ..., n} 220. After that, we apply the online learning method, which is a probabilistic topic model, to grasp the probability distribution of which topics belong to each word constituting each trajectory.

In this case, it is assumed that a document representing a bundle of trajectories is composed of a plurality of topics generated by a multinomial distribution. In addition, it is assumed that each trajectory constituting the input image belongs to one of latent topics that represent representative K flows in the input image.

Referring to FIG. 2B, the set {W _1i, i = 1, 2, ..., n} 220 of words constituting Tr ₁ 210 indicates a probability distribution such as 230, and topic 1 (T1, It appears that the probability of belonging to the straight trajectory) is high.

Similarly, Tr ₂ (211) also appears to belong to topic 1 (T1, straight trajectory), Tr ₃ (213) is likely to belong to topic 2 (T2, left turn trajectory), Tr _m (213) Appears to belong to topic 3 (T3, right turn trajectory).

According to an exemplary embodiment of the present invention, as shown in the embodiment of FIG. 2B, the input image is classified into document units representing a bundle of trajectories, and the probability distribution states of the topics included in the words are included. Through this process, the positions of the trajectories can be clustered for each topic. For example, the trajectories may be classified by topic for the entire input image. The trajectories belonging to topic 1 (T1, straight trajectory) are Tr ₁ (210), Tr ₂ (211), ..., etc. The trajectories belonging to topic 2 (T2, left turn trajectory) are Tr ₃ (213), ... Etc., the trajectories belonging to topic 3 (T3, right turn trajectory) can be clustered by Tr _m (213), ... Through this process, it is possible to analyze the distribution of topics that occur most frequently in each time zone in the input image and the position of the trajectory for each topic.

FIG. 3 is a diagram illustrating an example in which trajectory information is obtained by extracting feature points of a moving object in an input image and tracking a change in position of the extracted feature points over time according to an exemplary embodiment of the present invention.

Referring to FIG. 3, feature points A, B, C, and D (310, 320, 330, and 340) are extracted from an input image of t1. After that, the position change over time such as t2, t3, ..., tT is tracked.

For example, the location of the feature point A changes with time, such as 310, 311, and 312, the feature point B is 320, 321, and 322, the feature point C is 330, 331, and 332, and the feature point D is 340, 341, and 342. Keep track of position changes.

4 illustrates an example of a method for displaying respective trajectory information as a preferred embodiment of the present invention.

Trajectory Tr (410) = {(x ₁ , y ₁ , 1), (x ₂ , y ₂ , 2), (x ₃ , y ₃ , 3), (x ₄ , y ₄ , 4) ..., (x _T , y _T , T)}

_Referring to FIG. 4, the trajectory Tr ₁ 410 is represented by a set of words w _{ji and} a set of vector differences v _{ji τ} . The word w _ji set represents a set of words indicating a grid point position through which each trajectory passes in the input image. _Referring to one embodiment of FIG. 4, w _ji 420 indicates that the j th trajectory crosses the i th grid, and the j th trajectory is a set of words {W _ji , i = 1,2, ..., N _j }.

The set of vector differences represents a set of difference v _jiτ of the vector value between the position 420 of the actual feature point in the word and the position of the actual feature point before the τ frame.

_Referring to FIG. 4, the vector difference v _ji1 430 between the position 420 of the actual feature point and the position (x ₁ , y ₁ , 1) 421 before the frame ₁ , and the position 420 and 2 frame before the actual feature point Vector difference between positions (x ₂ , y ₂ , 2) 422 v _ji2 440 and vector difference between position 420 of the actual feature point and the position before frame τ (x _T , y _T , T) v _jiτ ( 450). Preferably, the vector difference v _{ji tau} 450 has a Gaussian distribution and is expressed in the form of N (μ, Σ). See Equation 6 below.

In this case, v _ji1 = (Δx _ji1 , Δy _ji1 ), v _ji2 = (Δx _ji2 , Δy _ji2 ),..., V _jiτ = (Δx _jiτ , Δy _jiτ ).

In a preferred embodiment of the present invention, a Gaussian distribution of how far the word is before the τ frame for each topic by using the information v _jiτ information between the position of the actual feature point through which the trajectory passes through the word and the position of the actual feature point before the τ frame. Can be identified.

FIG. 5 illustrates an example of identifying a spatiotemporal association using a K-means clustering technique as a preferred embodiment of the present invention.

In a preferred embodiment of the present invention, it is possible to determine the probability distribution θ _d between a document and a topic using a K-means clustering technique. For details, refer to the content related to FIG. 8B.

6 illustrates an example of learning a movement pattern by inferring a spatiotemporal relationship between topics in accordance with a preferred embodiment of the present invention.

In an exemplary embodiment of the present invention, an online learning model as shown in FIG. 7 is used to learn a region, a velocity, and a direction for each topic as described above, and to learn a movement pattern by inferring a spatiotemporal relationship between identified topics.

7 illustrates a graphic type of an online learning method, which is a stochastic topic model, according to a preferred embodiment of the present invention.

As a preferred embodiment of the present invention, the trajectory determined based on the KLT tracking technique is transformed into an online learning model based on the stochastic topic model shown in FIG.

In a preferred embodiment of the present invention, the online learning method assumes that each trajectory z _j belongs to one of the latent topics representing the representative K flows in the input image. Potential topics refer to major movement patterns such as going straight and turning left in the image.

In addition, a document representing a bundle of trajectories is assumed to be composed of a plurality of topics generated by a multinomial distribution. In this case, the probability distribution of the topics in the document is associated with the parameter probability vector θ _d of the polynomial distribution generated by state s _d . Here, the state s _d means a set of behavioral topics that occur at each time unit. Therefore, by checking the state transition over time, it can be confirmed what actions are mainly shown by time.

In the stochastic topic model, which is a preferred embodiment of the present invention, Equations 1 to 6 are used.

In Equation 1, s _d represents a current state, s _d-1 represents a previous state, and Multi () represents a polynomial distribution. A state holds information about which movement patterns (or topics) were the main ones in order d.

Equation 1 means that what state (s _d ) appears in the current order d is affected only by what state (s _d-1 ) in the previous order d-1.

In Equation 2, s _d represents a current state, θ _d represents a parameter probability vector of a polynomial distribution, Dir () represents a Dirichlet distribution, and α represents a Hyper-parameter of the Dirichlet distribution.

Equation 2 means that θ _d appears in the form of a probability indicating a certain distribution, assuming that the overall model is bayesian.

In Equation 3, φ _k represents a word-topic distribtion, Dir () represents a Dirichlet distribution, and β represents a Hyper-parameter of the Dirichlet distribution.

Equation 4 assumes that the variable z for determining which topic index each word has is derived by θ _d , which is a probability distribution indicating which topic the document belongs to.

Equation 5 indicates that each word is generated in a topic distribution, and in which topic distribution each word is generated is determined by topic index z.

(R은 R차원을 표시함)를 파라미터로 하는 다항분포를 따른다.In Equation 5, the word set W _ji of the j th trajectory is

Follow the polynomial distribution with R as the parameter.

In Equation 6, the vector difference v _jiτ belonging to each word has a Gaussian distribution and is expressed in the form of N (μ, Σ). In addition, the vector car v _jiτ is You can display the speed.

However, when the trajectories extracted through the KLT tracking method are transformed into a form that can be used in the online learning method by using Equations 1 to 6, it is impossible to infer hidden variables using the modified input. Do. This is because the online learning method based on the stochastic topic model shown in FIG. 6 cannot infer a hidden variable because the integral is not defined.

In an exemplary embodiment of the present invention, an approximation method such as a sampling method is additionally applied to the online learning method based on the stochastic topic model shown in FIG. 6 to solve this problem.

In a preferred embodiment of the present invention, as an example of the approximation method, a cascade inference method is applied. 7 (a) to 7 (c) show an example of a stepwise reasoning method as a preferred embodiment of the present invention.

In an embodiment of the present invention, a three-step inference method is applied, and the hidden variables inferred in each step are inferred in each step by using the hidden variables inferred in the next step.

The three-step estimating method is as follows.

1. Clustering locations of trajectories by topic within a document

2. Identify the spatiotemporal association of topics

3. Modeling the unquantized motion pattern in Gaussian form using the information of clustered trajectories as observations

Clustering the positions of the trajectories for each topic in the document to infer the value of the polynomial distribution parameter probability vector (θ _d, φ _k ) (FIG. 8A, 820, 821) indicating the histogram distribution of the topic in the document, and inferring Using the observed values of θ _{d and} φ _k as observations, we identify the spatiotemporal relationships among the topics. Through the process of identifying the temporal and temporal association of the topics, the general movement behavior pattern is identified.

Subsequently, learning is performed by displaying a non-quantized behavior pattern in the form of Gaussian using the topic information value z _dji (FIGS. 8A and 830) to which each word belongs as an observation value. Using the trained Gaussian learning results, the probability that an incoming trajectory exists in each frame can be determined to determine the trajectories with low probability as abnormal.

Looking at each step in detail as follows.

1. Clustering locations of trajectories by topic within a document

First, as shown in FIG. 8A, a probability of association between a document-topic distribution θ _d 820, a topic z _dji 830, and a word w _dji 840 is determined. The probability distribution of which topic z _dji 730 each of the words N _j words w _dji 740 included in the M trajectories in the document 800 belongs to is approximated. Subsequently, stepwise inference may be applied to infer the value of the document-topic probability distribution (θ _d ) 820 and the topic-word probability distribution φ _k 821. Through this process, the positions of the trajectories can be clustered for each topic. A preferred embodiment of clustering positions of trajectories for each topic is described with reference to FIGS. 9A and 9B.

Referring to FIG. 8A, given the Dirichlet parameters α, β (810, 811) via Equations 2 to 3, the document-topic distribution (θ _d ) 820, topic (z _dji ) 830 and words The coupling probability between (w _dji ) 840 is expressed by Equation 7.

The document consists of a number of topics created by a multinomial distribution, and the probability distribution of the topics in the document is associated with a parameter probability vector θ _d of the polynomial distribution created by each state s _d . Here, the polynomial distribution parameter probability vectors θ _{d and} φ 820 and 821 indicate histogram distributions of the topics in the document.

In Equation 7, φ denotes a topic-word probability distribution 821, a document-topic distribution probability distribution θ _d (820) denotes a distribution of a topic in a document, z _dji (830) denotes a topic, and w _dji (840) Marks each word. In this case, the subscript d is an identifier for identifying the document, indicating j∈ {1,2, ..., M} (M is the number of trajectories in the document) and the index of each trajectory included in the document. And i∈ {1,2, ..., N _dj } (where N _dj is the number of words in the j th trajectory) indicates an index of words in the trajectory.

In Equation 7, inference is made through an online LDA learning method using Variational Bayes (VB). In this process, the Variational distribution of latent variables is set as shown in Equation 8. A latent variable is any variable that cannot actually be observed in a probability model.

는 수학식 7을 근사화 하기 위해 이용되는 Variational parameter를 나타낸다. 수학식 8은 종래에는 서로 조건적으로 의존적이던(conditionally dependent) 변수를 각각 독립적이라고 가정한다.

Variational parameter 값들은 입력으로 들어오는 궤적을 변환한 정보들의 로그 라이클리우드(log-likelihood) 값의 하한에 접근하는 방법으로 추론된다. 이러한 추론 과정을 통해 각 궤적 내의 단어들이 어떠한 토픽을 가지고 있는지를 알 수 있다. 추론 과정은 수학식 8이 수학식 7에 얼마나 유사하게 접근하는지를

값들을 변화시켜가며 찾아가는 과정으로 이루어져 있다. In equation (8)

Denotes a Variational parameter used to approximate Equation 7 . Equation 8 conventionally assumes that each of the variables conditionally dependent on each other is independent.

Variational parameter values are inferred by approaching the lower limit of the log-likelihood value of the information that transforms the trajectory coming into the input. Through this reasoning process, we can know what topics the words in each trajectory have. The inference process determines how similarly Equation 8 approaches Equation 7.

It consists of a process of changing values.

After that, in the preferred embodiment of the present invention, a histogram between the word (w _dji ) 840 and the topic (z _dji ) 830 is created for every M trajectories in the document 800, and then the mode is taken to take the mode. Clustering is done by topic definition. In this process, errors can be minimized by using the mode.

9A and 9B illustrate a preferred embodiment of the present invention, which clusters positions of trajectories by topic.

Referring to FIG. 9A, a probability distribution of a topic to which each word belongs is determined for each trajectory Tr1, Tr2, Tr3, ... (910, 920, 930) (911, 921, and 931). After that, the topic most likely to occur most likely is determined for each trajectory. For example, the topic 1 (T1) in the Tr1 trajectory 910, the topic 1 (T1) in the Tr2 trajectory 920, and the topic 2 (T2) in the Tr3 trajectory 930 are the most likely topics. In this way, we examine the topic of the entire trajectory of the input image.

Thereafter, as shown in FIG. 9B, the trajectories are clustered for each topic.

For example, the trajectories with the highest probability of occurrence of topic 1 (T1) are Tr1, Tr2, Tr10, Tr11, Tr 12, Tr 17, Tr18, and the trajectories with the highest probability of occurrence of topic 2 (T2) are Tr3, Tr5, Tr6, Tr21 and T3 are the most likely to occur in Tr4, T4 is the most likely to occur in Tr7, Tr8 and Tk are the most likely to occur in Tr9. do.

2. Identify the spatiotemporal association of topics

In the preferred embodiment of the present invention, the basic concept of identifying the spatiotemporal association of topics is described with reference to FIGS. 5 to 6.

8B shows a conceptual diagram of a model for identifying the spatiotemporal association of topics. The probability distribution shown in FIG. 8B is associated with the polynomial distribution parameter θ _d generated by state s _d . In this case, each state s _d represents a set of behavioral topics that occur each time. Therefore, by checking the transition of the state over time, it is possible to confirm which actions are mainly shown each time.

In this step, the transition state between states is inversely checked by using the θ _d value obtained in the step of clustering the positions of the trajectories for each topic in the document, thereby identifying the spatiotemporal relationship between the topics.

In this process, in the preferred embodiment of the present invention, a minimum state group is selected to determine the spatiotemporal relationship using the K-meas clustering method.

In the step of identifying the spatiotemporal association of topics, θ _d represents a K-dimensional vector. In other words, we assume a vector of K-dimensionality, considering each probability that a given K topic appears as an element.

이 주어졌을 때 이를 Nc개의 그룹 {C₁, C₂, ...,C_NC}로 나누었다. 이 때 Nc는 디자인 파라미터(Design Paramter)로 각각 상태(State)의 가짓수를 의미한다. 시공간적 연관관계를 파악하기 위하여 선택하는 그룹의 수 Nc는 수학식 9를 최소로 하는 값으로 설정되며, 일반적으로 2~3 사이의 수가 선택된다.In a preferred embodiment of the present invention, D histograms

Is given, it is divided into Nc groups {C ₁ , C ₂ , ..., C _NC }. In this case, Nc is a design parameter and indicates the number of states of each state. The number of groups Nc selected to grasp the spatiotemporal relationship is set to a value that minimizes Equation 9, and a number between 2 and 3 is generally selected.

안의 벡터의 평균을

으로 정의하며

는 토픽 개수를 나타내는 K차원별로

를 노말라이즈한 결과를 나타낸다. 본 발명의 바람직한 일 실시예에서는 수학식 9를 최소화 하기 위하여 K Means- clustering을 수행하여 {s_d}와

를 구하였다. 이 경우 K 값은 Nc 값이 된다. {s_d}는 해당하는

의 클러스터링 인덱스를 나타낸다. 이 경우 s_d 에서는 어떠한 토픽 주요하게 나오는지에 관한 정보를 지니고 있다. Set from Equation 9

The average of the vectors inside

Is defined as

Is for each K dimension that represents the number of topics

Shows the result of normalizing. In a preferred embodiment of the present invention by performing the K Means-clustering to minimize the equation (9) and {s _d } and

Was obtained. In this case, the K value becomes the Nc value. {s _d } is equivalent to

Represents the clustering index of. In this case, s _d contains information about which topic is the main topic.

As an example, when Nc is 3, it is displayed as shown in FIG. 8B, and in FIG. 8B, s _d-1 850 is a previous state, s _d 860 is a current state, and s _{d + 1} 870 is immediately after. Each state is represented and implemented in a form similar to the Hidden Markov model.

In addition, each state (e.g., s _d ) is similar to the Hide Marcode model, and the state (e.g., s _d-1 , s _{d + 1} ).

The Hidden Markov model assumes that observation occurs in the state of the markov chain, where the states that make up the markov chain are not actually observed.

은 가로로 움직이는 자동차들의 움직임(

)이나 세로로 움직이는 자동차들의 움직임(

)과 같은, 영상 내 행동들의 시공간적인 연관관계를 기준으로 묶은 일반적인 움직임패턴들을 의미한다. 상태 인덱스의 집합

는 시간이 지남에 따라 발생하는 영상 내 상태 변화를 표현한다. As a preferred embodiment of the present invention, the parameters defined as above

Is the movement of cars moving horizontally

) Or vertically moving cars

) Refers to general movement patterns grouped based on the spatiotemporal relationship of actions in the image. Set of state indices

Represents the state change in the image that occurs over time.

As a preferred embodiment of the present invention, the online learning method can adapt to the change of the image by using the size of the sliding window designated in order to use the progressive learning method. The size of the sliding window is related to the mini batch size of the Online LDA.

3. Modeling the unquantized motion pattern in Gaussian form using the information of clustered trajectories as observations

In this step, each topic has a result of modeling a movement pattern. For each grid that divides the image, the location of the trajectory passing through that location before t1..t _T time is stored in the form of a single Gaussian model.

FIG. 8C illustrates a conceptual diagram for modeling a non-quantized movement pattern in Gaussian form using the clustered trajectory information Z _j and 880 as observation values. In a preferred embodiment of the present invention, the behavior pattern of each cell is identified on the assumption that the pixels of adjacent positions have similar behavior patterns.

Equation 10 represents an incremental update of the Gaussian model and updates the mean and variance when a new input value is input.

In Equation 10, ρ means a learning rate. In order to learn the variance, the mean value of squares is updated as shown in Equation (11).

Using the result of Equation 11, a variance matrix can be obtained as shown in Equation 12.

Equation 12 is an equation for incrementally updating Sigma.

In the preferred embodiment of the present invention, since the behaviors and associations that appear for each topic (movement pattern) are modeled in the form of a probability distribution using the above equations, the behaviors having a high probability value due to the probability distribution among the input actions are normal, Divide bad behavior into abnormalities.

So far I looked at the center of the preferred embodiment for the present invention. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

Claims (23)

A feature point tracking unit for extracting feature points of a moving object in an input image and identifying a trajectory of the extracted feature points by tracking a change in position of the extracted feature points over time;
A topic online learning unit classifying the input image into document units which are the bundles of the trajectories, and determining a probability distribution state of topics constituting the classified documents by applying an online learning method that is a stochastic topic model; And
A motion pattern online learning unit learning a motion pattern by learning a region, a speed, and a direction for each identified topic, and inferring a spatiotemporal relationship between the identified topics; Including;
The trajectory is represented by a set of words and a set of vector differences, and the set of words indicates a set of words indicating grid point positions through which the respective trajectories pass in the input image. The method of claim 1,
And an abnormality detection unit for distinguishing a trajectory included in each frame of the input image into an abnormal motion pattern by using the Gaussian learning result of the learned motion pattern when the probability of including the trajectory included in each frame of the input image is low. Abnormal motion detection device. Claim 3 has been abandoned upon payment of a set-up fee. The method of claim 1, wherein the feature point tracking unit
Abnormal motion detection device characterized in that to extract the feature points of the moving object in the input image by the KLT tracking method. According to claim 1, wherein in the topic online learning unit
The document is composed of a plurality of topics generated by a multinomial distribution, and infers a polynomial distribution parameter probability vector value indicating a probability distribution state of a plurality of topics constituting the document. Abnormal motion detection device characterized in that for clustering. The method of claim 1,
The polynomial distribution parameter probability vector value indicating a probability distribution state of a plurality of topics constituting the document includes a document-topic probability distribution θ _d and a topic-word probability distribution φ _k . Detection device. The method of claim 1, wherein the trajectory is
Of the word w _ji A set of words and a set of vector differences v _{ji τ} , wherein the set of words represents a set of words indicating grid point positions through which the respective trajectories _pass through the input image, in which case w _ji is the i th trajectory And passing through a grid, wherein the set of vector differences represents a set of difference v _jiτ between a vector value position between the position of the actual feature point in the word and the position of the actual feature point before the τ frame. Claim 7 was abandoned upon payment of a set-up fee. The method of claim 6,
The word w _ji follows a polynomial distribution, and the vector difference v _jiτ belonging to each word has a Gaussian distribution. Claim 8 has been abandoned upon payment of a set-up fee. The method of claim 6,
Abnormal motion detection device characterized in that the speed is determined for each word position by using the position value of the current frame of the particular grid and the actual position value before the τ frame of the particular grid. Claim 9 was abandoned upon payment of a set-up fee. The method of claim 1,
The online learning method is an abnormal motion detection device, characterized in that the Online LDA (Latent dirichlet distribution) learning method. Claim 10 has been abandoned upon payment of a setup registration fee. The method of claim 9,
The online LDA learning method is an abnormal motion detection device, characterized in that to infer the LDA using Variational Bayes (VB). Claim 11 was abandoned upon payment of a set-up fee. The method of claim 1, wherein the movement pattern online learning unit
Abnormal motion detection apparatus characterized by inferring the spatiotemporal relationship using the K-means clustering method. A feature point tracking unit for extracting feature points of a moving object in an input image and identifying a trajectory of the extracted feature points by tracking a change in position of the extracted feature points over time;
The input image is classified into document units representing the bundles of the trajectories, and a polynomial distribution parameter probability vector value indicating a histogram distribution of topics constituting each document is inferred by an online learning method, which is a stochastic topic model. Trajectory classification unit for clustering the position of the trajectories for each topic in the;
A spatiotemporal relationship inference unit for inferring a spatiotemporal relationship based on the inferred polynomial distribution parameter probability vector value; and
A motion pattern online learning unit learning a motion pattern by learning a region, a speed, and a direction for each identified topic, and inferring a spatiotemporal relationship between the identified topics;
The trajectory is represented by a set of words and a set of vector differences, and the set of words indicates a set of words indicating grid point positions through which the respective trajectories pass in the input image. The method of claim 12,
And an abnormality detection unit for distinguishing a trajectory included in each frame of the input image into an abnormal motion pattern by using the Gaussian learning result of the learned motion pattern when the probability of including the trajectory included in each frame of the input image is low. Abnormal motion detection device. Claim 14 was abandoned upon payment of a set-up fee. The method of claim 12, wherein the polynomial distribution parameter probability vector value is
An abnormal motion detection apparatus comprising a document-topic probability distribution (θ _d ) and a topic-word probability distribution (φ _k ). Claim 15 was abandoned upon payment of a set-up fee. The method of claim 12, wherein the space-time relationship inference unit
Abnormal motion detection apparatus characterized by inferring the spatiotemporal relationship using the K-means clustering method. The method of claim 12, wherein the trajectory is
Of the word w _ji A set of words and a set of vector differences v _{ji τ} , wherein the set of words represents a set of words indicating grid point positions through which the respective trajectories _pass through the input image, in which case w _ji is the i th trajectory And passing through a grid, wherein the set of vector differences represents a set of difference v _jiτ between a vector value position between the position of the actual feature point in the word and the position of the actual feature point before the τ frame. Claim 17 was abandoned upon payment of a set-up fee. The method of claim 16,
The word w _ji follows a polynomial distribution, and the vector difference v _jiτ belonging to each word has a Gaussian distribution. Claim 18 was abandoned when the set registration fee was paid. The method of claim 16,
Abnormal motion detection device characterized in that the speed is determined for each word position by using the position value of the current frame of the particular grid and the actual position value before the τ frame of the particular grid. Claim 19 was abandoned upon payment of a set-up fee. The method of claim 12, wherein the feature point tracking unit
Abnormal motion detection device characterized in that to extract the feature points of the moving object in the input image by the KLT tracking method. Claim 20 was abandoned when the set registration fee was paid. 13. The apparatus of claim 12, wherein the online learning method is an online latent distribution distribution (LDA) learning method. Extracting a feature point of an object moving in the input image by a feature point tracker, and tracking a positional change of the extracted feature points over time to identify a trajectory of the extracted feature points;
Classifying the input image into a document unit which is a bundle of the trajectories in a topic online learning unit, and determining a probability distribution state of topics constituting a classified document by applying an online learning method that is a stochastic topic model; And
Learning a motion pattern by learning a region, a speed, and a direction for each identified topic in the motion pattern online learning unit, and inferring a spatiotemporal relationship between the identified topics;
The trajectory is represented by a set of words and a set of vector differences, and the set of words indicates a set of words indicating a grid point position through which the respective trajectories pass in the input image. Claim 22 was abandoned upon payment of a set-up fee. 22. The method of claim 21, wherein the abnormal detection unit uses the Gaussian learning result of the learned motion pattern to distinguish the abnormal motion pattern when the trajectory included in each frame of the input image has a low probability of being included in the learned motion pattern. Abnormal motion detection method characterized in that it further comprises. Extracting a feature point of an object moving in the input image by a feature point tracker, and tracking a positional change of the extracted feature points over time to identify a trajectory of the extracted feature points;
The trajectory classification unit classifies the input image into document units representing the bundle of trajectories, and infers a polynomial distribution parameter probability vector value indicating a histogram distribution of topics constituting each document by an online learning method, which is a stochastic topic model. Clustering positions of trajectories for each topic within each document; and
Inferring a spatiotemporal relationship based on the polynomial distribution parameter probability vector value inferred by the spatiotemporal relationship inference unit;
Learning a motion pattern by learning a region, a speed, and a direction for each of the identified topics in the motion pattern online learning unit, and inferring the spatiotemporal relationship between the identified topics;
The trajectory is represented by a set of words and a set of vector differences, and the set of words indicates a set of words indicating a grid point position through which the respective trajectories pass in the input image.

Images