CN111950363B - Video anomaly detection method based on open data filtering and domain adaptation - Google Patents
Video anomaly detection method based on open data filtering and domain adaptation Download PDFInfo
- Publication number
- CN111950363B CN111950363B CN202010646631.6A CN202010646631A CN111950363B CN 111950363 B CN111950363 B CN 111950363B CN 202010646631 A CN202010646631 A CN 202010646631A CN 111950363 B CN111950363 B CN 111950363B
- Authority
- CN
- China
- Prior art keywords
- data
- open
- visible
- training
- test
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Probability & Statistics with Applications (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of application of computer vision, in particular to a video anomaly detection method based on open data filtering and domain adaptation. The method can definitely detect the open normal event and the open abnormal event in the data set, and uses different classifiers to process the open data and the visible data, thereby reducing the influence of the open data on the abnormal detection and obviously improving the abnormal detection performance. First, the method directly trains a classifier for visible data in test data using a training video. Then, in the testing phase, the testing data is divided into visible data and open data by using the open data filtering module. Finally, for the visible data, directly utilizing a visible data classifier to generate an abnormal score for the visible data; for open test data, a domain adaptation method is used to reduce the distribution difference between the open test data and the training data, and then a classifier is retrained for generating the abnormal score.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a video anomaly detection method based on open data filtering and domain adaptation.
Background
The anomaly detection is a key technology for realizing intelligent video monitoring, and can automatically detect abnormal events in videos. The existing video anomaly detection method can be divided into three categories according to a supervision mode: semi-supervised, unsupervised and supervised. The semi-supervised anomaly detection method only uses normal data to train a detection model, and determines an event which does not accord with the characteristics of the normal data as an abnormal event during testing; the unsupervised anomaly detection method detects anomalies by estimating the frame's resolvability only with reference to the video context, without using known normal data. If a frame can be easily distinguished from other frames in the same video, it is marked as an abnormal frame; the supervised anomaly detection method uses a large amount of normal data and a small amount of abnormal data to train an anomaly detection model, and treats anomaly detection as an unbalanced two-classification problem.
Existing methods typically use the difference between an exceptional event and a normal event to measure whether a frame in a video contains an exceptional event. However, due to the diversity and openness of events in video, it is not realistic to obtain all possible normal events, and there may be also a large difference between the types of normal events not seen in the test set and the known normal events in the training set. Then the unseen normal events will not satisfy the characteristics of the known normal events in the training set, and there is a high possibility that they are erroneously determined as abnormal events, resulting in a high false positive rate.
Disclosure of Invention
In order to solve the technical problems, the invention provides a supervised video anomaly detection method based on open data filtering and domain adaptation to measure whether anomalies exist in a video frame or not and improve the video anomaly detection performance.
The invention discloses a video anomaly detection method based on open data filtering and domain adaptation, which comprises the following steps:
extracting appearance characteristics of all frames of a training video and a testing video by using a vgg-f model trained on ImageNet, and training a classifier for visible data by using the characteristics of the training video;
designing an open data filter, wherein the module is used for dividing the test data into visible data and open data; the filter firstly divides known training data into a plurality of classes by a k-means clustering method, then takes the minimum distance between the testing data and the class center of the known training data as the open probability of the testing data, and finally determines the optimal filtering proportion by a U-test method after obtaining the open probability of all frames of a testing video to obtain the final visible data and open data;
meanwhile, a Joint Distribution Adaptation (JDA) method is introduced for reducing the distribution difference between the open data and the training data and obtaining the new feature representation of the open data and the training data; training a classifier for the open data using the new training data feature representation;
and generating an abnormal score of the visible data in the test set by using a visible data classifier, generating an abnormal score of the open data in the test set by using an open data classifier, and synthesizing the abnormal scores to obtain a detection result of the whole test video.
The beneficial effects of the invention are as follows: when the video is subjected to the anomaly detection, the difference between the abnormal event and the known normal event is considered, and the difference between the open normal event and the known normal event is also considered, so that the problem that the unseen normal event can be mistakenly judged as the abnormal event can be solved, and the anomaly detection performance is improved; the test data are effectively divided into visible data and open data by the open data filter, and the abnormal scores of the two data are obtained by adopting different classifiers, so that the influence of open events in a test set on the abnormal detection performance is effectively reduced.
Drawings
FIG. 1 is a schematic of the present invention.
Detailed Description
The following detailed description of the present invention is provided in connection with the accompanying drawings and examples. The following examples are intended to illustrate the present invention, but are not intended to limit the scope of the present invention.
Constructing a training set and a testing set: selecting abnormal frames in a certain proportion from the test set and adding the abnormal frames into the training set, thereby obtaining a training set containing normal events and abnormal events, wherein the test set also contains open events in a certain proportion;
in the training stage, the pre-training neural network is used to extract the features of the training video, and the visible data classifier is trained by using the features.
In the testing stage, after the features of the test video are obtained by utilizing the pre-training neural network, the features of the test video are divided into known data and open data by the open data filter. The filter firstly divides known training data into a plurality of classes through a k-means clustering method, and then takes the minimum distance between the test data and the class center of the known training data as the open probability of the test data:
wherein
J frame, X, representing the ith video train Representing the entire training video, C n Representing the nth class center of the training data.
After the open probabilities of all the frames of the test video are obtained, determining the optimal filtering proportion by a U-test method:
wherein P is nor (r) represents a P value between a mean of visible data in the test data and a mean of normal data in the training data at a filtering ratio r; p abnor (r) represents the value of P between the mean of the visible data in the test data and the mean of the outlier data in the training data at the filter ratio r. The larger the sum of the two P values is, the smaller the difference between the visible data and the training data obtained after filtering the open data in the test data is, so that the optimal ratio of the visible data and the open data can be obtained.
Since the sigmoid function can smoothly map real numbers to [0,1], the function value can be used as a probability of testing for frame anomalies. For the visible data, directly taking the probability that the test frame output by the visible data classifier belongs to the abnormal class as an abnormal score:
wherein g is s Representing a visual data classifier, theta s A parameter representing a classifier of the visible data,
representing number of testsThe j frame of the i video from which the data is visible.
For the open data, firstly, a joint distribution adaptation method is adopted to reduce the distribution difference between the open data and the training data, and new feature representations of the open data and the training data are obtained:
X train_new ＝4 T X s
X open_new ＝4 T X t
where 4 denotes an optimal transformation matrix obtained by a joint distribution adaptation method, X s Is a feature representation of the source training data, X t Is a characteristic representation of open data.
The open data classifier is then trained with the feature representation of the new training data and used to generate an anomaly score for the open data:
wherein g is n Represents an open data classifier, θ n A parameter representing an open data classifier is used,
is a new feature representation of the ith video frame j.
And finally, combining the abnormal scores of the visible data and the open data to obtain the abnormal detection result of the whole test video.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, it is possible to make various improvements and modifications without departing from the technical principle of the present invention, and those improvements and modifications should be also considered as the protection scope of the present invention.
Claims (5)
1. A video anomaly detection method based on open data filtering and domain adaptation is characterized by comprising the following steps:
(1) Extracting data features: extracting appearance characteristics of training data and test data by using a pre-trained neural network, and training a visible data classifier by using the characteristics of the training data;
(2) Open data filtering: analyzing the characteristics of the test data by using an open data filter, and dividing the characteristics into visible data and open data;
the filter firstly divides known training data into a plurality of classes through a k-means clustering method, and then takes the minimum distance between the test data and the class center of the known training data as the open probability of the test data:
wherein
J frame, X, representing the ith video train Representing the entire training video, C n An nth class center representing training data;
after obtaining the open probability of all the frames of the test video, determining the optimal filtering proportion by a U-test method:
wherein P is nor (r) represents a P value between a mean of visible data in the test data and a mean of normal data in the training data at a filtering ratio r; p abnor (r) represents the P value between the mean of the visible data in the test data and the mean of the anomalous data in the training data at a filter ratio r; the larger the sum of the two P values is, the smaller the difference between the visible data and the training data obtained after filtering the open data in the test data is, so that the visible data and the open data with the optimal proportion can be obtained;
(3) Open data field adaptation: using Joint Distribution Adaptation (JDA) to reduce the distribution difference between the open data and the training data and obtain new feature representations of the open data and the training data:
X train_new ＝A T X s
X open_new ＝A T X t
wherein A represents an optimal transformation matrix obtained by a joint distribution adaptation method, X s Is a feature representation of the source training data, X t Is a characteristic representation of open data;
then, the open data classifier is trained by using the feature representation of the new training data, and the classifier is used for generating an abnormal score of the open data:
wherein g is n Represents an open data classifier, θ n A parameter representing an open data classifier is used,
is a new feature representation of the ith video frame j;
(4) And (3) abnormal score: for the visible data in the test set, obtaining the probability of the visible data belonging to the abnormality by using a visible data classifier, and taking the probability as the abnormality score of the visible data; and for the open data in the test set, obtaining the probability of the open data belonging to the anomaly by using an open data classifier, and taking the probability as the anomaly score of the visible data.
2. The method for detecting video abnormality based on open data filtering and domain adaptation of claim 1, wherein the pre-trained neural network is a vgg-f model trained on ImageNet.
3. The method for detecting video anomaly based on open data filtering and domain adaptation according to claim 1, wherein the visible data classifier and the open data classifier are both focal loss based unbalanced classifiers.
4. The method for detecting video anomaly based on open data filtering and domain adaptation as claimed in claim 1, wherein said open data filter is implemented based on k-means method and U-test method.
5. The method for detecting video anomaly based on open data filtering and domain adaptation as claimed in claim 1, wherein said data is a frame of a video.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010646631.6A CN111950363B (en) | 2020-07-07 | 2020-07-07 | Video anomaly detection method based on open data filtering and domain adaptation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010646631.6A CN111950363B (en) | 2020-07-07 | 2020-07-07 | Video anomaly detection method based on open data filtering and domain adaptation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111950363A CN111950363A (en) | 2020-11-17 |
| CN111950363B true CN111950363B (en) | 2022-11-29 |
Family
ID=73340363
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010646631.6A Active CN111950363B (en) | 2020-07-07 | 2020-07-07 | Video anomaly detection method based on open data filtering and domain adaptation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111950363B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107133654A (en) * | 2017-05-25 | 2017-09-05 | 大连理工大学 | A kind of method of monitor video accident detection |
| CN111079539A (en) * | 2019-11-19 | 2020-04-28 | 华南理工大学 | Video abnormal behavior detection method based on abnormal tracking |
| WO2020135392A1 (en) * | 2018-12-24 | 2020-07-02 | 杭州海康威视数字技术股份有限公司 | Method and device for detecting abnormal behavior |
-
2020
- 2020-07-07 CN CN202010646631.6A patent/CN111950363B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107133654A (en) * | 2017-05-25 | 2017-09-05 | 大连理工大学 | A kind of method of monitor video accident detection |
| WO2020135392A1 (en) * | 2018-12-24 | 2020-07-02 | 杭州海康威视数字技术股份有限公司 | Method and device for detecting abnormal behavior |
| CN111079539A (en) * | 2019-11-19 | 2020-04-28 | 华南理工大学 | Video abnormal behavior detection method based on abnormal tracking |
Non-Patent Citations (3)
| Title |
|---|
| 《融合视觉感知特性的视频编码率失真优化》;王茜 等;《计算机辅助设计与图形学学报》;20151015;第27卷(第10期);全文 * |
| Spatiotemporal Representation Learning for Video Anomaly Detection;Zhaoyan Li;《IEEE Access》;20200130;第08卷;全文 * |
| 基于卷积神经网络的时空融合的无参考视频质量评价方法;王春峰 等;《中国科学院大学学报》;20180715;第35卷(第04期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111950363A (en) | 2020-11-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113554089B (en) | Image classification countermeasure sample defense method and system and data processing terminal | |
| CN110132598B (en) | Fault noise diagnosis algorithm for rolling bearing of rotating equipment | |
| CN111680614B (en) | Abnormal behavior detection method based on video monitoring | |
| CN112087442B (en) | Time sequence related network intrusion detection method based on attention mechanism | |
| CN107368854A (en) | A kind of circuit breaker failure diagnostic method based on improvement evidence theory | |
| CN111783534B (en) | Sleep stage method based on deep learning | |
| CN111507371A (en) | Method and apparatus | |
| CN109543760A (en) | Confrontation sample testing method based on image filters algorithm | |
| CN111652290A (en) | Detection method and device for confrontation sample | |
| CN111414969B (en) | Smoke detection method in foggy environment | |
| CN116192500A (en) | Malicious flow detection device and method for resisting tag noise | |
| CN111563468B (en) | Driver abnormal behavior detection method based on attention of neural network | |
| CN111310719B (en) | Unknown radiation source individual identification and detection method | |
| CN109117810A (en) | Fatigue driving behavioral value method, apparatus, computer equipment and storage medium | |
| CN115758083A (en) | Motor bearing fault diagnosis method based on time domain and time-frequency domain fusion | |
| CN113076980B (en) | Method for detecting images outside distribution based on attention enhancement and input disturbance | |
| CN111431937A (en) | Method and system for detecting abnormal flow of industrial network | |
| CN111950363B (en) | Video anomaly detection method based on open data filtering and domain adaptation | |
| CN113343123A (en) | Training method and detection method for generating confrontation multiple relation graph network | |
| CN113343926A (en) | Driver fatigue detection method based on convolutional neural network | |
| CN117118718A (en) | Intrusion detection method and system based on multi-generator GAN data enhancement | |
| CN117596011A (en) | Power grid flow anomaly detection method and system based on countermeasure convolutional neural network | |
| CN116385935A (en) | Abnormal event detection algorithm based on unsupervised domain self-adaption | |
| CN116070137A (en) | Open set identification device and method for malicious traffic detection | |
| CN112668655A (en) | Method for detecting out-of-distribution image based on generation of confrontation network uncertainty attention enhancement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |