CN109614893B - Intelligent abnormal behavior track identification method and device based on situation reasoning - Google Patents
Intelligent abnormal behavior track identification method and device based on situation reasoning Download PDFInfo
- Publication number
- CN109614893B CN109614893B CN201811435888.6A CN201811435888A CN109614893B CN 109614893 B CN109614893 B CN 109614893B CN 201811435888 A CN201811435888 A CN 201811435888A CN 109614893 B CN109614893 B CN 109614893B
- Authority
- CN
- China
- Prior art keywords
- behavior
- abnormal
- video
- data
- normal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 206010000117 Abnormal behaviour Diseases 0.000 title claims abstract description 106
- 238000000034 method Methods 0.000 title claims abstract description 34
- 230000006399 behavior Effects 0.000 claims abstract description 150
- 238000001514 detection method Methods 0.000 claims abstract description 22
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 14
- 238000010801 machine learning Methods 0.000 claims abstract description 14
- 238000012544 monitoring process Methods 0.000 claims description 20
- 230000002159 abnormal effect Effects 0.000 claims description 13
- 230000009471 action Effects 0.000 claims description 13
- 238000012545 processing Methods 0.000 claims description 10
- 238000005286 illumination Methods 0.000 claims description 9
- 239000012634 fragment Substances 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 5
- 238000012549 training Methods 0.000 claims description 5
- 239000013598 vector Substances 0.000 claims description 5
- 230000001939 inductive effect Effects 0.000 claims description 4
- 238000002360 preparation method Methods 0.000 claims description 3
- 238000004590 computer program Methods 0.000 claims description 2
- 238000004458 analytical method Methods 0.000 description 4
- 238000010276 construction Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000002372 labelling Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 241000969729 Apteryx rowi Species 0.000 description 1
- 238000012935 Averaging Methods 0.000 description 1
- 230000004308 accommodation Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000009987 spinning Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24147—Distances to closest patterns, e.g. nearest neighbour classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Social Psychology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an abnormal behavior track intelligent identification method and device based on situation reasoning, wherein the method comprises the following steps: constructing a data set under multiple scenes, and establishing an abnormal behavior database and a normal behavior database according to the data set, wherein abnormal behaviors in the data set are marked; and according to the abnormal behavior feature library and the normal behavior feature library, adopting a pattern matching-based abnormal behavior trace detection method and a machine learning algorithm-based abnormal behavior trace detection method to conduct intelligent recognition of the abnormal behavior trace.
Description
Technical Field
The invention relates to the field of intelligent video monitoring, in particular to an abnormal behavior trace intelligent identification method and device based on situation reasoning.
Background
With the gradual development of information technology to intelligence and digitization and the improvement of people's safety consciousness, intelligent video monitoring plays an increasingly important role in identifying abnormal behavior whereabouts of people. The behaviors generated by the target object in different stages of collusion, implementation, hiding and the like have abnormal characteristics, on one hand, the behaviors of the target object are abnormal compared with most behaviors of common people, and on the other hand, the behaviors of the target object are abnormal compared with the daily behaviors of the target object. Aiming at the communication behavior, network behavior, economic behavior, posting behavior, telecommunication trace, vehicle trace, network trace, consumption trace, accommodation trace, aviation trace, railway trace and other behavior data information of the target object, the behavior habit of the target object is analyzed, and the actual situation is combined with the daily behavior of the target object or the behavior of other common people to develop, compare, excavate and comprehensively study and judge, so that the understanding and recognition of individual behaviors of people, the interaction behaviors among people and the external environment are realized, the abnormal behavior is judged, and the prediction and the early warning of the abnormal behavior are supported.
At present, most abnormal behavior trace intelligent recognition technologies mainly adopt expert knowledge modes, are based on experience modes of simple rules, and can not be used for spinning and cocoon stripping, seedling finding and clue finding from massive data which are seemingly scattered, disordered and irrelevant. The explored and developed analysis early warning model based on machine learning is also subject to understanding cognitive level of an advanced algorithm, becomes a 'black box model', and cannot be scientifically evaluated in adaptability and practicability in specific business scenes, and the accuracy and effectiveness of an analysis result cannot be scientifically determined.
Disclosure of Invention
The embodiment of the invention provides an abnormal behavior trace intelligent identification method and device based on situation reasoning, which are used for solving the problems in the prior art.
The embodiment of the invention provides an abnormal behavior trace intelligent identification method based on situation reasoning, which comprises the following steps:
constructing a data set under multiple scenes, and establishing an abnormal behavior feature library and a normal behavior feature library according to the data set, wherein abnormal behaviors in the data set are marked;
and according to the abnormal behavior feature library and the normal behavior feature library, adopting a pattern matching-based abnormal behavior trace detection method and a machine learning algorithm-based abnormal behavior trace detection method to conduct intelligent recognition of the abnormal behavior trace.
Preferably, constructing the data set in the multiple scenes specifically includes:
and processing the video data by adopting the video data of the normal behavior and the abnormal behavior recorded in the existing indoor and outdoor monitoring video system, analyzing and judging the behavior type appearing in the video, and acquiring labels of the behavior type to form a data set under multiple scenes.
Preferably, the video data of normal behavior and abnormal behavior recorded in the existing indoor and outdoor monitoring video system specifically includes:
a ready-made monitoring video acquisition system is adopted as a video acquisition environment established by a data set, a monitoring camera with higher resolution of a camera and better shooting angle is selected to acquire basic data; and selecting cameras with different resolutions and shooting angles to enrich the diversity of the data sets, wherein the acquisition of the basic data is carried out under the conditions of good illumination in the daytime, and the diversity of the data sets is increased by later preparation of the data under various illumination environments.
Preferably, processing the video data, analyzing and judging a behavior type appearing in the video, and acquiring a label of the behavior type, wherein forming a data set under multiple scenes specifically includes:
aiming at the characteristics of the abnormal behaviors of individuals/groups in indoor/outdoor designated scenes, acquiring definition of the abnormal behaviors, classification description of positive examples, negative examples and suspected examples of the abnormal behaviors, and acquiring definition of actions inducing the abnormal behaviors according to priori knowledge;
for the collected original video containing abnormal behaviors and normal behaviors, carrying out the normalization processing of spatial resolution and time resolution on the video according to the occurrence area of the abnormal behaviors and the intensity of the behaviors in the video;
aiming at acquiring the original video, firstly, intercepting video fragments according to the definition of behaviors, and regularizing the screened video fragments according to a certain frame rate and resolution; and obtaining labels of a start frame and an end frame of the behavior, labels of the position of the behavior object, labels of the behavior type and labels of specific properties of the behavior on the normalized video segment.
Preferably, the method for detecting the abnormal behavior trace based on pattern matching specifically comprises the following steps of:
extracting behavior trace characteristic data from real-time video data;
usage pattern identification I i Marking the behavior in the whereabouts characteristic data, a behavior I given a sliding window W i The occurrence is described asWherein I is i For behavior identification +.>For action I i Takes on the value of F, the value range is { zero, few, management }, NUM represents the behavior I i The number of occurrences in the sliding window W, +.>Is taken as the value of I i The maximum support degree value when the occurrence number is NUM; describing the occurrence of each behavior in the sliding window W to obtain a behavior sequence L of the target in the sliding window:
and matching the behavior sequence L with the abnormal feature library, if the matching is successful, determining the abnormal behavior trace, and if the matching is failed, determining the normal behavior trace.
Preferably, the abnormal behavior trace detection method based on the machine learning algorithm specifically comprises the following steps:
finding K of samples to be classified in training set i Nearest neighbor samples according to K i Predicting the class of the sample to be classified by a majority voting method according to class labels of nearest neighbor samples, and constructing a k-NN model by using normal behavior data;
based on the k-NN model, assume that X ' is a sample set containing n ' normal samples, namely a normal behavior feature library, and X ' = [ X ] 1 ，x 2 ，...，x n′ ]，X i For the sample to be detected, X is calculated according to equation 1 i And each sample X in X i Cosine similarity of 1.ltoreq.i.ltoreq.n', if X i And a certain sample X in X j Cosine similarity equal to 1, then X i And X is j Complete matching, X i Judging normal behavior, otherwise, finding out the sample X from n' samples contained in X i K' samples with highest cosine similarity are calculated again X i Average cosine similarity to the k' samples, if the averageThe similarity is greater than a predetermined similarity thresholdThen X is taken i Judging normal behavior; if X i If the normal behavior is not determined by the k-NN module, X i Is finally determined as abnormal behavior:
where "·" represents performing a dot product operation on two feature vectors.
The embodiment of the invention also provides an abnormal behavior trace intelligent identification device based on the situation reasoning, which comprises the following steps: a memory, a processor and a computer program stored on the memory and executable on the processor, which when executed by the processor, performs the steps of the method described above.
By adopting the embodiment of the invention, the mode of combining pattern matching and machine learning is utilized, and the intelligent recognition technology of the abnormal behavior trace based on the situation reasoning is explored by carrying out intelligent analysis on the monitoring video, so that the accurate recognition of the abnormal behavior of the specific target can be effectively realized.
The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a flow chart of a method for intelligent identification of abnormal behavior whereabouts based on contextual reasoning in accordance with an embodiment of the present invention;
FIG. 2 is a schematic diagram of an exemplary abnormal behavior data set construction process under multiple scenarios in an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The abnormal behavior track intelligent identification method based on the context reasoning mainly comprises two stages, namely a data set construction stage and an abnormal identification stage. The method specifically comprises the following steps:
step 1, constructing a data set under multiple scenes, and building an abnormal behavior feature library and a normal behavior feature library according to the data set, wherein abnormal behaviors in the data set are marked;
specifically, video data of normal behaviors and abnormal behaviors recorded in an existing indoor and outdoor monitoring video system are adopted, the video data are processed, behavior types appearing in the video are analyzed and judged, labels of the behavior types are obtained, and a data set under multiple scenes is formed. Wherein: the video data of normal behavior and abnormal behavior recorded in the existing indoor and outdoor monitoring video system specifically comprises:
a ready-made monitoring video acquisition system is adopted as a video acquisition environment established by a data set, a monitoring camera with higher resolution of a camera and better shooting angle is selected to acquire basic data; and selecting cameras with different resolutions and shooting angles to enrich the diversity of the data sets, wherein the acquisition of the basic data is carried out under the conditions of good illumination in the daytime, and the diversity of the data sets is increased by later preparation of the data under various illumination environments.
Processing video data, analyzing and judging the behavior type appearing in the video, and acquiring labels of the behavior type, wherein the forming of the data set under multiple scenes specifically comprises the following steps:
aiming at the characteristics of the abnormal behaviors of individuals/groups in indoor/outdoor designated scenes, acquiring definition of the abnormal behaviors, classification description of positive examples, negative examples and suspected examples of the abnormal behaviors, and acquiring definition of actions inducing the abnormal behaviors according to priori knowledge;
for the collected original video containing abnormal behaviors and normal behaviors, carrying out the normalization processing of spatial resolution and time resolution on the video according to the occurrence area of the abnormal behaviors and the intensity of the behaviors in the video;
aiming at acquiring the original video, firstly, intercepting video fragments according to the definition of behaviors, and regularizing the screened video fragments according to a certain frame rate and resolution; and obtaining labels of a start frame and an end frame of the behavior, labels of the position of the behavior object, labels of the behavior type and labels of specific properties of the behavior on the normalized video segment.
And 2, according to the abnormal behavior feature library and the normal behavior feature library, adopting an abnormal behavior track detection method based on pattern matching and an abnormal behavior track detection method based on a machine learning algorithm to intelligently identify the abnormal behavior track.
The method for detecting the abnormal behavior trace based on pattern matching specifically comprises the following steps of:
extracting behavior trace characteristic data from real-time video data;
usage pattern identification I i Marking the behavior in the whereabouts characteristic data, a behavior I given a sliding window W i The occurrence is described asWherein I is i For behavior identification +.>For action I i The frequency of occurrence F of (a) takes on a value of { zero, few, y }, NUM representing the rowIs I i The number of occurrences in the sliding window W, +.>Is taken as the value of I i The maximum support degree value when the occurrence number is NUM; describing the occurrence of each behavior in the sliding window W to obtain a behavior sequence L of the target in the sliding window:
and matching the behavior sequence L with the abnormal feature library, if the matching is successful, determining the abnormal behavior trace, and if the matching is failed, determining the normal behavior trace.
The abnormal behavior trace detection method based on the machine learning algorithm specifically comprises the following steps:
finding K of samples to be classified in training set i Nearest neighbor samples according to K i Predicting the class of the sample to be classified by a majority voting method according to class labels of nearest neighbor samples, and constructing a k-NN model by using normal behavior data;
based on the k-NN model, assume that X ' is a sample set containing n ' normal samples, namely a normal behavior feature library, and X ' = [ X ] 1 ，x 2 ，...，x n′ ]，X i For the sample to be detected, X is calculated according to equation 1 i And each sample X in X i Cosine similarity of 1.ltoreq.i.ltoreq.n', if X i And a certain sample X in X j Cosine similarity equal to 1, then X i And X is j Complete matching, X i Judging normal behavior, otherwise, finding out the sample X from n' samples contained in X i K' samples with highest cosine similarity are calculated again X i Average cosine similarity to the k' samples, if the average similarity is greater than a predetermined similarity thresholdThen X is taken i Determine as normal lineIs that; if X i If the normal behavior is not determined by the k-NN module, X i Is finally determined as abnormal behavior:
where "·" represents performing a dot product operation on two feature vectors.
The following describes the above technical solution of the embodiment of the present invention in detail.
1. Constructing datasets in multiple scenarios
In the practice of the present invention, abnormal behavior generally refers to unusual, behavior that violates a behavior criterion or law, such as fighting, stepping, etc. in public places, while the definition of abnormal behavior varies according to different scenes and time passes, such as wrapping legacy behavior, because legacy bomb events in recent years are progressively defined as potentially dangerous behavior in public places. Therefore, definition of typical abnormal behavior under multiple scenes and establishment of data sets thereof play an extremely important role for an intelligent video monitoring system.
The method and the device for establishing the data set under the multiple scenes mainly adopt video data of normal behaviors and abnormal behaviors recorded in the existing indoor and outdoor monitoring video system, and analyze and judge the behavior types appearing in the video to mark after the video data are processed, so that an abnormal behavior feature library and a normal behavior feature library under the multiple scenes are formed. The setup procedure is as shown in fig. 1:
1) Abnormal behavior definition
Firstly, aiming at the characteristics of abnormal behaviors of individuals/groups in indoor/outdoor designated scenes, such as illumination, time, characters, people, positions, appearance, gestures, actions, duration, spatial relations among the characters, time sequence logic, action characteristics and the like, the definition of the abnormal behaviors and the classification description of positive examples, negative examples and suspected examples of the abnormal behaviors are carried out for applications. Meanwhile, according to priori knowledge, actions inducing abnormal behaviors are defined, and data support is provided for subsequent abnormal intelligent recognition.
2) Video acquisition
The method comprises the steps that a ready-made monitoring video acquisition system is adopted as a video acquisition environment established by a data set, a monitoring camera with higher resolution of a camera and better shooting angle is selected to acquire basic data; and selecting the diversity of the camera rich data sets with different resolutions and shooting angles. The basic data is carried out under the conditions of good illumination in the daytime, and the data under various illumination environments are prepared in the later period to increase the diversity of the data set. In particular, video data corresponding to abnormal behaviors is enriched pertinently.
3) Data annotation and normalization
And labeling the original data which contains abnormal behaviors and normal behaviors and is acquired through the high-definition monitoring video system. Firstly, intercepting video clips according to the definition of behaviors, and regularizing the screened video clips according to a certain frame rate and resolution; on the normalized video clip, a start frame and an end frame of a behavior, a position of a behavior object, a type of behavior, a behavior specific attribute, and the like are marked. The labeling results of a plurality of labeling persons are comprehensively corrected by adopting a voting method and an averaging method to ensure the reliability of the labeling work. And for the acquired video, carrying out the normalization processing of the spatial resolution and the time resolution on the video according to the abnormal behavior occurrence area and the behavior intensity in the video.
2. Intelligent recognition and research judgment for abnormal behavior trace
The behavior trace data with high quality marking can be obtained through the previous processing, on the basis, the abnormal characteristics are summarized and analyzed through expert knowledge, and abnormal behavior trace detection based on pattern matching and an abnormal behavior trace detection method based on a machine learning algorithm are adopted, so that intelligent recognition and judgment of the abnormal behavior trace are realized.
1. Abnormal behavior trace identification detection based on pattern matching
The abnormal behavior trace identification detection is mainly based on the following assumptions: if a behavior is normal, the behavior will occur in large amounts over a longer period of time. Conversely, if a behavior is an abnormal behavior, the behavior may occur little or no during a longer period of time. Thus, in theory, a complete behavior pattern set can be extracted from behavior data over a period of time, and any behavior that does not conform to the behavior pattern set is determined to be abnormal.
The abnormal behavior trace identification and detection flow is as follows: firstly, extracting behavior trace characteristic data from real-time data, then comparing and analyzing the current behavior with a behavior pattern library, and judging whether the target behavior is abnormal or not.
In the pattern training phase, the behavior construction is performed on the basis of divided time slices, but in the pattern matching phase, the data stream arrives in real time, and at this time, the behavior sequence construction is performed on the basis of sliding windows.
In the behavior matching phase, pattern identification I is used i Marking the behavior in the whereabouts characteristic data, a behavior I given a sliding window W i The occurrence is described asWherein I is i For behavior identification, representing the mode to which the behavior belongs, < ->For action I i Takes on the value of F, the value range is { zero, few, management }, NUM represents the behavior I i The number of occurrences in the sliding window W, +.>Is taken as the value of I i The maximum support degree value when the occurrence number is NUM; for example, if the action a occurs 6 times in W, where the zero support is zero (6) =0, the few support is few (6) =0.1, and the many support is many (6) =0.9, the action a occurrence frequency F should be many, and the action a occurrence is described as (a, many). Describing the occurrence of each behavior in the sliding window W to obtain a behavior sequence L of the target in the sliding window: />
The matching process of the behavior sequence L is a process of finding a matching pattern from the behavior sequence pattern set MTX (i.e. abnormal behavior feature library) of the target, if the matching is successful, if the matching is not successful, the matching is failed.
The following provides an algorithm implementation of the sliding window based behavior sequence pattern matching as follows:
input: behavior sequence pattern set MTX, behavior set D in sliding window and target set U
And (3) outputting: pattern identification MTX i Or null (MTX) i For pattern identification of successful match, null represents failure of match
2. Abnormal behavior trace detection based on machine learning algorithm
The embodiment of the invention adopts an anomaly detection algorithm method based on K nearest neighbor to realize the anomaly behavior trace detection based on a machine learning algorithm, and the specific flow is as follows:
the operation flow of the k-NN classification algorithm is as follows: finding k of samples to be classified in training set i Nearest neighbor samples are then taken from this k i And predicting the class label of the nearest neighbor sample by a majority voting method. The present example uses only the normal sample set to construct the k-NN model. In addition, the embodiment of the invention uses cosine (cosine) as an index for measuring the similarity between samples, so that the value range of the similarity is [0,1 ]]. Two feature vectors X i And X j Cosine similarity cos (x i ,x j ) The calculation method of (2) is as follows:
where "·" in the formula represents performing a dot product operation on two feature vectors.
Let X ' be a sample set containing n ' normal samples, X ' = [ X ] 1 ，x 2 ，...，x n′ ]，X i For the sample to be detected, the k-NN detection algorithm constructed in the project is to calculate X i And each sample X in X i Cosine similarity of 1.ltoreq.i.ltoreq.n', if X i And a certain sample X in X j The cosine similarity is equal to 1 (i.e. cos (x i ，x j ) =1), then means X i And X is j Complete matching, and thus X can be directly used i And judging normal behavior. Otherwise, find the sum X from the n' samples contained in X i K' samples with highest cosine similarity are calculated again X i Average cosine similarity to the k' samples. If the average similarity is greater than a predetermined similarity threshold(wherein->Usually take a larger value, e.g. 0.99), X will be i Judging normal behavior; if X i If the normal behavior (corresponding false positive) is not determined by the k-NN module, X is i Is finally determined to be abnormal behavior.
In summary, the embodiment of the invention provides an abnormal behavior trace intelligent identification method and device based on situation reasoning, which are used for realizing intelligent identification of the target object abnormal behavior trace by constructing an abnormal behavior data set under multiple scenes and adopting a method of combining abnormal user behavior detection based on pattern matching and abnormal behavior intelligent detection based on machine learning. By means of mode matching and machine learning, intelligent analysis is conducted on the monitoring video, an abnormal behavior track intelligent recognition technology based on situation reasoning is explored, and accurate recognition of specific target abnormal behaviors can be effectively achieved.
The computer readable storage medium of the present embodiment includes, but is not limited to: ROM, RAM, magnetic or optical disks, etc.
It will be appreciated by those skilled in the art that the modules or steps of the invention described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may alternatively be implemented in program code executable by computing devices, so that they may be stored in a memory device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than that shown or described, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps within them may be fabricated into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (5)
1. The intelligent recognition method for the abnormal behavior trace based on the context reasoning is characterized by comprising the following steps of:
constructing a data set under multiple scenes, and establishing an abnormal behavior feature library and a normal behavior feature library according to the data set, wherein abnormal behaviors in the data set are marked;
according to the abnormal behavior feature library and the normal behavior feature library, adopting a pattern matching-based abnormal behavior trace detection method and a machine learning algorithm-based abnormal behavior trace detection method to conduct intelligent recognition of the abnormal behavior trace;
the method for detecting the abnormal behavior trace based on pattern matching specifically comprises the following steps of:
extracting behavior trace characteristic data from real-time video data;
usage pattern identification I i Marking the behavior in the whereabouts characteristic data, a behavior I given a sliding window W i The occurrence is described asWherein I is i For behavior identification +.>For action I i Takes on the value of F, the value range is { zero, few, management }, NUM represents the behavior I i The number of occurrences in the sliding window W, +.>Is taken as the value of I i The maximum support degree value when the occurrence number is NUM; describing the occurrence of each behavior in the sliding window W to obtain a behavior sequence L of the target in the sliding window:
matching the behavior sequence L with an abnormal feature library, if the matching is successful, determining that the behavior sequence L is abnormal, and if the matching is failed, determining that the behavior sequence L is normal;
the abnormal behavior trace detection method based on the machine learning algorithm specifically comprises the following steps:
finding K of samples to be classified in training set i Nearest neighbor samples according to K i Predicting the class of the sample to be classified by a majority voting method according to class labels of nearest neighbor samples, and constructing a k-NN model by using normal behavior data;
based on the k-NN model, assume that X ' is a sample set containing n ' normal samples, namely a normal behavior feature library, and X ' = [ X ] 1 ，x 2 ，...，x n′ ]，X i For the sample to be detected, X is calculated according to equation 2 i And each sample X in X i Cosine similarity of 1.ltoreq.i.ltoreq.n', if X i And a certain sample X in X j Cosine similarity equal to 1, then X i And x j Complete matching, X i Judging normal behavior, otherwise, finding out the sample X from n' samples contained in X i K' samples with highest cosine similarity are calculated again X i Average cosine similarity to the k' samples, if the average cosine similarity is greater than a predetermined similarity thresholdThen X is taken i Judging normal behavior; if X i If the normal behavior is not determined by the k-NN module, X i Is finally determined as abnormal behavior:
where "·" represents performing a dot product operation on two feature vectors.
2. The method of claim 1, wherein constructing the dataset in the multiple scenarios specifically comprises:
and processing the video data by adopting the video data of the normal behavior and the abnormal behavior recorded in the existing indoor and outdoor monitoring video system, analyzing and judging the behavior type appearing in the video, and acquiring labels of the behavior type to form a data set under multiple scenes.
3. The method of claim 2, wherein using video data of normal and abnormal behavior recorded in an existing indoor and outdoor surveillance video system specifically comprises:
a ready-made monitoring video acquisition system is adopted as a video acquisition environment established by a data set, a monitoring camera with higher resolution of a camera and better shooting angle is selected to acquire basic data; and selecting cameras with different resolutions and shooting angles to enrich the diversity of the data sets, wherein the acquisition of the basic data is carried out under the conditions of good illumination in the daytime, and the diversity of the data sets is increased by later preparation of the data under various illumination environments.
4. The method of claim 3, wherein processing the video data, analyzing the behavior types occurring in the video and obtaining labels for the behavior types, and forming the dataset in the multiple scenarios comprises:
aiming at the characteristics of the abnormal behaviors of individuals/groups in indoor/outdoor designated scenes, acquiring definition of the abnormal behaviors, classification description of positive examples, negative examples and suspected examples of the abnormal behaviors, and acquiring definition of actions inducing the abnormal behaviors according to priori knowledge;
for the collected original video containing abnormal behaviors and normal behaviors, carrying out the normalization processing of spatial resolution and time resolution on the video according to the occurrence area of the abnormal behaviors and the intensity of the behaviors in the video;
aiming at acquiring the original video, firstly, intercepting video fragments according to the definition of behaviors, and regularizing the screened video fragments according to a certain frame rate and resolution; and obtaining labels of a start frame and an end frame of the behavior, labels of the position of the behavior object, labels of the behavior type and labels of specific properties of the behavior on the normalized video segment.
5. The utility model provides an unusual behavior whereabouts intelligent identification device based on situation reasoning which characterized in that includes: memory, a processor and a computer program stored on the memory and executable on the processor, which when executed by the processor performs the steps of the method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811435888.6A CN109614893B (en) | 2018-11-28 | 2018-11-28 | Intelligent abnormal behavior track identification method and device based on situation reasoning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811435888.6A CN109614893B (en) | 2018-11-28 | 2018-11-28 | Intelligent abnormal behavior track identification method and device based on situation reasoning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109614893A CN109614893A (en) | 2019-04-12 |
| CN109614893B true CN109614893B (en) | 2023-10-24 |
Family
ID=66006366
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811435888.6A Active CN109614893B (en) | 2018-11-28 | 2018-11-28 | Intelligent abnormal behavior track identification method and device based on situation reasoning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109614893B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112905619A (en) * | 2020-10-10 | 2021-06-04 | 陈夏焱 | Abnormal label classification method and system based on artificial intelligence and big data |
| CN113254495A (en) * | 2021-04-06 | 2021-08-13 | 苏立双 | Big data service platform based on artificial intelligence |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8515160B1 (en) * | 2007-10-04 | 2013-08-20 | Hrl Laboratories, Llc | Bio-inspired actionable intelligence method and system |
| CN103854027A (en) * | 2013-10-23 | 2014-06-11 | 北京邮电大学 | Crowd behavior identification method |
| CN102811343B (en) * | 2011-06-03 | 2015-04-29 | 南京理工大学 | Intelligent video monitoring system based on behavior recognition |
| CN107491749A (en) * | 2017-08-11 | 2017-12-19 | 南京邮电大学 | Global and local anomaly detection method in a kind of crowd's scene |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9355306B2 (en) * | 2013-09-27 | 2016-05-31 | Konica Minolta Laboratory U.S.A., Inc. | Method and system for recognition of abnormal behavior |
-
2018
- 2018-11-28 CN CN201811435888.6A patent/CN109614893B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8515160B1 (en) * | 2007-10-04 | 2013-08-20 | Hrl Laboratories, Llc | Bio-inspired actionable intelligence method and system |
| CN102811343B (en) * | 2011-06-03 | 2015-04-29 | 南京理工大学 | Intelligent video monitoring system based on behavior recognition |
| CN103854027A (en) * | 2013-10-23 | 2014-06-11 | 北京邮电大学 | Crowd behavior identification method |
| CN107491749A (en) * | 2017-08-11 | 2017-12-19 | 南京邮电大学 | Global and local anomaly detection method in a kind of crowd's scene |
Non-Patent Citations (1)
| Title |
|---|
| 视频异常行为识别与分级预警系统;杨谦等;《科学技术与工程》;20150518(第14期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109614893A (en) | 2019-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108133172B (en) | Method for classifying moving objects in video and method and device for analyzing traffic flow | |
| Bertini et al. | Multi-scale and real-time non-parametric approach for anomaly detection and localization | |
| CN105787472B (en) | A kind of anomaly detection method based on the study of space-time laplacian eigenmaps | |
| Rabiee et al. | Novel dataset for fine-grained abnormal behavior understanding in crowd | |
| CN112491796B (en) | Intrusion detection and semantic decision tree quantitative interpretation method based on convolutional neural network | |
| US10956753B2 (en) | Image processing system and image processing method | |
| EP1835462A1 (en) | Tracing device, and tracing method | |
| CN104915655A (en) | Multi-path monitor video management method and device | |
| KR102028930B1 (en) | method of providing categorized video processing for moving objects based on AI learning using moving information of objects | |
| CN109033476A (en) | A kind of intelligent space-time data event analysis method based on event clue network | |
| CN112507860A (en) | Video annotation method, device, equipment and storage medium | |
| CN109117771B (en) | System and method for detecting violence events in image based on anchor nodes | |
| US20230186634A1 (en) | Vision-based monitoring of site safety compliance based on worker re-identification and personal protective equipment classification | |
| CN109614893B (en) | Intelligent abnormal behavior track identification method and device based on situation reasoning | |
| CN110826390A (en) | Video data processing method based on face vector characteristics | |
| Turchini et al. | Convex polytope ensembles for spatio-temporal anomaly detection | |
| CN116311063A (en) | Personnel fine granularity tracking method and system based on face recognition under monitoring video | |
| Hirzi et al. | Literature study of face recognition using the viola-jones algorithm | |
| CN110969645A (en) | Unsupervised abnormal track detection method and unsupervised abnormal track detection device for crowded scenes | |
| CN109271859A (en) | It combines related cases method and apparatus, electronic equipment, computer storage medium | |
| Zhang et al. | A Multiple Instance Learning and Relevance Feedback Framework for Retrieving Abnormal Incidents in Surveillance Videos. | |
| KR101848367B1 (en) | metadata-based video surveillance method using suspective video classification based on motion vector and DCT coefficients | |
| Ortego et al. | Long-term stationary object detection based on spatio-temporal change detection | |
| Nayak et al. | Exploiting spatio-temporal scene structure for wide-area activity analysis in unconstrained environments | |
| CN111062294B (en) | Passenger flow queuing time detection method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |