WO2020135392A1

WO2020135392A1 - Method and device for detecting abnormal behavior

Info

Publication number: WO2020135392A1
Application number: PCT/CN2019/127797
Authority: WO
Inventors: 赵飞
Original assignee: 杭州海康威视数字技术股份有限公司
Priority date: 2018-12-24
Filing date: 2019-12-24
Publication date: 2020-07-02
Also published as: CN111353352B; CN111353352A

Abstract

The present disclosure relates to the field of video surveillance, and disclosed thereby are a method and device for detecting abnormal behavior. The method comprises: acquiring behavior data to be detected; inputting the behavior data into a feature extraction model and outputting a behavior feature of the behavior data, wherein the feature extraction model is used to output a behavior feature within a feature space range according to normal behavior data and output a behavior feature outside of the feature space range according to abnormal behavior data, and the distance between each behavior feature within the feature space range is less than a distance threshold; according to the distance between the behavior feature of the behavior data and a normal behavior feature center and the distance threshold, acquiring a detection result of the behavior data, the detection result being used to indicate whether the behavior data is abnormal behavior data, and the normal behavior feature center being used to represent the behavior features within the feature space range. The described method for detecting abnormal behavior on the basis of distance measurement described in the present disclosure has high accuracy.

Description

Abnormal behavior detection method and device

This application requires the priority of the Chinese patent application with the application number 201811581954.0 and the invention titled "abnormal behavior detection method and device" filed on December 24, 2018, the entire content of which is incorporated by reference in this application.

Technical field

The present disclosure relates to the field of video surveillance, and in particular to an abnormal behavior detection method and device.

Background technique

Abnormal behavior detection means that in the video surveillance scene, the computer equipment replaces the video surveillance personnel, and automatically detects the abnormal behavior in the video surveillance scene, so that the alarm can be promptly performed. Among them, abnormal behavior generally refers to behaviors in the scene that are significantly different from other behaviors or have a low probability of occurring in the scene, such as behaviors that endanger others or harm the public interest. Abnormal behavior detection makes video surveillance personnel free from massive monitoring data and tedious manual operations, and has extremely extensive applications in the field of video surveillance.

In related technologies, OneClassSVM (One Class Support Vector Machine) is generally used to detect abnormal behavior. Specifically, a large number of surveillance videos where normal behavior occurs are collected, image sequences are extracted from the surveillance videos as normal behavior data, and a class one classifier is trained based on the normal behavior data. After acquiring an unknown video, if you want to determine whether the behavior data includes abnormal behavior, you can extract the image sequence as behavior data to extract the behavior characteristics of the behavior data. Determine whether the behavioral characteristics of the behavioral data are consistent with the behavioral characteristics included in the above-mentioned classifier. If the behavioral characteristics are inconsistent with the behavioral characteristics included in the classifier, then the behavioral data is considered to correspond to abnormal behavior, that is, the unknown video includes Abnormal behavior. Among them, the behavior features included in this class of classifier are obtained by performing feature extraction on normal behavior data.

The above technology is based on the behavior characteristics extracted from the normal behavior data to determine whether there is abnormal behavior in the unknown video. Since the difference between the normal behavior data and the abnormal behavior data is not learned, the detection result is prone to large deviations and the accuracy of abnormal behavior detection is poor.

Summary of the invention

The embodiments of the present disclosure provide an abnormal behavior detection method and device, which can solve the problem of poor accuracy of related technologies. The technical solution is as follows:

In a first aspect, a method for detecting abnormal behavior is provided. The method includes:

Obtain the behavior data to be detected;

Inputting the behavior data into a feature extraction model and outputting behavior characteristics of the behavior data, the feature extraction model is used to output behavior characteristics within the feature space range according to normal behavior data and out of the feature space range according to abnormal behavior data Behavioral characteristics, the distance between the behavioral characteristics within the feature space is less than the distance threshold;

Acquiring the detection result of the behavior data according to the distance between the behavior feature of the behavior data and the center of the normal behavior feature and the distance threshold, the detection result is used to indicate whether the behavior data is abnormal behavior data, the normal The behavior feature center is used to represent behavior features within the range of the feature space.

In a possible implementation manner, the training process of the feature extraction model includes:

Acquiring a plurality of first behavior data pairs and a plurality of second behavior data pairs according to the normal behavior data set and the abnormal behavior data set, each first behavior data pair containing two normal behavior data in the normal behavior data set, Each second behavior data pair includes one normal behavior data in the normal behavior data set and one abnormal behavior data in the abnormal behavior data set;

Extracting a plurality of first behavior feature pairs of the plurality of first behavior data pairs and a plurality of second behavior feature pairs of the plurality of second behavior data pairs, each first behavior feature pair containing two normal behavior data Behavior characteristics, each second behavior characteristic pair contains a behavior characteristic of normal behavior data and a behavior characteristic of abnormal behavior data;

According to the distance between the two behavioral features included in each of the first behavioral feature pairs and the distance between the two behavioral features included in each of the second behavioral feature pairs, supervise training through a loss function to obtain the Feature extraction model.

In a possible implementation manner, before acquiring multiple first behavior data pairs and multiple second behavior data pairs according to the normal behavior data set and the abnormal behavior data set, the method further includes:

Acquiring the normal behavior data set based on a plurality of first videos, the plurality of first videos being videos of normal behavior targeted by the target;

Based on the plurality of second videos, the abnormal behavior data set is acquired, and the plurality of second videos are videos of the target performing abnormal behavior.

In a possible implementation manner, the acquiring the normal behavior data set based on multiple first videos includes:

For each first video in the plurality of first videos, detect and track the target in the first video to obtain the spatial motion range of the target in the first time period, the spatial motion range is The spatial range covered by the target motion, the duration of the first time period is less than the duration of the first video period;

According to the spatial motion range and the first video, performing image interception in a first video sequence corresponding to the first time period to obtain a first image sequence of the first video, the first video sequence including A multi-frame video image of the first video, the first image sequence includes an area corresponding to the spatial motion range in the multi-frame video image;

The first image sequence of the plurality of first videos is used as the normal behavior data set.

In a possible implementation manner, the acquiring the abnormal behavior data set based on multiple second videos includes:

For each second video in the plurality of second videos, detect and track the target in the second video to obtain the spatial motion range of the target in the second time period, the spatial motion range is The spatial range covered by the target motion, the duration of the second time period is less than the duration of the second video period;

According to the spatial motion range and the second video, performing image interception in a second video sequence corresponding to the second time period to obtain a second image sequence of the second video, the second video sequence including A multi-frame video image of the second video, the second image sequence includes an area corresponding to the spatial motion range in the multi-frame video image;

The second image sequence of the plurality of second videos is used as the abnormal behavior data set.

In a possible implementation manner, the behavior data to be detected is multiple behavior data,

After obtaining the detection result of the behavior data according to the distance between the behavior characteristic of the behavior data and the center of the normal behavior characteristic, the method further includes:

Determine abnormal behavior data among the plurality of behavior data according to respective detection results of the plurality of behavior data;

Adding abnormal behavior data from the plurality of behavior data to the abnormal behavior data set;

Based on the updated abnormal behavior data set, the training process of the feature extraction model is performed to obtain the updated feature extraction model.

In a possible implementation manner, the adding abnormal behavior data from the plurality of behavior data to the abnormal behavior data set includes:

Acquiring manual confirmation information of abnormal behavior data among the plurality of behavior data;

Adding the abnormal behavior data indicated by the manual confirmation information to the abnormal behavior data set.

In a possible implementation manner, the behavior data to be detected is a plurality of behavior data;

Determine the normal behavior data among the plurality of behavior data according to the respective detection results of the plurality of behavior data;

Adding normal behavior data from the plurality of behavior data to the normal behavior data set;

The updating based on the abnormal behavior data set, performing the training process of the feature extraction model, and obtaining the updated feature extraction model includes:

Based on the updated abnormal behavior data set and the updated normal behavior data set, the training process of the feature extraction model is performed to obtain the updated feature extraction model.

In a possible implementation manner, the method further includes:

Get multiple videos;

For each of the plurality of videos, detect and track the target in the video to obtain the spatial motion range of the target in the third time period, the spatial motion range is covered by the target motion The spatial range of the third time period is less than the time period of the video;

According to the spatial motion range and the video, performing image interception in the video sequence corresponding to the third time period to obtain an image sequence of the video, the video sequence including multiple frames of video images of the video, so The image sequence includes an area corresponding to the spatial motion range in the multi-frame video image;

The image sequence of the multiple videos is used as the multiple behavior data.

In a possible implementation manner, the method further includes:

For the abnormal behavior data among the plurality of behavior data, in the process of playing the video to which the abnormal behavior data belongs, the image sequence of the video to which the abnormal behavior data belongs is displayed.

In a possible implementation manner, the acquiring the detection result of the behavior data according to the distance between the behavior feature of the behavior data and the center of the normal behavior feature and the distance threshold includes:

When the distance between the behavior feature of the behavior data and the center of the normal behavior feature is greater than the distance threshold, it is determined that the behavior data is abnormal behavior data;

When the distance between the behavior feature of the behavior data and the center of the normal behavior feature is less than or equal to the distance threshold, it is determined that the behavior data is normal behavior data.

In a possible implementation manner, the process of acquiring the normal behavior feature center includes:

Obtain multiple normal behavior data;

For each normal behavior data in the plurality of normal behavior data, input the normal behavior data into the feature extraction model, and output the behavior characteristics of the normal behavior data;

Obtain the normal behavior characteristic center according to the behavior characteristic of the plurality of normal behavior data.

In a possible implementation manner, the behavior characteristic of each normal behavior data in the plurality of normal behavior data is characterized by a feature vector;

The obtaining the normal behavior characteristic center according to the behavior characteristics of the plurality of normal behavior data includes:

Calculating an average value of the feature vectors of the plurality of normal behavior data in each dimension to obtain a target feature vector characterized by a set of average values consisting of the average value of each dimension;

Use the target feature vector as the normal behavior feature center.

In a second aspect, an abnormal behavior detection device is provided, the device including:

The acquisition module is used to acquire the behavior data to be detected;

An extraction module is used to input the behavior data into a feature extraction model and output behavior characteristics of the behavior data, and the feature extraction model is used to output behavior characteristics within a feature space range according to normal behavior data and output data based on abnormal behavior data. The behavior features outside the feature space, the distance between the behavior features in the feature space is less than the distance threshold;

The acquiring module is further configured to acquire the detection result of the behavior data according to the distance between the behavior feature of the behavior data and the center of the normal behavior feature and the distance threshold, and the detection result is used to indicate whether the behavior data is For abnormal behavior data, the normal behavior feature center is used to represent behavior features within the feature space.

In a possible implementation manner, the obtaining module is further used to:

In a possible implementation manner, the acquisition module is used to:

The acquiring module is further configured to determine abnormal behavior data in the plurality of behavior data according to the detection results of the plurality of behavior data; add the abnormal behavior data in the plurality of behavior data to the abnormal behavior In the data set; based on the updated abnormal behavior data set, the training process of the feature extraction model is performed to obtain the updated feature extraction model.

In a possible implementation manner, the behavior data to be detected is multiple behavior data;

After acquiring the detection result of the behavior data according to the distance between the behavior characteristic of the behavior data and the center of the normal behavior characteristic, the acquisition module is further used to:

In a possible implementation manner, the acquiring module is configured to acquire artificial confirmation information of abnormal behavior data among the plurality of behavior data; add abnormal behavior data indicated by the manual confirmation information to the abnormal behavior data set .

In a possible implementation manner, the obtaining module is further used to:

Get multiple videos;

The image sequence of the plurality of videos is used as the plurality of behavior data.

In a possible implementation manner, the device further includes:

The display module is configured to display the image sequence of the video to which the abnormal behavior data belongs during the playing of the video to which the abnormal behavior data belongs to among the plurality of abnormal behavior data.

In a possible implementation manner, the acquisition module is used to:

In a possible implementation manner, the obtaining module is further used to:

Obtain multiple normal behavior data;

The acquisition module is used to:

Use the target feature vector as the normal behavior feature center.

In a third aspect, a computer device is provided, including a processor and a memory; the memory is used to store at least one instruction; the processor executes at least one instruction stored on the memory to implement the first Aspects of abnormal behavior detection methods.

According to a fourth aspect, a computer-readable storage medium is provided, in which a computer program is stored, and when the computer program is executed by a processor, the method for detecting abnormal behavior of the first aspect described above is implemented.

The beneficial effects brought by the technical solutions provided by the embodiments of the present disclosure include at least:

Use the feature extraction model to extract the behavior features of the behavior data, and determine whether the behavior data is abnormal behavior data according to the distance and distance threshold between the extracted behavior features and the normal behavior center. Since the feature extraction model is trained based on the distance constraint method, normal The behavioral features extracted by the behavioral data through the feature extraction model are in a relatively small feature space, and the behavioral features extracted by the characteristic extraction model through the feature extraction model are outside the feature space, which ensures that the normal behavioral features are relatively compact and the abnormal behavior There is a clear distance between the feature and the normal behavior feature. Since the difference between normal behavior and abnormal behavior is learned, this method of abnormal behavior detection based on distance measurement has high accuracy.

BRIEF DESCRIPTION

In order to more clearly explain the technical solutions in the embodiments of the present disclosure, the drawings required in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For a person of ordinary skill in the art, without paying any creative work, other drawings can be obtained based on these drawings.

1 is a flowchart of a method for detecting abnormal behavior provided by an embodiment of the present disclosure;

2 is a flowchart of an abnormal behavior detection method provided by an embodiment of the present disclosure;

3 is a training flowchart of a feature extraction model provided by an embodiment of the present disclosure;

4 is a flowchart of an abnormal behavior detection provided by an embodiment of the present disclosure;

5 is a feedback update flowchart of abnormal behavior detection provided by an embodiment of the present disclosure;

6 is a schematic structural diagram of an abnormal behavior detection device provided by an embodiment of the present disclosure;

7 is a schematic structural diagram of an abnormal behavior detection device provided by an embodiment of the present disclosure;

FIG. 8 is a schematic structural diagram of a computer device 800 according to an embodiment of the present disclosure.

detailed description

To make the objectives, technical solutions, and advantages of the present disclosure more clear, the embodiments of the present disclosure will be described in further detail below in conjunction with the accompanying drawings.

FIG. 1 is a flowchart of an abnormal behavior detection method provided by an embodiment of the present disclosure. Referring to Figure 1, the method includes:

101. Obtain behavior data to be detected.

102. Input the behavior data into a feature extraction model, and output behavior characteristics of the behavior data. The feature extraction model is used to output behavior characteristics within a characteristic space range according to normal behavior data and output behavior outside the characteristic space range according to abnormal behavior data. Feature, the distance between each behavior feature in the feature space is less than the distance threshold.

103. Obtain the detection result of the behavior data according to the distance between the behavior feature of the behavior data and the center of the normal behavior feature and the distance threshold, the detection result is used to indicate whether the behavior data is abnormal behavior data, and the normal behavior feature center uses Yu represents behavioral features within the range of the feature space.

The method provided in the embodiment of the present disclosure extracts the behavior characteristics of the behavior data through the feature extraction model, and determines whether the behavior data is abnormal behavior data according to the distance and the distance threshold between the extracted behavior characteristics and the normal behavior center. Because the feature extraction model is trained based on the distance constraint method, the behavior features extracted by the normal behavior data through the feature extraction model are in a relatively small feature space, and the behavior features extracted by the abnormal behavior data through the feature extraction model are in the feature space Outside the scope, it can be seen that the normal behavior characteristics are relatively compact, and there is a clear distance between the abnormal behavior characteristics and the normal behavior characteristics. In this way, since the difference between the normal behavior and the abnormal behavior is learned, the abnormal behavior can be detected based on the difference (that is, the abnormal behavior is detected based on the distance between the abnormal behavior characteristic and the normal behavior characteristic), so that the accuracy of the abnormal behavior detection is high.

According to the normal behavior data set and the abnormal behavior data set, multiple first behavior data pairs and multiple second behavior data pairs are obtained, and each first behavior data pair includes two normal behavior data in the normal behavior data set, each A second behavior data pair includes a normal behavior data in the normal behavior data set and an abnormal behavior data in the abnormal behavior data set;

Extracting a plurality of first behavior feature pairs of the plurality of first behavior data pairs and a plurality of second behavior feature pairs of the plurality of second behavior data pairs, each first behavior feature pair containing two behaviors of normal behavior data Characteristics, each second behavior characteristic pair includes a behavior characteristic of normal behavior data and a behavior characteristic of abnormal behavior data;

According to the distance between the two behavior features included in each first behavior feature pair and the distance between the two behavior features included in each second behavior feature pair, the feature extraction model is obtained through supervised training by a loss function .

For each first video in the plurality of first videos, the target in the first video is detected and tracked to obtain the spatial motion range of the target in the first time period, the spatial motion range is determined by the target motion The spatial range covered, the duration of the first time period is less than the duration of the first video period;

According to the spatial motion range and the first video, perform image interception in the first video sequence corresponding to the first time period to obtain a first image sequence of the first video, the first video sequence including the first video sequence A multi-frame video image, the first image sequence includes an area corresponding to the spatial motion range in the multi-frame video image;

In a possible implementation manner, the process of obtaining the abnormal behavior data set includes:

For each second video in the plurality of second videos, the target in the second video is detected and tracked to obtain the spatial motion range of the target in the second time period, the spatial motion range is the target motion range The spatial range covered, the duration of the second time period is less than the duration of the second video period;

According to the spatial motion range and the second video, performing image interception in the second video sequence corresponding to the second time period to obtain a second image sequence of the second video, the second video sequence including the second video A multi-frame video image, the second image sequence includes an area corresponding to the spatial motion range in the multi-frame video image;

Determine abnormal behavior data among the plurality of behavior data according to the detection results of the plurality of behavior data;

Add the abnormal behavior data in the multiple behavior data to the abnormal behavior data set;

In a possible implementation manner, the addition of abnormal behavior data in the plurality of behavior data to the abnormal behavior data set includes:

Obtain manual confirmation information of abnormal behavior data among the multiple behavior data;

The abnormal behavior data indicated by the manual confirmation information is added to the abnormal behavior data set.

Add the normal behavior data from the multiple behavior data to the normal behavior data set;

Based on the updated abnormal behavior data set, the training process of the feature extraction model is performed to obtain the updated feature extraction model, including:

In a possible implementation manner, the method further includes:

Get multiple videos;

For each video in the plurality of videos, detect and track the target in the video to obtain the spatial motion range of the target in the third time period, the spatial motion range is the spatial range covered by the target motion, the The duration of the third time period is less than the duration of the video period;

According to the spatial motion range and the video, perform image interception in the video sequence corresponding to the third time period to obtain an image sequence of the video, the video sequence includes multiple frames of the video image of the video, and the image sequence includes the multiple frames The area in the video image corresponding to the spatial motion range;

In a possible implementation manner, the method further includes:

For the abnormal behavior data in the plurality of behavior data, in the process of playing the video to which the abnormal behavior data belongs, the image sequence of the video to which the abnormal behavior data belongs is displayed.

In a possible implementation manner, obtaining the detection result of the behavior data according to the distance between the behavior feature of the behavior data and the center of the normal behavior feature and the distance threshold includes:

When the distance between the behavior feature of the behavior data and the center of the normal behavior feature is greater than the distance threshold, the behavior data is determined to be abnormal behavior data;

When the distance between the behavior feature of the behavior data and the center of the normal behavior feature is less than or equal to the distance threshold, the behavior data is determined to be normal behavior data.

Obtain multiple normal behavior data;

According to the behavior characteristics of the plurality of normal behavior data, the normal behavior characteristic center is obtained.

Obtaining the normal behavior characteristic center according to the behavior characteristics of the plurality of normal behavior data includes:

Calculate the average value of the feature vectors of the multiple normal behavior data in each dimension to obtain a target feature vector characterized by a set of average values composed of the average values of each dimension;

Use the target feature vector as the normal behavior feature center.

All of the above optional technical solutions may be combined in any combination to form optional embodiments of the present disclosure, and details are not repeated herein.

2 is a flowchart of an abnormal behavior detection method provided by an embodiment of the present disclosure. The method is executed by a computer device. Referring to FIG. 2, the method includes:

201. Acquire normal behavior data collection and abnormal behavior data collection.

Among them, the normal behavior data set contains multiple normal behavior data, and the abnormal behavior data set contains multiple abnormal behavior data.

In a possible implementation manner, the behavior data in the normal behavior data set and the abnormal behavior data set may be obtained based on video, and accordingly, step 201 may include: acquiring the normal behavior data set based on multiple first videos, The plurality of first videos are videos of the target performing normal behavior; based on the plurality of second videos, the abnormal behavior data set is acquired, and the plurality of second videos are videos of the target performing abnormal behavior.

The plurality of first videos and the plurality of second videos may be collected by relevant personnel according to preset normal behavior categories and stored on the computer device. For normal behavior and abnormal behavior, since the normal behavior category can be preset, the category of normal behavior can be arbitrarily specified according to the application scenario, and behaviors different from the normal behavior are considered abnormal behaviors. Normal behaviors can include, but are not limited to, normal walking, meditation, and a series of normal behaviors related to a specific scene, and abnormal behaviors can include, but are not limited to, riots, conflicts, and a series of specific scene-related behaviors. For example, for daily life scenarios, you can specify that normal walking and meditation are normal behaviors, while behaviors such as riots and conflicts are abnormal behaviors. For the bank counter scene, it can be specified that the behavior of standing upright and counting banknotes is normal behavior, while the behavior of making calls and putting banknotes in pockets is abnormal behavior.

For the process of acquiring the normal behavior data set, the process of acquiring the normal behavior data set based on multiple first videos may include the following steps a1 to a3:

Step a1. For each first video of the plurality of first videos, detect and track the target in the first video to obtain the spatial motion range of the target in the first time period, the spatial motion range is The spatial range covered by the target motion, the duration of the first time period is less than the duration of the first video period.

The first time period may be a time period of a video sequence of the first video, where the video sequence includes multiple frames of video images of the first video, such as f1, ..., fn. For example, the first time period is 15 seconds, the first video is 3 minutes, and the first time period is any consecutive 15 seconds of the first video. Targets refer to people, cars, animals, etc.

The computer device may use a target detection and tracking algorithm to detect and track the target in the first video, determine the position of the target at each moment in the first time period, and thereby determine the spatial motion range of the target and the space of the target The motion range is used to characterize the range of changes in the position of the target in the video image during the first time period. Among them, target detection and tracking algorithms include but are not limited to DPM (Deformable Part Model), FRCNN (Faster Region-based Convolutional Neural Networks, rapid detection model of convolutional neural network based on candidate regions), YOLO (You Only Look, Once,), SSD (Single Shot Multibox Detector,), etc. The position of the target at each moment can be represented by adding a target frame to the video image. The form of the target frame includes but is not limited to a circumscribed rectangular frame, a circumscribed circular frame, and a circumscribed polygonal frame.

Taking the target frame as an external rectangular frame as an example, in the video sequence f1, ..., fn, it is assumed that the computer device uses a target detection and tracking algorithm to perform target detection and tracking on the video sequence, and in turn obtains how many The target frame in the frame video image, from which you can get a series of target frames R1, ..., Rm, where m and n are positive integers, m<=n (when m<n indicates that there is a frame loss phenomenon in tracking, in m=n indicates that each frame of video image includes a target frame). Any target frame can be expressed as: R = [left_top_x, left_top_y, right_bottom_x, right_bottom_y], where left_top_x and left_top_y are used to describe the upper left corner coordinates of the target frame under the rectangular coordinate system, right_bottom_x and right_bottom_y are used to describe the rectangular coordinate system The coordinates of the lower right corner of the lower target frame (the origin of the rectangular coordinate system may be the upper left corner of the image where the target frame is located). After the computer device obtains a series of target frames, the spatial motion range of the target can be obtained based on the target frame. The spatial motion range of the target can be expressed as: R _tube =[min({left_top_x}),min({left_top_y}),max ({right_bottom_x}),max({right_bottom_y})]. Among them, min({left_top_x}) represents the minimum horizontal coordinate of the upper left corner of the target frame, min({left_top_y}) represents the minimum vertical coordinate of the upper left corner of the target frame, and max({right_bottom_x}) represents the maximum of the lower right corner of the target frame The abscissa, max({right_bottom_y}) represents the maximum ordinate of the lower right corner of the target frame.

Step a2: According to the spatial motion range and the first video, perform image interception in the first video sequence corresponding to the first time period to obtain the first image sequence of the first video. The first video sequence includes multi-frame video images of the first video, and the first image sequence includes regions corresponding to the spatial motion range in the multi-frame video images.

The computer device may perform image interception on each frame of the video image included in the video sequence according to the spatial motion range in step a1, and intercept the area corresponding to the spatial motion range from each frame of the video image. The sequence constitutes the first image sequence, which can reflect the motion information of the target in time and space. For example, use R _tube =[min({left_top_x}),min({left_top_y}),max({right_bottom_x}),max({right_bottom_y})], and then intercept the upper left corner in the video sequence f1,...,fn The rectangular area with the coordinates of (min({left_top_x}), min({left_top_y})), and the coordinates of the bottom right corner (max({right_bottom_x}), max({right_bottom_y})) gets the video of each frame in the video sequence The rectangular area in the image, the rectangular areas sequentially intercepted constitute the first image sequence. This method of image sequence extraction can greatly reduce background information while retaining the target's behavior without losing information, and is more conducive to the feature extraction model to extract the target's behavioral features.

In this way, since there are multiple first videos, it is possible to obtain first image sequences corresponding to the multiple first videos, that is, there are multiple first image sequences.

Step a3: Use the first image sequence of the plurality of first videos as the normal behavior data set.

Through step a1 and step a2, the computer device can obtain the first image sequence of each first video in the plurality of first videos, and use each first image sequence as a behavior data (or behavior sequence) to form a normal behavior data set .

For the process of acquiring abnormal behavior data sets, the process of acquiring the abnormal behavior data sets based on multiple second videos may include the following steps b1 to b3:

Step b1: For each second video in the plurality of second videos, the computer device detects and tracks the target in the second video to obtain the spatial motion range of the target in the second time period, the spatial motion range For the spatial range covered by the target motion, the duration of the second time period is less than the duration of the second video period.

Step b2. The computer device performs image interception in the second video sequence corresponding to the second time period according to the spatial motion range and the second video to obtain a second image sequence of the second video, the second video sequence includes The multi-frame video image of the second video, and the second image sequence includes an area corresponding to the spatial motion range in the multi-frame video image.

Step b3: Use the second image sequence of the plurality of second videos as the abnormal behavior data set.

Step b1 to step b3 are the same as step a1 to step a3, and the specific process will not be repeated here.

It should be noted here that the duration of the second time period and the first time period may be equal or different.

It should be noted that, in any scenario, the probability of normal behavior is much greater than the probability of abnormal behavior, so the first video is easier to collect than the second video. It can be understood that, since the number of multiple first videos is much larger than the number of multiple second videos, the normal behavior data set acquired based on multiple first videos contains a large amount of normal behavior data, while the number based on multiple second videos The abnormal behavior data collection obtained by the video contains a small amount of abnormal behavior data.

202. Acquire multiple first behavior data pairs and multiple second behavior data pairs according to the normal behavior data set and the abnormal behavior data set, and each first behavior data pair includes two normal behavior data in the normal behavior data set , Each second behavior data pair contains one normal behavior data in the normal behavior data set and one abnormal behavior data in the abnormal behavior data set.

In the embodiment of the present disclosure, the computer device may construct a plurality of first behavior data pairs (“normal-normal” behavior data pairs) based on the normal behavior data set, and construct a plurality of second behavior data sets based on the normal behavior data set and the abnormal behavior data set Behavior data pairs ("normal-abnormal" behavior data pairs).

The specific processing is: the computer device can combine the normal behavior data in the normal behavior data set in twos to obtain multiple first behavior data pairs. For each normal behavior data in the normal behavior data set, the computer device may combine the normal behavior data with each abnormal behavior data in the abnormal behavior data set to obtain multiple second behavior data pairs.

Since the normal behavior data set contains a large amount of normal behavior data, and the abnormal behavior data set contains a small amount of abnormal behavior data, the computer device can form a large amount of "normal-normal" based on a large amount of normal behavior data and a small amount of abnormal behavior data Behavior data pairs and a large number of "normal-abnormal" behavior data pairs.

For example, the normal behavior data set containing normal behavior data is SN={n ₁ ,n ₂ ,...,n _k }, and each element n ₁ ,n ₂ ,...,n _k in the SN represents a Normal behavior data; the collection of abnormal behavior data containing abnormal behavior data is

Where k and p are positive integers, and k is much greater than p, each element in SA

Represents an abnormal behavior data. The subscript of a refers to the abnormal behavior category. The superscript is used to distinguish the abnormal behavior data under the same abnormal behavior category. Computer equipment uses SN to form "normal-normal" behavior data pair NN_Pair = {<n _i ,n _j >,i≠j}; uses SN and SA to form "normal-abnormal" behavior data pair

Among them, sizeof(a _q ) refers to the amount of abnormal behavior data of the same abnormal behavior category. The computer device trains based on NN_Pair and NA_Pair to obtain a feature extraction model. For the specific process, see

subsequent steps

203 and 204.

203. Extract a plurality of first behavior feature pairs of the plurality of first behavior data pairs and a plurality of second behavior feature pairs of the plurality of second behavior data pairs, and each first behavior feature pair includes two normal behavior data Behavior characteristics, each second behavior characteristic pair contains a behavior characteristic of normal behavior data and a behavior characteristic of abnormal behavior data.

In a possible implementation manner, the computer device may obtain the initial extraction model, and use the initial extraction model to perform behavior feature extraction on the normal behavior data in the plurality of first behavior data pairs to obtain a plurality of first behavior feature pairs (p. A behavior feature pair includes behavior features extracted from two normal behavior data). The computer device uses the initial extraction model to perform behavior feature extraction on the normal behavior data and the abnormal behavior data in the plurality of second behavior data pairs, respectively, to obtain a plurality of second behavior characteristic pairs (the second behavior characteristic pair includes one normal behavior data Behavior features extracted and behavior features extracted from an abnormal behavior data). Wherein, the initial feature extraction model has the ability to output behavior features based on the input behavior data. The initial feature extraction model can be trained by a computer device or sent to the computer device by other devices. The training process of the initial feature extraction model may include: training the convolutional neural network based on multiple sample behavior data to obtain an initial feature extraction model.

Specifically, for each first behavior data pair in the plurality of first behavior data pairs, the computer device may separately input each normal behavior data in the first behavior data pair into an initial feature extraction model, and output the first behavior data pair The behavioral characteristic pair, that is, the first behavioral characteristic pair. For each second behavior data pair in the plurality of second behavior data pairs, the computer device may input the normal behavior data and the abnormal behavior data in the second behavior data pair into the initial feature extraction model, respectively, and output the second behavior data A pair of behavioral characteristics, that is, a pair of second behavioral characteristics.

204. According to the distance between the two behavior features included in each first behavior feature pair and the distance between the two behavior features included in each second behavior feature pair, the computer device supervises training through a loss function to obtain The feature extraction model.

Among them, the feature extraction model is used to map the behavior features of normal behavior data into the feature space and map the behavior features of abnormal behavior data outside the feature space. The distance between the behavior features in the feature space is less than Distance threshold. The feature space range refers to a high-dimensional spherical space composed of high-dimensional feature vectors of all normal behavior data, so that the behavior features of normal behavior data are always inside the spherical surface. For example, if the distance threshold is selected as the diameter of the spherical surface, the distance between all the behavioral characteristics of normal behavior is less than the distance threshold, and the behavioral characteristics of the abnormal behavior data are mapped to the outside of the spherical surface. It is called the center of the feature space, that is, the distance between the normal behavior feature centers) is greater than the distance threshold.

In the embodiment of the present disclosure, for each first behavior feature pair, the computer device may use a preset distance algorithm to calculate the distance between the two behavior features contained in each first behavior feature pair to obtain each first behavior feature The corresponding distance. For each second behavior feature pair, the computer device may use a preset distance algorithm to calculate the distance between the two behavior features contained in each second behavior feature pair to obtain the distance corresponding to each second behavior feature pair. The distance includes but is not limited to Euclidean distance, cosine distance and Hamming distance.

Further, the computer device can obtain the feature extraction model through the loss function supervised training according to the calculated distance. The specific process includes: the computer device calculates the error between the distance corresponding to each first behavior feature pair and the first distance threshold In this way, for the distances corresponding to multiple first behavior feature pairs, the computer device can obtain multiple errors. The computer device calculates the error between the distance of each second behavior feature pair and the second distance threshold, so that for the distances of multiple second behavior feature pairs, multiple errors can be acquired. The first distance threshold is the distance from the first behavior feature to the expected distance, and the second distance threshold is the distance from the second behavior feature to the expected distance. The first distance threshold is less than the second distance threshold. For example, the first distance threshold may be 0 , The second distance threshold may be greater than 0.

Then, the computer device can calculate the loss through the loss function according to all the errors obtained, such as summing all the obtained errors to obtain the summation result, and the computer device returns the summation result as a supervised signal to update the initial feature extraction model Parameters to get the updated initial feature extraction model. Then, based on the first behavior feature pair and the second behavior feature pair, continue to train the updated initial feature extraction model until the loss meets the requirements, and the feature extraction model is obtained. Among them, the loss function includes but is not limited to Contrastive Loss (contrast loss) and Triplet Loss (triple loss) and other loss functions. The feature extraction model can be a 3D (three-dimensional) convolutional neural network model, including but not limited to resnet (Residual Neural Network, residual neural network) 18, resnet50, resnet101, resnet152, inception-v1 and VGG (Visual Geometry Group, visual Geometry group). The computer device uses the loss function to supervise the training method, with the constraints of reducing the corresponding distance of the first behavior feature pair and increasing the corresponding distance of the second behavior feature pair as constraints, training the initial feature extraction model, and finally obtaining the feature extraction model.

It should be noted that the above steps 202 to 204 are the training process of the feature extraction model. The steps 202 to 204 are optional steps, which are steps that need to be performed before the behavior data is detected, not every time the behavior data are detected, to ensure that training has been performed when the behavior data is detected The feature extraction model can be obtained.

Embodiments of the present disclosure may use behavior data to build behavior data pairs for specific scenarios for training. Through the distance-constrained training method based on behavioral data pairs, an end-to-end training scheme is adopted (end-to-end training refers to the training loss from the given input data to the back propagation training, without any human intervention in the middle of training), which improves The degree of automation of the system, which restricts the distance between the behavioral characteristics, can ensure that the normal behavioral characteristics are more compact, and there is a clear distance between the abnormal behavioral characteristics and the normal behavioral characteristics (abnormal behavioral characteristics are the behavioral characteristics of abnormal behavioral data, normal behavioral characteristics Behavioral characteristics of normal behavioral data). This training method based on distance constraints can adapt to the problem of a wide variety of abnormal behavior data and insufficient data in real scenes, and has the ability to detect unknown abnormal behavior.

It should be noted that, after inputting normal behavior data into the trained feature extraction model, it can output behavior features within the feature space range, and after inputting abnormal behavior data into the trained feature extraction model, it can output out of the feature space range. Behavioral characteristics. That is, if the input behavior data of the feature extraction model is normal behavior data, the output behavior feature will be within the range of the feature space, and if the input behavior data of the feature extraction model is abnormal behavior data, the output behavior The feature will be outside the feature space. Among them, the feature space range is a relatively small space range in the feature space.

205. Acquire multiple behavior data to be detected.

In the embodiment of the present disclosure, the computer device may acquire multiple behavior data based on massive videos, and use the multiple behavior data as behavior data to be detected. These videos may be collected by relevant personnel and stored on the computer device. These videos The behavior of the target in the behavior category is unknown.

In a possible implementation manner, this step 205 may include: the computer device acquiring multiple videos; for each of the multiple videos, detecting and tracking the target in the video, and acquiring the video in the third time period The spatial motion range of the target, the spatial motion range is the spatial range covered by the target motion, the duration of the third time period is less than the duration of the video time period; according to the spatial motion range and the video, at the third time Perform image interception in the video sequence corresponding to the segment to obtain the image sequence of the video. The video sequence includes multi-frame video images of the video. The image sequence includes the region corresponding to the spatial motion range in the multi-frame video image; The image sequence of each video is used as the multiple behavior data.

The process of obtaining multiple behavior data by the computer device is the same as that of obtaining the normal behavior data set and the abnormal behavior data set in step 201, and will not be repeated here. It should be noted here that the duration of the third time period and the first time period may be equal or unequal, and the duration of the third time period and the second time period may be equal or unequal.

It should be noted that this step 205 takes the behavior data to be detected as multiple behavior data as an example for description. It can be understood that, in this step 205, the computer device may also obtain only one behavior data to be detected. The disclosed embodiments do not limit this.

206. For each behavior data in the plurality of behavior data, input the behavior data into a feature extraction model, and output the behavior characteristics of the behavior data.

In the embodiment of the present disclosure, the computer device may use the feature extraction model to extract behavior characteristics of multiple behavior data. For each behavior data, if the behavior data is normal behavior data, the distance between the behavior features extracted by the feature extraction model and the behavior features of each normal behavior data is small, for example, the distance is less than or equal to the distance threshold. If the behavior data is abnormal behavior data, the distance between the behavior features extracted by the feature extraction model and the behavior features of each normal behavior data is large, for example, the distance is greater than the distance threshold.

207. The computer device obtains the detection result of the behavior data according to the distance and the distance threshold of the behavior characteristic of the behavior data and the center of the normal behavior characteristic. The detection result is used to indicate whether the behavior data is abnormal behavior data. The normal behavior characteristic center It is used to represent the behavior features within the feature space.

In the embodiment of the present disclosure, the computer device may use the normal behavior feature center to represent the behavior feature within the feature space, where the normal behavior feature may be a behavior feature extracted from a plurality of normal behavior data extracted by the feature extraction model obtained by the above training.

In a possible implementation manner, the process of acquiring the normal behavior feature center may include: acquiring multiple normal behavior data; for each normal behavior data in the multiple normal behavior data, input the normal behavior data into the feature extraction The model outputs the behavior characteristics of the normal behavior data; according to the behavior characteristics of the plurality of normal behavior data, the normal behavior characteristic center is obtained.

Among them, the plurality of normal behavior data belong to the normal behavior data set. The behavior features of the multiple normal behavior data may be multiple feature vectors, such as 128-dimensional feature vectors. The computer device may calculate an average value of each feature vector in each dimension. In this way, for each dimension, an average value is obtained. The computer device uses the average value of each dimension of the feature vector as the normal behavior feature center.

The computer device can separately calculate the distance between the behavior feature of multiple behavior data and the center of the normal behavior feature, and the distance includes but is not limited to the Euclidean distance, the cosine distance, and the Hamming distance. For each behavior data in multiple behavior data, determine the distance between the behavior feature of the behavior data and the center of the normal behavior feature and the distance threshold, when the distance between the behavior feature of the behavior data and the center of the normal behavior feature is greater than the distance At the threshold, the computer device may determine that the behavior data is abnormal behavior data, that is, the detection result of the behavior data indicates that the behavior data is abnormal behavior data. When the distance between the behavior feature of the behavior data and the center of the normal behavior feature is less than or equal to the distance threshold, the computer device may determine that the behavior data is normal behavior data, that is, the detection result of the behavior data indicates that the behavior data is normal Behavioral data.

It should be noted that the embodiments of the present disclosure take the computer device to obtain a normal behavior feature center, and use the normal behavior feature center to represent the behavior characteristics of all normal behavior data as an example for description. Of course, the computer device can also obtain multiple normal behavior feature centers, and each normal behavior feature center is used to represent the behavior characteristics of one or more types of normal behavior data, so that for each behavior data in the multiple behavior data, the computer device The distance between the behavior feature of the behavior data and the centers of the plurality of normal behavior features can be calculated separately. Then, the computer device adopts a preset judgment algorithm to judge whether the behavior data is abnormal behavior data, thereby obtaining a detection result of the behavior data. Among them, the preset judgment algorithm includes but is not limited to KNN (K-Nearest Neighbor, nearest neighbor algorithm) algorithm and clustering algorithm.

208. Determine abnormal behavior data among the plurality of behavior data according to respective detection results of the plurality of behavior data.

In the embodiment of the present disclosure, the computer device may determine behavior data whose detection result is abnormal behavior data among the plurality of behavior data. Optionally, when the amount of the multiple behavior data is relatively large, by testing the multiple behavior data and combining with simple manual confirmation, it is possible to collect a batch of abnormal behavior data, and the collected abnormal behavior may be used Data to expand the existing abnormal behavior data collection.

In a possible implementation manner, for the abnormal behavior data in the plurality of behavior data, during playback of the video to which the abnormal behavior data belongs, the computer device may display the image sequence of the video to which the abnormal behavior data belongs.

The computer device can highlight the image sequence in the video (the image sequence is the image sequence of the video to which the abnormal behavior data belongs), and the specific display method includes but is not limited to adding a rectangular frame to the spatial motion range of the target, that is, including the image sequence The area is marked with a rectangular frame.

The image sequence is a spatiotemporal cubic image sequence, which may also be called Tube/Tubelet(), and the images it contains can reflect the motion information of the target in time and space. Display the abnormal behavior detection results by highlighting the sequence information of the space-time cube in the original video. While displaying the abnormal behavior detection results, you can also display the abnormal behavior alarm results, such as in the area corresponding to the added rectangular frame It displays "abnormal" and "alarm" text prompts.

By displaying the abnormal behavior detection result, the user can know the start time, end time and spatial position of the abnormal behavior. When the image sequence of any video is abnormal behavior data, the area included in the image sequence in the video can be For display, the start time and end time of the abnormal behavior are the start time and end time of the image sequence, that is, the first frame video image and the last frame video of the multi-frame video images corresponding to the image sequence The time the image is in the entire target video. The spatial position where the abnormal behavior occurs is the three-dimensional spatial position indicated by the area contained in the image sequence. This adopts the form of space-time cube to record and display, which can facilitate users to view and manage. By displaying the alarm result, it is convenient for the user to confirm. Since the image sequence of the entire video is recorded, the user can also view other abnormal behavior detection results near the alarm time for a more comprehensive correlation.

The embodiments of the present disclosure use behavioral space-time cube structure for behavior testing and analysis and display. The behavioral space-time cube analysis method can effectively use the information of the target behavior, remove a large amount of background irrelevant information, and can alleviate the problem that the target ratio is too small. The stable recognition performance also greatly reduces the space resource consumption of the system. The real-time anomaly detection result display method adopted by the embodiment of the present disclosure can highlight the abnormal behavior that occurs in the video stream for a long time, can intuitively observe the abnormal behavior and alarm, and improve the level of intelligence.

209. Add abnormal behavior data from the plurality of behavior data to the abnormal behavior data set.

In the embodiment of the present disclosure, after determining abnormal behavior data in the plurality of behavior data through steps 206 to 208, the computer device may use the abnormal behavior data in the plurality of behavior data to expand the abnormal behavior data set.

Steps 205 to 208 are a process of acquiring multiple behavior data and using the feature extraction model to automatically determine abnormal behavior data among the multiple behavior data. In order to ensure accuracy, the detection result may be further confirmed manually, and if manually confirmed as abnormal behavior data, the computer device may perform this step 209. Correspondingly, in a possible implementation manner, the computer device can obtain manual confirmation information of abnormal behavior data among the plurality of behavior data (the manual confirmation information is used to indicate whether it is abnormal behavior data); the abnormality indicated by the manual confirmation information Behavior data is added to the abnormal behavior data collection.

For example, the computer device can display the detection result of the abnormal behavior data, and the human can input the confirmation information through a manual confirmation information input interface. The computer device obtains the manual confirmation information of the detection result, and if the manual confirmation information indicates a certain abnormal behavior data, it is added to the abnormal behavior data set, thereby expanding the abnormal behavior data set.

It should be noted that the embodiment of the present disclosure takes the expansion of the abnormal behavior data set as an example for description. Optionally, the computer device may also expand the normal behavior data set. For example, in step 208, in addition to determining abnormal behavior data in the plurality of behavior data, the computer device may also determine normal behavior data in the plurality of behavior data, and the computer device may add it to the normal behavior data set to achieve normal Expansion of behavioral data collection.

Of course, when the amount of behavior data in the normal behavior data set is relatively small (this case may be a case where the normal behavior data that is pre-calibrated is relatively small), the computing device may add the normal behavior data from the multiple behavior data to the normal behavior data In the collection, the normal behavior data collection is also expanded. Of course, when the computer device adds the normal behavior data set determined in step 208 to the normal behavior data set, it may add the behavior data manually confirmed as normal behavior data to the normal behavior data set.

210. Perform steps 202 to 204 to obtain an updated feature extraction model.

In the embodiment of the present disclosure, after the computer device expands the abnormal behavior data through steps 205 to 209, it can perform steps 202 to 204 (the feature extraction model training process) again to obtain an updated feature extraction model.

The computer device may form a new second behavior data pair according to the newly added abnormal behavior data in the normal behavior data set and the abnormal behavior data set (that is, the abnormal behavior data added in step 209). For example, for each newly added abnormal behavior data, the abnormal behavior data may be combined with each normal behavior data in the normal behavior data set to obtain a new second behavior data pair. The computer device may retain the original multiple first behavior data pairs and the multiple second behavior data pairs unchanged, and obtain new second behavior data pairs, thereby achieving the purpose of expanding the "normal-abnormal" behavior data pairs.

In addition, the computer device may determine a new first behavior data pair and a new second behavior data pair based on the normal behavior data in the updated normal behavior data set and the abnormal behavior data in the updated abnormal behavior data set. The computer device can keep the original multiple first behavior data pairs and multiple second behavior data pairs unchanged, and acquire new first behavior data pairs and new second behavior data pairs, thereby achieving expansion of "normal-normal" The purpose of behavior data pairs and "normal-abnormal" behavior data pairs.

It should be noted that the above steps 208 to 210 are optional steps. By detecting a large amount of behavior data and manually confirming the behavior data whose detection result is abnormal behavior data, the abnormal behavior data set after manual confirmation can be used to expand the abnormal behavior data set. After the expansion is completed, the above training feature extraction model is executed again. The process gets an updated feature extraction model. Then, the computer device can use the updated feature extraction model to perform abnormal behavior detection on any video. Since the updated feature extraction model learns more behavior features, it can obtain higher detection performance when detecting behavior data. More abundant abnormal behaviors are detected.

It should be noted that steps 205 to 210 are processes of acquiring multiple behavior data as test data, collecting abnormal behavior data, updating the training data set, and updating the feature extraction model based on updating the training data set. This process can be executed cyclically, and the abnormal behavior data set (or the normal behavior data set and the abnormal behavior data set) can be updated for each execution, to obtain an updated feature extraction model, and to obtain better abnormal behavior detection performance.

The feature extraction model provided by an embodiment of the present disclosure is an end-to-end deep learning model, and its acquisition process may be divided into a training phase (steps 201 to 204 above), a deployment phase (steps 205 to step 207 above), and a feedback update phase (Steps 208 to 210 above).

Referring to FIG. 3, a training flowchart of a feature extraction model is provided. As shown in FIG. 3, according to behavior data (or behavior sequence), a pair of “normal-normal” behavior data and “normal-abnormal” behavior data are formed. Yes, a feature extraction model is trained. Referring to FIG. 4, a flowchart of abnormal behavior detection is provided. As shown in FIG. 4, behavior characteristic extraction is performed on the test behavior data (any of the plurality of behavior data in step 205 ). Distance, according to the distance to determine abnormal behavior, and then collect abnormal behavior data. Referring to FIG. 5, a feedback update flowchart for abnormal behavior detection is provided. As shown in FIG. 5, an initial training data set (a plurality of first behavior data pairs and a plurality of second behavior data pairs in step 202) is used to execute the diagram The model training process in the training phase shown in 3 uses massive test data (a number of behavior data in step 205) to perform the abnormal behavior detection process in the deployment phase shown in FIG. 4, and then updates the training data set according to the collected abnormal behavior data. Then execute the model training process shown in FIG. 3 to obtain an updated feature extraction model, and use the updated feature extraction model to execute the abnormal behavior detection process shown in FIG. 4 on the new test data to obtain more accurate detection performance.

According to the normal behavior data and abnormal behavior data, the behavior data pair is formed, and the feature extraction model is trained based on the behavior data pair, and then the feature extraction model can be used to collect more abnormal behavior data, expand the "normal-abnormal" behavior data pair, update The feature extraction model. The above technical solution can train a feature extraction model based on a large amount of normal behavior data and a small amount of abnormal behavior data, and then detect a large amount of behavior data to collect more abnormal behavior data to solve the lack of abnormal behavior in real scenes For data problems, the feature extraction model based on more abnormal behavior data has better abnormal behavior detection performance.

The method provided by the embodiment of the present disclosure extracts the behavior characteristics of the behavior data through the feature extraction model, and determines whether the behavior data is abnormal behavior data according to the distance and the distance threshold between the extracted behavior characteristics and the normal behavior center, because the feature extraction model is based on The distance constraint method is trained. The behavior features extracted by the normal behavior data through the feature extraction model are in a relatively small feature space range, and the behavior features extracted by the abnormal behavior data through the feature extraction model are outside the feature space range, which ensures that The normal behavior feature is relatively compact, and there is a clear distance between the abnormal behavior feature and the normal behavior feature. Since the difference between the normal behavior and the abnormal behavior is learned, the detection method of the abnormal behavior based on the distance measurement has high accuracy.

6 is a schematic structural diagram of an abnormal behavior detection device provided by an embodiment of the present disclosure. 6, the device includes:

The obtaining module 601 is used to obtain behavior data to be detected;

An extraction module 602 is used to input the behavior data into a feature extraction model and output behavior characteristics of the behavior data. The feature extraction model is used to output behavior characteristics within a characteristic space range according to normal behavior data and output the characteristic space according to abnormal behavior data Behaviour features outside the range, the distance between each behavior feature within the feature space is less than the distance threshold;

The obtaining module 601 is further used to obtain the detection result of the behavior data according to the distance between the behavior feature of the behavior data and the center of the normal behavior feature and the distance threshold, and the detection result is used to indicate whether the behavior data is abnormal behavior data. The normal behavior feature center is used to represent the behavior features within the feature space.

In a possible implementation manner, the obtaining module 601 is further used to:

In a possible implementation manner, the obtaining module 601 is used to:

The obtaining module 601 is further configured to determine abnormal behavior data in the plurality of behavior data according to the detection results of the plurality of behavior data; add the abnormal behavior data in the plurality of behavior data to the abnormal behavior data set; Based on the updated abnormal behavior data set, the training process of the feature extraction model is performed to obtain the updated feature extraction model.

The acquisition module 601 is also used to:

In a possible implementation manner, the obtaining module 601 is configured to obtain artificial confirmation information of abnormal behavior data among the plurality of behavior data; add abnormal behavior data indicated by the manual confirmation information to the abnormal behavior data set.

Get multiple videos;

In a possible implementation manner, referring to FIG. 7, the device further includes:

The display module 603 is configured to display the image sequence of the video to which the abnormal behavior data belongs during the process of playing the video to which the abnormal behavior data belongs.

In a possible implementation manner, the obtaining module 601 is used to:

Obtain multiple normal behavior data;

The acquisition module 601 is used to:

Use the target feature vector as the normal behavior feature center.

In the embodiment of the present disclosure, the behavior feature of the behavior data is extracted through the feature extraction model, and whether the behavior data is abnormal behavior data is determined according to the distance and the distance threshold between the extracted behavior feature and the normal behavior center, because the feature extraction model is based on the distance constraint The method is trained, the behavior features extracted by the normal behavior data through the feature extraction model are in a relatively small feature space, and the behavior features extracted by the abnormal behavior data through the feature extraction model are outside the feature space, which ensures normal behavior. The features are relatively compact, and there is a clear distance between the abnormal behavior features and the normal behavior features. Since the difference between the normal behavior and the abnormal behavior is learned, this method of detecting abnormal behavior based on the distance measurement has high accuracy.

It should be noted that the abnormal behavior detection device provided in the above embodiment only uses the division of the above functional modules as an example for the detection of abnormal behavior. In practical applications, the above functions can be allocated by different functional modules according to needs. That is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the abnormal behavior detection device and the abnormal behavior detection method embodiment provided in the above embodiments belong to the same concept. For the specific implementation process, see the method embodiment, and details are not described here.

8 is a schematic structural diagram of a computer device 800 according to an embodiment of the present disclosure. The computer device 800 may have a relatively large difference due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU). 801 and one or more memories 802, where at least one instruction is stored in the memory 802, and the at least one instruction is loaded and executed by the processor 801 to implement the abnormal behavior detection method provided by the foregoing method embodiments. Of course, the computer device 800 may also have components such as a wired or wireless network interface, a keyboard, and an input-output interface for input and output. The computer device 800 may also include other components for implementing device functions, which will not be repeated here.

In an exemplary embodiment, there is also provided a computer-readable storage medium storing at least one instruction, for example, a memory storing at least one instruction, where the at least one instruction is executed by a processor to implement the abnormal behavior in the above embodiment Detection method. For example, the computer-readable storage medium may be ROM (Read-Only Memory), RAM (Random Access Memory), CD-ROM (Compact Disc Read-Only Memory, CD-ROM), Magnetic tapes, floppy disks, optical data storage devices, etc.

A person of ordinary skill in the art may understand that all or part of the steps for implementing the above embodiments may be completed by hardware, or may be completed by a program instructing related hardware. The above program may be stored in a computer-readable storage medium. The storage medium can be read-only memory, magnetic disk or optical disk, etc.

The above are only preferred embodiments of the present disclosure and are not intended to limit the present disclosure. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure Inside.

Claims

An abnormal behavior detection method, characterized in that the method includes:

Obtain the behavior data to be detected;

Inputting the behavior data into a feature extraction model and outputting behavior characteristics of the behavior data, the feature extraction model is used to output behavior characteristics within the feature space range according to normal behavior data and out of the feature space range according to abnormal behavior data Behavioral characteristics, the distance between the behavioral characteristics within the feature space is less than the distance threshold;

Acquiring the detection result of the behavior data according to the distance between the behavior feature of the behavior data and the center of the normal behavior feature and the distance threshold, the detection result is used to indicate whether the behavior data is abnormal behavior data, the normal The behavior feature center is used to represent behavior features within the range of the feature space.
The method according to claim 1, wherein the training process of the feature extraction model comprises:

Acquiring a plurality of first behavior data pairs and a plurality of second behavior data pairs according to the normal behavior data set and the abnormal behavior data set, each first behavior data pair containing two normal behavior data in the normal behavior data set, Each second behavior data pair includes one normal behavior data in the normal behavior data set and one abnormal behavior data in the abnormal behavior data set;

Extracting a plurality of first behavior feature pairs of the plurality of first behavior data pairs and a plurality of second behavior feature pairs of the plurality of second behavior data pairs, each first behavior feature pair containing two normal behavior data Behavior characteristics, each second behavior characteristic pair contains a behavior characteristic of normal behavior data and a behavior characteristic of abnormal behavior data;

According to the distance between the two behavioral features included in each of the first behavioral feature pairs and the distance between the two behavioral features included in each of the second behavioral feature pairs, supervise training through a loss function to obtain the Feature extraction model.
The method according to claim 2, wherein before acquiring a plurality of first behavior data pairs and a plurality of second behavior data pairs according to the normal behavior data set and the abnormal behavior data set, the method further comprises:

Acquiring the normal behavior data set based on a plurality of first videos, the plurality of first videos being videos targeted for normal behavior;

The abnormal behavior data set is acquired based on multiple second videos, and the multiple second videos are videos of the target performing abnormal behavior.
The method according to claim 3, wherein the acquiring the normal behavior data set based on the plurality of first videos includes:

For each first video in the plurality of first videos, detect and track the target in the first video to obtain the spatial motion range of the target in the first time period, the spatial motion range is The spatial range covered by the target motion, the duration of the first time period is less than the duration of the first video period;

According to the spatial motion range and the first video, performing image interception in a first video sequence corresponding to the first time period to obtain a first image sequence of the first video, the first video sequence including A multi-frame video image of the first video, the first image sequence includes an area corresponding to the spatial motion range in the multi-frame video image;

The first image sequence of the plurality of first videos is used as the normal behavior data set.
The method according to claim 3, wherein the acquiring the abnormal behavior data set based on the plurality of second videos includes:

For each second video in the plurality of second videos, detect and track the target in the second video to obtain the spatial motion range of the target in the second time period, the spatial motion range is The spatial range covered by the target motion, the duration of the second time period is less than the duration of the second video period;

According to the spatial motion range and the second video, performing image interception in a second video sequence corresponding to the second time period to obtain a second image sequence of the second video, the second video sequence including A multi-frame video image of the second video, the second image sequence includes an area corresponding to the spatial motion range in the multi-frame video image;

The second image sequence of the plurality of second videos is used as the abnormal behavior data set.
The method according to claim 2, wherein the behavior data to be detected is a plurality of behavior data,

After obtaining the detection result of the behavior data according to the distance between the behavior characteristic of the behavior data and the center of the normal behavior characteristic, the method further includes:

Determine abnormal behavior data among the plurality of behavior data according to respective detection results of the plurality of behavior data;

Adding abnormal behavior data from the plurality of behavior data to the abnormal behavior data set;

Based on the updated abnormal behavior data set, the training process of the feature extraction model is performed to obtain the updated feature extraction model.
The method according to claim 6, wherein the behavior data to be detected is a plurality of behavior data;

After obtaining the detection result of the behavior data according to the distance between the behavior characteristic of the behavior data and the center of the normal behavior characteristic, the method further includes:

Determine the normal behavior data among the plurality of behavior data according to the respective detection results of the plurality of behavior data;

Adding normal behavior data from the plurality of behavior data to the normal behavior data set;

The updating based on the abnormal behavior data set, performing the training process of the feature extraction model, and obtaining the updated feature extraction model includes:

Based on the updated abnormal behavior data set and the updated normal behavior data set, the training process of the feature extraction model is performed to obtain the updated feature extraction model.
The method according to claim 6, wherein the method further comprises:

Get multiple videos;

For each of the plurality of videos, detect and track the target in the video to obtain the spatial motion range of the target in the third time period, the spatial motion range is covered by the target motion The spatial range of the third time period is less than the time period of the video;

According to the spatial motion range and the video, performing image interception in the video sequence corresponding to the third time period to obtain an image sequence of the video, the video sequence including multiple frames of video images of the video, so The image sequence includes an area corresponding to the spatial motion range in the multi-frame video image;

The image sequence of the plurality of videos is used as the plurality of behavior data.
The method according to claim 1, wherein the process of acquiring the normal behavior feature center includes:

Obtain multiple normal behavior data;

For each normal behavior data in the plurality of normal behavior data, input the normal behavior data into the feature extraction model, and output the behavior characteristics of the normal behavior data;

Obtain the normal behavior characteristic center according to the behavior characteristic of the plurality of normal behavior data.
The method according to claim 9, wherein the behavior characteristic of each normal behavior data in the plurality of normal behavior data is characterized by a feature vector;

The obtaining the normal behavior characteristic center according to the behavior characteristics of the plurality of normal behavior data includes:

Calculating an average value of the feature vectors of the plurality of normal behavior data in each dimension to obtain a target feature vector characterized by a set of average values consisting of the average value of each dimension;

Use the target feature vector as the normal behavior feature center.
An abnormal behavior detection device, characterized in that the device includes:

The acquisition module is used to acquire the behavior data to be detected;

An extraction module is used to input the behavior data into a feature extraction model and output behavior characteristics of the behavior data, and the feature extraction model is used to output behavior characteristics within a feature space range according to normal behavior data and output data based on abnormal behavior data. The behavior features outside the feature space, the distance between the behavior features in the feature space is less than the distance threshold;

The acquiring module is further configured to acquire the detection result of the behavior data according to the distance between the behavior feature of the behavior data and the center of the normal behavior feature and the distance threshold, and the detection result is used to indicate whether the behavior data is For abnormal behavior data, the normal behavior feature center is used to represent behavior features within the feature space.
The apparatus according to claim 11, wherein the acquisition module is further configured to:

Acquiring a plurality of first behavior data pairs and a plurality of second behavior data pairs according to the normal behavior data set and the abnormal behavior data set, each first behavior data pair containing two normal behavior data in the normal behavior data set, Each second behavior data pair includes one normal behavior data in the normal behavior data set and one abnormal behavior data in the abnormal behavior data set;

Extracting a plurality of first behavior feature pairs of the plurality of first behavior data pairs and a plurality of second behavior feature pairs of the plurality of second behavior data pairs, each first behavior feature pair containing two normal behavior data Behavior characteristics, each second behavior characteristic pair contains a behavior characteristic of normal behavior data and a behavior characteristic of abnormal behavior data;

According to the distance between the two behavior features included in each first behavior feature pair and the distance between the two behavior features included in each second behavior feature pair, through the loss function supervised training, the Feature extraction model.
The apparatus according to claim 12, wherein the acquisition module is further configured to:

Acquiring the normal behavior data set based on a plurality of first videos, the plurality of first videos being videos of normal behavior targeted by the target;

The abnormal behavior data set is acquired based on multiple second videos, and the multiple second videos are videos of the target performing abnormal behavior.
The apparatus according to claim 13, wherein the acquisition module is configured to:

For each first video in the plurality of first videos, detect and track the target in the first video to obtain the spatial motion range of the target in the first time period, the spatial motion range is The spatial range covered by the target motion, the duration of the first time period is less than the duration of the first video period;

According to the spatial motion range and the first video, performing image interception in a first video sequence corresponding to the first time period to obtain a first image sequence of the first video, the first video sequence including A multi-frame video image of the first video, the first image sequence includes an area corresponding to the spatial motion range in the multi-frame video image;

The first image sequence of the plurality of first videos is used as the normal behavior data set.
The apparatus according to claim 13, wherein the acquisition module is configured to:

For each second video in the plurality of second videos, detect and track the target in the second video to obtain the spatial motion range of the target in the second time period, the spatial motion range is The spatial range covered by the target motion, the duration of the second time period is less than the duration of the second video period;

According to the spatial motion range and the second video, performing image interception in a second video sequence corresponding to the second time period to obtain a second image sequence of the second video, the second video sequence including A multi-frame video image of the second video, the second image sequence includes an area corresponding to the spatial motion range in the multi-frame video image;

The second image sequence of the plurality of second videos is used as the abnormal behavior data set.
The device according to claim 12, wherein the behavior data to be detected is a plurality of behavior data;

The acquiring module is further configured to determine abnormal behavior data in the plurality of behavior data according to the detection results of the plurality of behavior data; add the abnormal behavior data in the plurality of behavior data to the abnormal behavior In the data set; based on the updated abnormal behavior data set, the training process of the feature extraction model is performed to obtain the updated feature extraction model.
The apparatus according to claim 16, wherein the acquisition module is further configured to:

Get multiple videos;

For each of the plurality of videos, detect and track the target in the video to obtain the spatial motion range of the target in the third time period, the spatial motion range is covered by the target motion The spatial range of the third time period is less than the time period of the video;

According to the spatial motion range and the video, performing image interception in the video sequence corresponding to the third time period to obtain an image sequence of the video, the video sequence including multiple frames of video images of the video, so The image sequence includes an area corresponding to the spatial motion range in the multi-frame video image;

The image sequence of the plurality of videos is used as the plurality of behavior data.
The apparatus according to claim 11, wherein the acquisition module is further configured to:

Obtain multiple normal behavior data;

For each normal behavior data in the plurality of normal behavior data, input the normal behavior data into the feature extraction model, and output the behavior characteristics of the normal behavior data;

Obtain the normal behavior characteristic center according to the behavior characteristic of the plurality of normal behavior data.
The apparatus according to claim 18, wherein the behavior characteristic of each normal behavior data in the plurality of normal behavior data is characterized by a feature vector;

The acquisition module is used to:

Calculating an average value of the feature vectors of the plurality of normal behavior data in each dimension to obtain a target feature vector characterized by a set of average values consisting of the average value of each dimension;

Use the target feature vector as the normal behavior feature center.
A computer device, comprising a processor and a memory; the memory is used to store at least one instruction; the processor executes at least one instruction stored on the memory to implement:

Obtain the behavior data to be detected;

Inputting the behavior data into a feature extraction model and outputting behavior characteristics of the behavior data, the feature extraction model is used to output behavior characteristics within the feature space range according to normal behavior data and out of the feature space range according to abnormal behavior data Behavioral characteristics, the distance between the behavioral characteristics within the feature space is less than the distance threshold;

Acquiring the detection result of the behavior data according to the distance between the behavior feature of the behavior data and the center of the normal behavior feature and the distance threshold, the detection result is used to indicate whether the behavior data is abnormal behavior data, the normal The behavior feature center is used to represent behavior features within the range of the feature space.
The computer device according to claim 20, wherein the processor executes at least one instruction stored on the memory, and is further used to implement:

Acquiring a plurality of first behavior data pairs and a plurality of second behavior data pairs according to the normal behavior data set and the abnormal behavior data set, each first behavior data pair containing two normal behavior data in the normal behavior data set, Each second behavior data pair includes one normal behavior data in the normal behavior data set and one abnormal behavior data in the abnormal behavior data set;

Extracting a plurality of first behavior feature pairs of the plurality of first behavior data pairs and a plurality of second behavior feature pairs of the plurality of second behavior data pairs, each first behavior feature pair containing two normal behavior data Behavior characteristics, each second behavior characteristic pair contains a behavior characteristic of normal behavior data and a behavior characteristic of abnormal behavior data;

According to the distance between the two behavioral features included in each of the first behavioral feature pairs and the distance between the two behavioral features included in each of the second behavioral feature pairs, supervise training through a loss function to obtain the Feature extraction model.
The computer device according to claim 21, wherein the processor executes at least one instruction stored on the memory, and is further used to implement:

Acquiring the normal behavior data set based on a plurality of first videos, the plurality of first videos being videos of normal behavior targeted by the target;

The abnormal behavior data set is acquired based on multiple second videos, and the multiple second videos are videos of the target performing abnormal behavior.
The computer device according to claim 22, wherein the processor executes at least one instruction stored on the memory, and is further used to implement:

For each first video in the plurality of first videos, detect and track the target in the first video to obtain the spatial motion range of the target in the first time period, the spatial motion range is The spatial range covered by the target motion, the duration of the first time period is less than the duration of the first video period;

According to the spatial motion range and the first video, performing image interception in a first video sequence corresponding to the first time period to obtain a first image sequence of the first video, the first video sequence including A multi-frame video image of the first video, the first image sequence includes an area corresponding to the spatial motion range in the multi-frame video image;

The first image sequence of the plurality of first videos is used as the normal behavior data set.
The computer device according to claim 22, wherein the processor executes at least one instruction stored on the memory, and is further used to implement:

For each second video of the plurality of second videos, detect and track the target in the second video to obtain the spatial motion range of the target in the second time period, the spatial motion range is The spatial range covered by the target motion, the duration of the second time period is less than the duration of the second video period;

According to the spatial motion range and the second video, performing image interception in a second video sequence corresponding to the second time period to obtain a second image sequence of the second video, the second video sequence including A multi-frame video image of the second video, the second image sequence includes an area corresponding to the spatial motion range in the multi-frame video image;

The second image sequence of the plurality of second videos is used as the abnormal behavior data set.
The computer device according to claim 21, wherein the behavior data to be detected is a plurality of behavior data;

The processor executes at least one instruction stored on the memory, and is also used to implement:

Determine the abnormal behavior data in the plurality of behavior data according to the detection results of the plurality of behavior data; add the abnormal behavior data in the plurality of behavior data to the abnormal behavior data set; based on the updated The abnormal behavior data set performs the training process of the feature extraction model to obtain an updated feature extraction model.
The computer device according to claim 25, wherein the processor executes at least one instruction stored on the memory, and is further used to implement:

Get multiple videos;

For each of the plurality of videos, detect and track the target in the video to obtain the spatial motion range of the target in the third time period, the spatial motion range is covered by the target motion The spatial range of the third time period is less than the time period of the video;

According to the spatial motion range and the video, performing image interception in the video sequence corresponding to the third time period to obtain an image sequence of the video, the video sequence including multiple frames of video images of the video, so The image sequence includes an area corresponding to the spatial motion range in the multi-frame video image;

The image sequence of the plurality of videos is used as the plurality of behavior data.
The computer device according to claim 20, wherein the processor executes at least one instruction stored on the memory, and is further used to implement:

Obtain multiple normal behavior data;

For each normal behavior data in the plurality of normal behavior data, input the normal behavior data into the feature extraction model, and output the behavior characteristics of the normal behavior data;

Obtain the normal behavior characteristic center according to the behavior characteristic of the plurality of normal behavior data.
The computer device according to claim 27, wherein the behavior characteristic of each normal behavior data in the plurality of normal behavior data is characterized by a feature vector;

The processor executes at least one instruction stored on the memory to implement:

Calculating an average value of the feature vectors of the plurality of normal behavior data in each dimension to obtain a target feature vector characterized by a set of average values consisting of the average value of each dimension;

Use the target feature vector as the normal behavior feature center.
A computer-readable storage medium, characterized in that a computer program is stored in the storage medium, and when the computer program is executed by a processor, the method steps of any one of claims 1-10 are realized.