KR102309111B1 - Ststem and method for detecting abnomalous behavior based deep learning - Google Patents

Abstract

An objective of the present invention is to provide a deep learning-based abnormal behavior detection system and method by detecting and recognizing an abnormal behavior, which can detect and recognize a person who truly acts an abnormal behavior and a person who falsely acts an abnormal behavior among people who act abnormally when analyzing behaviors of people and detecting a normal behavior and an abnormal behavior from the behaviors of people. In order to achieve the above objective, according to the present invention, the deep learning-based abnormal behavior detection system for detecting and recognizing an abnormal behavior includes a first step of performing a convolution calculation on image data provided by a photographing unit and analyzing the behavior of pedestrians.

Description

Abnormal behavior detection system and detection method for detecting and recognizing abnormal behavior based on deep learning

The present invention relates to an abnormal behavior detection system and detection method for detecting and recognizing deep learning-based abnormal behavior, and more particularly, to detecting and recognizing abnormal behavior based on deep learning that detects and recognizes abnormal behavior of a moving human body. It relates to a behavior detection system and detection method.

In the case of a large-scale CCTV system, management personnel have to monitor dozens to hundreds of images, so it takes a lot of people, but often misses important situations due to reduced concentration, fatigue, or arbitrary judgment of monitoring personnel. , there are many difficulties in the problem of storing a huge amount of time.

Therefore, in order to monitor public places, public places are filmed with CCTV cameras, and images are automatically analyzed to extract a large number of unspecified people, and by analyzing behaviors, when abnormal behavior is detected, it automatically warns the manager or other related automation. The demand for an intelligent video surveillance system that delivers information to the system is growing.

Conventional intelligent video surveillance systems first extract individual humans from an image, track the extracted individual humans to obtain a trajectory, and analyze the trajectory to estimate individual human behavior.

Then, it is analyzed whether the estimated behavior of the human is normal or not.

These conventional systems require a lot of calculations and errors while going through the steps of human extraction, tracking, trajectory analysis, and behavior estimation.

For example, when monitoring a place where crowds are concentrated, such as a square, a park, a train station, or a playground, it is not easy to extract and analyze objects using conventional intelligent video surveillance systems.

In particular, when detecting normal behavior and abnormal behavior from human behavior, there is a problem in that it is not easy to detect and recognize genuinely abnormal people and falsely abnormal behaviors among people who perform abnormal behaviors.

Domestic Registered Patent Publication No. 10-1472674

An object of the present invention to solve the problems of the prior art as described above is to analyze human behavior to detect normal behavior and abnormal behavior from human behavior. To provide an abnormal behavior detection system and detection method that can detect and recognize abnormal behavior based on deep learning that can detect and recognize humans performing abnormal behavior with

In order to achieve the above object, the abnormal behavior detection system for detecting and recognizing abnormal behavior based on deep learning according to the present invention performs a convolution calculation on the image data provided by the photographing unit to determine the behavior of pedestrians. A first step of analyzing; characterized in that it includes.

In order to achieve the above object, the abnormal behavior detection method for detecting and recognizing abnormal behavior based on deep learning according to the present invention performs a convolution calculation on the image data provided by the photographing unit to determine the behavior of pedestrians. a first step of analyzing; and a second step of analyzing the pedestrian's behavior in detail through an Attention Mechanism Model.

In addition, in the deep learning-based abnormal behavior detection method for detecting and recognizing abnormal behavior according to the present invention, the convolution operation is performed by a dense convolutional network, and the image data has a 224 × 22 image size. The RGB 3 channel format of is input to the dense convolutional neural network through feature block processing.

In addition, in the abnormal behavior detection method for detecting and recognizing the deep learning-based abnormal behavior according to the present invention, when each of the image data is input to the dense convolutional neural network, the dense convolutional neural network receives each of the input image data It is characterized in that it is input to the transform block and continues the learning process.

In addition, in the abnormal behavior detection method for detecting and recognizing a deep learning-based abnormal behavior according to the present invention, the feature block and the transform block are a convolution layer and a pooling layer. .

In addition, in the deep learning-based abnormal behavior detection method for detecting and recognizing abnormal behavior according to the present invention, the attention mechanism model includes a channel attention model and a special attention model (Spatial Attention Model), and the The attention mechanism model is characterized in that the attention conducts learning to ignore irrelevant information and focus on key information.

In addition, in the deep learning-based abnormal behavior detection method for detecting and recognizing abnormal behavior according to the present invention, a third step of analyzing whether the pedestrian behavior is normal in the image data through the channel attention model as a time period; characterized by including.

In addition, in the deep learning-based abnormal behavior detection method for detecting and recognizing abnormal behavior according to the present invention, the channel attention model samples a picture of a point requiring attention in the image, and a behavior image of a pedestrian from the sampled picture characterized by processing.

In addition, in the abnormal behavior detection method for detecting and recognizing the deep learning-based abnormal behavior according to the present invention, the channel attention model is characterized in that it has continuity between specific channels.

In addition, in the deep learning-based abnormal behavior detection method for detecting and recognizing abnormal behavior according to the present invention, the special dimension of the input feature map is compressed using a combination of the maximum pooling layer and the average pooling layer to increase the efficiency of channel attention. characterized in that

In addition, in the deep learning-based abnormal behavior detection method for detecting and recognizing abnormal behavior according to the present invention, a hidden layer of a multi-layer perceptron is added after two pooling layers to reduce parameter overhead. .

In addition, in the abnormal behavior detection method for detecting and recognizing the deep learning-based abnormal behavior according to the present invention, two descriptors are characterized in that the channel attention map transmitted to the hidden layer is generated.

In addition, in the deep learning-based abnormal behavior detection method for detecting and recognizing abnormal behavior according to the present invention, a fourth step of analyzing whether the behavior of the pedestrian is normal through the channel attention model related to the special attention model; includes; characterized in that

In addition, in the abnormal behavior detection method for detecting and recognizing the deep learning-based abnormal behavior according to the present invention, the special attention model is characterized in that the analysis is concentrated on the location.

In addition, in the abnormal behavior detection method for detecting and recognizing abnormal behavior based on deep learning according to the present invention, the behavior of pedestrians is divided into normal behavior and abnormal behavior through semantic analysis, and the abnormal behavior is classified into unintentional normal behavior and intentional abnormal behavior. A fifth step of analyzing as; characterized in that it includes.

In addition, in the deep learning-based abnormal behavior detection method for detecting and recognizing abnormal behavior according to the present invention, before the first step, a first pre-processing step of collecting image data of a pedestrian in the corresponding area by a photographing unit; a second pre-processing step of adjusting the resolution of the image data; and a third pre-processing step of transmitting the image data to a cloud server.

In order to achieve the above object, an abnormal behavior detection system for detecting and recognizing a deep learning-based abnormal behavior according to the present invention comprises: a receiver for receiving input image data; a calculation unit for performing a convolution operation on the image data to analyze a pedestrian's behavior; an attention mechanism model unit for detailed analysis of the pedestrian's behavior; and a semantic analysis unit that classifies the pedestrian's behavior into normal behavior and abnormal behavior through semantic analysis, and analyzes the abnormal behavior into unintentional normal behavior and intentional abnormal behavior; characterized in that it includes a clouding server comprising a.

In addition, an abnormal behavior detection system for detecting and recognizing a deep learning-based abnormal behavior according to the present invention includes: a camera unit for collecting image data of pedestrians in a corresponding area; a resolution adjusting unit for adjusting the resolution of the image data; and a communication unit that transmits the image data to a cloud server.

Specific details of other embodiments are included in "Details for carrying out the invention" and the accompanying "drawings".

Advantages and/or features of the present invention, and methods for achieving them, will become apparent with reference to the various embodiments described below in detail in conjunction with the accompanying drawings.

However, the present invention is not limited to the configuration of each embodiment disclosed below, but may also be implemented in a variety of different forms, and each embodiment disclosed herein only makes the disclosure of the present invention complete, It is provided to fully inform those of ordinary skill in the art to which the scope of the present invention belongs, and it should be understood that the present invention is only defined by the scope of each of the claims.

According to the present invention, when human behavior is analyzed to detect normal behavior and abnormal behavior from human behavior, it is possible to detect and recognize genuinely abnormal humans and falsely abnormal behaviors among humans performing abnormal behaviors. It has the effect of detecting and recognizing abnormal behavior based on deep learning.

1 is a block diagram showing a connection structure of complex features in a dense block.
Fig. 2 is a block diagram showing the overall architecture of a dense neural network with four blocks.
3 is a conceptual diagram illustrating the overall concept of an abnormal behavior detection method for detecting and recognizing a deep learning-based abnormal behavior according to an embodiment of the present invention.
4 is a block diagram showing an overall architecture based on a channel attention model 3D dense convolutional neural network.
5 is a block diagram illustrating a channel attention module;
6 is a block diagram illustrating a special attention module;
7 is a flowchart illustrating a flow of a pre-processing step of an abnormal behavior detection system for detecting and recognizing a deep learning-based abnormal behavior according to an embodiment of the present invention.
8 is a flowchart illustrating an overall flow of an abnormal behavior detection method for detecting and recognizing a deep learning-based abnormal behavior according to an embodiment of the present invention.
9 is a block diagram showing the overall configuration of an abnormal behavior detection system for detecting and recognizing a deep learning-based abnormal behavior according to an embodiment of the present invention.
10 is a block diagram illustrating a configuration of a photographing unit in an abnormal behavior detection system for detecting and recognizing a deep learning-based abnormal behavior according to an embodiment of the present invention;
11 is a block diagram showing the configuration of a clouding server in an abnormal behavior detection system that detects and recognizes a deep learning-based abnormal behavior according to an embodiment of the present invention.

Before describing the present invention in detail, the terms or words used herein should not be construed as being unconditionally limited to their ordinary or dictionary meanings, and in order for the inventor of the present invention to describe his invention in the best way It should be understood that the concepts of various terms can be appropriately defined and used, and further, these terms or words should be interpreted as meanings and concepts consistent with the technical idea of the present invention.

That is, the terms used herein are only used to describe preferred embodiments of the present invention, and are not used for the purpose of specifically limiting the content of the present invention, and these terms represent various possibilities of the present invention. It should be understood that the term has been defined with consideration in mind.

Also, in the present specification, it should be understood that, unless the context clearly indicates otherwise, the expression in the singular may include a plurality of expressions, and even if it is similarly expressed in plural, it should be understood that the meaning of the singular may be included. .

In the case where it is stated throughout this specification that a component "includes" another component, it does not exclude any other component, but further includes any other component unless otherwise stated. It could mean that you can.

Furthermore, when it is described that a component is "exists in or is connected to" of another component, this component may be directly connected to or installed in contact with another component, and a certain It may be installed spaced apart by a distance, and in the case of being installed spaced apart by a certain distance, a third component or means for fixing or connecting the corresponding component to another component may exist, and now It should be noted that the description of the components or means of 3 may be omitted.

On the other hand, when it is described that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the third element or means does not exist.

Similarly, other expressions describing the relationship between components, such as "between" and "immediately between", or "neighboring to" and "directly adjacent to", have the same meaning. should be interpreted as

In addition, if terms such as "one side", "other side", "one side", "other side", "first", "second" are used in this specification, for one component, this one component is It is used to be clearly distinguished from other components, and it should be understood that the meaning of the component is not limitedly used by such terms.

In addition, if used in this specification, terms related to positions such as "upper", "lower", "left", "right", it should be understood as indicating a relative position in the drawing with respect to the corresponding component, Unless an absolute position is specified with respect to their position, these position-related terms should not be construed as referring to an absolute position.

In addition, in this specification, in specifying the reference numerals for each component of each drawing, the same component has the same reference number even if the component is shown in different drawings, that is, the same reference is made throughout the specification. Symbols indicate identical components.

In the drawings attached to this specification, the size, position, coupling relationship, etc. of each component constituting the present invention are partially exaggerated, reduced, or omitted for convenience of explanation or in order to sufficiently clearly convey the spirit of the present invention. may be described, and therefore the proportion or scale may not be exact.

In addition, in the following, in describing the present invention, a detailed description of a configuration determined that may unnecessarily obscure the subject matter of the present invention, for example, a detailed description of a known technology including the prior art may be omitted.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the related drawings.

In recent years, with the development of deep learning, motion recognition in video has been attracting attention.

Its powerful capabilities have been demonstrated in numerous applications such as surveillance, robotics, medical care, video search, virtual reality, and human-computer interaction.

Unlike static image comprehension, moving image comprehension requires a more stable motion function to reflect the dynamic changes that occur in the movie.

We have briefly divided behavior recognition into two categories: manual and deep learning-based methods. In the manual expression learning method, in general, spatiotemporal points of interest are detected, and these points are identified as STIP (Space-Time Interest Points), gradient histograms and optical flow histograms, gradient 3D histograms, and density trajectories (iDTs) of state-of-the-art handicraft methods. Described in a local expression such as

By pooling rich descriptors along the density trajectory, we explicitly take into account motion features and compensate for camera motion.

Then, by the encoding method, the descriptor is aggregated into a video-level representation.

1 is a block diagram illustrating a connection structure of complex features in a dense block.

Referring to Figure 1, in recent years, with the advent of deep learning, especially CNN neural networks and the emergence of powerful and excellent processing power, more and more CNN neural networks are used to study behavioral recognition in video.

For example, a 3D convolutional neural network (C3D) uses a 3D convolution kernel to extract features from a series of dense RGB frames.

In Temporal Segment Networks (TSN) sample frames and optical flows for different time segments to extract information for activity recognition, the I3D neural network is a two-stream CNN with 3D convolutions dilated in both dense RGB and optical flow sequences. to achieve state-of-the-art performance on kinetic data sets.

And recently widely used CNN neural networks classify video by combining spatiotemporal effects, optical flow estimation, and fusion of two streams for behavioral recognition.

Additionally, adding a pre-trained network model using transfer learning makes it effective to transfer knowledge within or between modalities and greatly improves performance.

2 is a block diagram showing the overall architecture of a dense neural network with four blocks.

Referring to Fig. 2, in a deep learning neural network, the problem of gradient extinction will become more and more apparent as the neural network deepens.

In the present invention, a dense convolutional neural network is used as a basic neural network.

Dense neural networks directly connect all layers of a neural network using feature maps of the same size, ensuring maximum information flow between layers.

Each layer needs information from all previous layers as input, and then passes a feature map.

The structure can be confirmed with reference to FIG. 1 .

In conventional convolutional neural networks, if there is an L layer, there are L connections, but in dense neural networks there are L(L + 1) / 2 connections.

Each layer has direct access to the loss function and gradients from the original input signal, leading to implicit deep monitoring.

The shorter the connections between the near-input and near-output layers, the deeper, more accurate and more effective the convolutional neural network can be.

Due to the dense block layer, dense neural networks have narrower networks and fewer parameters than ResNets.

At the same time, this connection makes the transmission of features and gradients more efficient and easier to train the network.

It replaces the standard transformation layer of the original Dense neural network framework, focuses on the feature region after convolution, reduces the neural network training parameters, and adds an attention model to achieve better results.

3 is a conceptual diagram illustrating the overall concept of an abnormal behavior detection method for detecting and recognizing a deep learning-based abnormal behavior according to an embodiment of the present invention.

Referring to Figure 3, in the abnormal behavior detection method for detecting and recognizing the deep learning-based abnormal behavior according to an embodiment of the present invention, a dense convolutional network combined with an attention mechanism model (Attention Mechanism Model) use the

In the case of a single attention mechanism model, it is not effective to extract the motion characteristics of image information with a large amount of information.

Therefore, a dual channel attention model of time and space is adopted and used.

This enhanced model for feature extraction improves the extraction of motion features from image data.

4 is a block diagram illustrating an overall architecture based on a channel attention model 3D dense convolutional neural network.

Referring to FIG. 4 , it is based on a 3D dense convolutional neural network, and the original neural network is modified by adding an attention model.

In the attention model, an effective attention mechanism is used based on a combination of channel attention and special attention.

- 3D Dense Neural Network -

Using deep convolutional neural networks can improve the overall performance of both streaming methods.

In the present invention, a 3D dense neural network is formed by using a dense neural network as a main network branch and adding a corresponding 3D module.

Based on the original 3D dense network, an attention module is added to the transformation layer between dense blocks to improve the feature recognition effect.

And bring densely connected multiple density blocks.

A denser dense block can make the neural network deeper and better, but it increases the parameters and complexity of the network.

Conversely, if the selected density block is too small, the number of layers of the neural network decreases, affecting the accuracy of the model.

In the present invention, four dense blocks are used as an example.

Each density block contains several complex functions connected in a Feed-Forward Manner.

The structure will be referred to FIG.

An attention module is added between two adjacent dense blocks to enhance the feature recognition effect.

In this neural network, a 2D dense neural network similar to a 3D dense block is used.

It connects the 3D output of a layer directly to all subsequent layers of the 3D dense block.

특징 맵(Map)을 입력으로 수신한다.In the lth layer, the complex function H _l is the value of all previous (l-1) layers.

Receives a feature map as an input.

_{The feature map of H l} in the l-th layer is given by Equation 1 below.

[Equation 1]

는 특징 맵과 연결되도록 표시된다.here,

is marked to be connected with the feature map.

특징 맵의 공간 사이즈는 동일하다.

The spatial size of the feature map is the same.

H _l is a complex function of the BN-ReLU-3DConv operation.

- Attention model -

Attention plays an important role in human perception.

In recent years, it has excelled in image recognition.

Because the attention mechanism uses selective focus on a specific area to better capture the desired result, we apply the attention mechanism to the video recognition task.

In the attention model, an attention model that combines channel attention and special attention (Spatial Attention) is used.

- Channel Attention Module -

5 is a block diagram illustrating a channel attention module.

Referring to FIG. 5 , the channel attention map is a map utilizing the relationship between channels of features.

The structure can be confirmed with reference to FIG. 5 .

Since each channel in the feature map is considered a feature detector, the channel attention focuses on a meaningful 'What' given in the input image.

In order to efficiently calculate the channel attention, the special dimension of the input feature map is compressed.

In this case, the average pooling and maximum pooling features are used at the same time.

와, 최대 풀링 특징인

의 두 가지 다른 스페셜 컨텍트 디스크립터(Descriptor)를 생성하여, 양쪽 특징을 모두 활용하면 신경망의 표현력이 크게 향상된다.average pooling characteristic

Wow, the max pooling feature

By creating two different special contact descriptors of

Behind the two pooling layers, a hidden layer of Multilayer Perceptron is added to reduce parametric overhead.

The two descriptors are transmitted to the hidden layer and then generate Mc, which is a channel attention map.

In other words, the channel attention is calculated by the following Equation (2).

[Equation 2]

는 시그모이드 함수(Sigmoid Function)이다.here,

is a sigmoid function.

- Special attention module (Spatial Attention Model) -

6 is a block diagram illustrating a special attention module.

Referring to FIG. 6 , the special attention map generates a special attention map by using a relationship between specials of features.

The structure can be confirmed with reference to FIG. 6 .

Unlike channel attention, special attention focuses on 'where', an information part that complements channel attention.

First, average pooling and maximum pooling operations are applied along the channel axis, and an efficient feature descriptor is generated by connecting them.

A convolution layer is applied to the connected feature descriptor to generate a special attention map, Ms(F).

을 획득하고, 두 개의 풀링은 특징 맵의 채널 정보 연산을 집계하여 두 개의 2D 맵을 생성한다.Next, two pooling features using average pooling and max pooling

, and the two pooling aggregates the channel information operation of the feature map to generate two 2D maps.

After that, it is connected and convolved by the standard convolutional layer to generate a 2D special attention map.

In other words, the special attention is calculated by Equation 3 below.

[Equation 3]

는 시그모이드 함수이고,

은 필터 크기가 7 × 7 사이즈를 갖는 컨볼루션 연산을 나타낸다.here,

is the sigmoid function,

denotes a convolution operation with a filter size of 7 × 7.

Two attention modules, Channel and Special, calculate complementary attention by focusing on 'what' and 'where', respectively.

Considering this, two modules can be arranged in parallel or sequentially.

In the present invention, a concatenated approach is used to combine channel attention and special attention, which has a better effect than parallel.

7 is a flowchart illustrating a flow of a pre-processing step of an abnormal behavior detection system for detecting and recognizing a deep learning-based abnormal behavior according to an embodiment of the present invention.

In the abnormal behavior detection method for detecting and recognizing abnormal behavior based on deep learning according to the present invention, the first method for analyzing the behavior of a pedestrian by performing a convolution calculation on the image data provided by the photographing unit 100 Before the step, it includes three pre-treatment steps.

In the first pre-processing step ( S10 ), image data of a pedestrian in the corresponding area is collected by the photographing unit 100 .

In the second pre-processing step S20, the resolution of the image data is adjusted.

In the third pre-processing step S30 , the image data is transmitted to the cloud server 200 .

By this pre-processing step, the photographing unit 100 adjusts the resolution of the photographed data and then provides it to the cloud server 200 .

8 is a flowchart illustrating an overall flow of an abnormal behavior detection method for detecting and recognizing a deep learning-based abnormal behavior according to an embodiment of the present invention.

Referring to FIG. 8 , the abnormal behavior detection method for detecting and recognizing the deep learning-based abnormal behavior according to an embodiment of the present invention includes five steps.

In the first step ( S100 ), the behavior of the pedestrian is analyzed by performing a convolution calculation on the image data provided by the photographing unit 100 .

Here, the convolution operation is performed by a dense convolutional network, and image data is input to the dense convolutional neural network in RGB 3-channel format with a 224 × 22 image size through feature block processing.

Here, the convolutional neural network is a kind of multi-layer feed-forward artificial neural network used to analyze visual images.

It is classified as a deep neural network in deep learning, and is mainly applied to visual image analysis.

It is also known as mutant invariant or spatially invariant artificial neural network (SIANN) based on shared weight structure and transform invariant properties.

It is applied to image and video recognition, recommendation system, image classification, medical image analysis, and natural language processing.

A convolutional neural network is a normalized version of a multilayer perceptron.

A multilayer perceptron is usually a fully connected network, i.e. a neural network structure in which each neuron in one layer connects to all neurons in the next layer.

When the network is fully connected like this, it tends to overfit to the given data.

For general regularization, adding a specific measure to the optimization function is common, but a different approach for regularizing convolutional neural networks is taken.

By leveraging hierarchical patterns in your data and using smaller, simpler patterns to represent more complex patterns, you achieve the same effect as regularization.

Therefore, the complexity of the connection structure of the convolutional neural network is extremely low compared to multilayer perceptrons with similar functions.

Convolutional neural networks are inspired by the fact that the pattern of connections between neurons resembles the organization of the animal visual cortex.

Individual cortical neurons respond to stimuli only in a limited area of the field of view known as the receptive field.

The receptive fields of different neurons partially overlap to see the entire field of view.

Image classification using a convolutional neural network uses relatively little preprocessing compared to other image classification algorithms.

This means that the neural network learns hand-crafted filters from existing algorithms.

The main advantage of convolutional neural networks is that there is no process in which the designer understands the characteristics of the image in advance and creates the algorithm in the existing image classification algorithm.

A convolutional neural network is largely composed of a convolution layer and a pooling layer.

In the second step (S200), the behavior of the pedestrian is analyzed in detail through the attention mechanism model (Attention Mechanism Model: 230).

When each image data is input to the dense convolutional neural network, the dense convolutional neural network connects each input image data and is input to a transform block to continue the learning process.

In this case, the feature block and the transform block may be a convolution layer and a pooling layer.

In addition, the attention mechanism model 230 includes a channel attention model (Channel Attention Model: 231) and a special attention model (Spatial Attention Model: 232).

Here, the attention mechanism model 230 performs learning such that the attention ignores irrelevant information and concentrates on the core information.

In addition, the channel attention model was first proposed by Hu et al. under the name of squeeze-and-excitation block.

A general convolution operation uses only local information, but the channel attention module applies global average pooling (GAP) to make it possible to use even non-local information.

In the third step ( S300 ), whether the pedestrian's behavior is normal in the image data is analyzed as a time period through the channel attention model 231 .

In addition, in the deep learning-based abnormal behavior detection method for detecting and recognizing abnormal behavior according to the present invention, the channel attention model 231 samples a picture of a point requiring attention in the image, and a behavior image of a pedestrian from the sampled picture to process

Such a channel attention model 231 has continuity between specific channels.

In addition, in the deep learning-based abnormal behavior detection method for detecting and recognizing abnormal behavior according to the present invention, the special dimension of the input feature map is compressed using a combination of the maximum pooling layer and the average pooling layer to increase the efficiency of channel attention. can

Behind these two pooling layers, a hidden layer of multi-layer perceptrons can be added to reduce parameter overhead.

On the other hand, in the abnormal behavior detection method for detecting and recognizing the deep learning-based abnormal behavior according to the present invention, two descriptors are characterized in that the channel attention map transmitted to the hidden layer is generated.

In the fourth step ( S400 ), it is analyzed whether the pedestrian's behavior is normal through the channel attention model 231 related to the special attention model 232 .

In particular, in the abnormal behavior detection method for detecting and recognizing the deep learning-based abnormal behavior according to the present invention, the special attention model 232 analyzes by focusing on the location.

In the fifth step (S500), the pedestrian's behavior is divided into normal behavior and abnormal behavior through semantic analysis, and the abnormal behavior is analyzed into unintentional normal behavior and intentional abnormal behavior.

9 is a block diagram showing the overall configuration of an abnormal behavior detection system for detecting and recognizing a deep learning-based abnormal behavior according to an embodiment of the present invention.

Referring to FIG. 9 , an abnormal behavior detection system 1000 for detecting and recognizing a deep learning-based abnormal behavior according to an embodiment of the present invention includes a photographing unit 100 and a clouding server 200 .

The photographing unit 100 provides the photographed image data to the clouding server 200 so that an unspecified number of humans can be extracted by analyzing the image data photographed through a camera, etc., and whether there is an abnormal behavior by analyzing the behavior perform the role

The clouding server 200 performs a role of analyzing the image data provided from the photographing unit 100 to extract a large number of unspecified humans and analyzing the behavior to analyze whether there is an abnormal behavior.

10 is a block diagram showing the configuration of a photographing unit in the abnormal behavior detection system for detecting and recognizing abnormal behavior based on deep learning according to an embodiment of the present invention.

Referring to FIG. 10 , in the abnormal behavior detection system 1000 for detecting and recognizing abnormal behavior based on deep learning according to the present invention, the photographing unit 100 includes a camera unit 110 , a resolution adjusting unit 120 , and a communication unit. (130).

Here, the camera unit 110 serves to collect image data of pedestrians in the corresponding area.

The resolution adjusting unit 120 serves to adjust the resolution of the image data captured and collected by the camera unit 110 .

The communication unit 130 serves to transmit the image data whose resolution is adjusted by the resolution adjustment unit 120 to the cloud server 200 .

11 is a block diagram showing the configuration of a clouding server in an abnormal behavior detection system that detects and recognizes a deep learning-based abnormal behavior according to an embodiment of the present invention.

In the abnormal behavior detection system 1000 for detecting and recognizing a deep learning-based abnormal behavior according to an embodiment of the present invention, the clouding server 200 includes a receiving unit 210, a calculating unit 220, and an attention mechanism model unit. 230 , a semantic analysis unit 240 , and an output unit 250 .

Here, the receiving unit 210 serves to receive the image data photographed by the photographing unit 100 by the communication unit 130 .

That is, the receiver 210 receives input image data.

The calculator 220 analyzes the behavior of the pedestrian by performing a convolution operation on the image data received by the receiver 210 .

The attention mechanism model unit 230 performs a role of detailed analysis of the pedestrian's behavior.

That is, the attention mechanism model of the attention mechanism model unit 230 performs learning so that the attention ignores irrelevant information and focuses on the core information.

The attention mechanism model unit 230 includes a channel attention model unit 231 and a special attention model unit 232 .

The channel attention model of the channel attention model unit 231 analyzes whether the pedestrian's behavior is normal in the image data as a time period.

The channel attention model of the channel attention model unit 231 samples a picture of a point requiring attention in the image, and processes a behavior image of a pedestrian from the sampled picture.

The channel attention model of the channel attention model unit 231 has continuity between specific channels.

The special attention model 232 analyzes whether the pedestrian's behavior is normal through the channel attention model.

The special attention model 232 performs analysis by focusing on the location.

The semantic analysis unit 240 divides the pedestrian's behavior into a normal behavior and an abnormal behavior through semantic analysis, and analyzes the abnormal behavior into an unintentional normal behavior and an intentional abnormal behavior.

The output unit 250 serves to output the final result data on which the semantic analysis is performed by the semantic analysis unit 240 .

As described above, according to the present invention, when human behavior is analyzed and normal behavior and abnormal behavior are detected from human behavior, among humans performing abnormal behavior, genuinely abnormal people and falsely abnormal behaviors are detected. It has the effect of detecting and recognizing an abnormal behavior based on deep learning that can be recognized.

In the above, although several preferred embodiments of the present invention have been described with some examples, the descriptions of various various embodiments described in the "Specific Contents for Carrying Out the Invention" item are merely exemplary, and the present invention Those of ordinary skill in the art will understand well that the present invention can be practiced with various modifications or equivalents to the present invention from the above description.

In addition, since the present invention can be implemented in various other forms, the present invention is not limited by the above description, and the above description is intended to complete the disclosure of the present invention, and is usually It should be understood that this is only provided to fully inform those with knowledge of the scope of the present invention, and that the present invention is only defined by each of the claims.

100: shooting department
110: camera unit
120: resolution adjustment unit
130: communication department
200 : Cloud Server
210: receiver
220: arithmetic unit
230: attention mechanism model part
231: channel attention model unit
232: special attention model part
240: semantic analysis unit
250: output unit
1000: Abnormal behavior detection system that detects and recognizes abnormal behavior based on deep learning

Claims (18)

delete As an abnormal behavior detection method for detecting and recognizing deep learning-based abnormal behavior performed by a clouding server,
A first step of analyzing the behavior of the pedestrian by performing a convolution operation (Convolution Calculation) on the image data provided by the photographing unit;
a second step of analyzing the pedestrian's behavior in detail through an attention mechanism model;
a third step of analyzing whether a pedestrian's behavior is normal in the image data using a channel attention model as a time period;
a fourth step of analyzing whether the pedestrian's behavior is normal through the channel attention model related to the special attention model; and
A fifth step of classifying pedestrian behavior into normal behavior and abnormal behavior through semantic analysis, and analyzing the abnormal behavior into unintentional normal behavior and intentional abnormal behavior;
The second step is a third step of analyzing whether the pedestrian's behavior is normal in the image data through the channel attention model as a time period and whether the pedestrian's behavior is normal through the channel attention model related to the special attention model Consists of including a fourth step to analyze for
The convolution operation is performed by a dense convolutional network,
The image data is input to the dense convolutional neural network through feature block processing in RGB 3-channel format with a 224 × 22 image size,
When each of the image data is input to the dense convolutional neural network, the dense convolutional neural network splices each of the input image data and is input to a transform block to continue the learning process,
The feature block and the transform block are a convolution layer and a pooling layer,
The attention mechanism model includes a channel attention model (Channel Attention Model) and a special attention model (Spatial Attention Model),
The attention mechanism model performs learning so that attention ignores irrelevant information and focuses on key information,
The channel attention model samples a picture of a point where attention is required in the image, and processes a behavior image of a pedestrian from the sampled picture,
The channel attention model has continuity between specific channels,
The dense convolutional neural network adds the attention mechanism model to the transform layer between two adjacent dense blocks, but in the l-th layer, the complex function Hl inputs the feature maps of all previous (l-1) layers as input. Received, characterized in that the feature map of Hl in the l-th layer is Equation 1 below,
An abnormal behavior detection method that detects and recognizes abnormal behavior based on deep learning.
[Equation 1]

- here,

is marked to be connected with the feature map,

The spatial size of the feature map is the same, and H _l is a complex function of the BN-ReLU-3DConv operation -
delete delete delete delete delete delete delete 3. The method of claim 2,
Characterized in increasing the efficiency of channel attention by compressing the special dimension of the input feature map using a combination of the maximum pooling layer and the average pooling layer,
An abnormal behavior detection method that detects and recognizes abnormal behavior based on deep learning.
11. The method of claim 10,
A hidden layer of multi-layer perceptron is added behind the two pooling layers to reduce the overhead of parameters,
An abnormal behavior detection method that detects and recognizes abnormal behavior based on deep learning.
12. The method of claim 11,
Two descriptors (Descriptor) characterized in that it generates a channel attention map transmitted to the hidden layer,
An abnormal behavior detection method that detects and recognizes abnormal behavior based on deep learning.
delete 3. The method of claim 2,
The special attention model is characterized in that the analysis is concentrated on the location,
An abnormal behavior detection method that detects and recognizes abnormal behavior based on deep learning.
delete 3. The method of claim 2,
Prior to the first step,
a first pre-processing step of collecting image data of pedestrians in the corresponding area by the photographing unit;
a second pre-processing step of adjusting the resolution of the image data; and
A third pre-processing step of transmitting the image data to the clouding server; characterized in that it comprises,
An abnormal behavior detection method that detects and recognizes abnormal behavior based on deep learning.
a receiver for receiving input image data;
a calculation unit for performing a convolution operation on the image data to analyze a pedestrian's behavior;
an attention mechanism model unit for detailed analysis of the pedestrian's behavior; and
A clouding server comprising; a semantic analysis unit that classifies pedestrian behavior into normal behavior and abnormal behavior through semantic analysis, and analyzes the abnormal behavior into unintentional normal behavior and intentional abnormal behavior;
a camera unit that collects image data of pedestrians in the corresponding area;
a resolution adjusting unit for adjusting the resolution of the image data; and
a communication unit for transmitting the image data to the clouding server; and a photographing unit comprising a,
The convolution operation is performed by a dense convolutional network,
The image data is input to the dense convolutional neural network through feature block processing in RGB 3-channel format with a 224 × 22 image size,
When each of the image data is input to the dense convolutional neural network, the dense convolutional neural network splices each of the input image data and is input to a transform block to continue the learning process,
The feature block and the transform block are a convolution layer and a pooling layer,
The attention mechanism model unit includes a channel attention model (Channel Attention Model) and a special attention model (Spatial Attention Model),
The attention mechanism model unit performs learning so that attention ignores irrelevant information and focuses on key information,
The channel attention model samples a picture of a point where attention is required in the image, and processes a behavior image of a pedestrian from the sampled picture,
The channel attention model has continuity between specific channels,
The dense convolutional neural network adds the attention mechanism model part to the transformation layer between two adjacent dense blocks, but in the l-th layer, the complex function Hl inputs the feature maps of all previous (l-1) layers as input. Received, characterized in that the feature map of Hl in the l-th layer is Equation 1 below,
An abnormal behavior detection system that detects and recognizes abnormal behavior based on deep learning.
[Equation 1]

- here,

is marked to be connected with the feature map,

The spatial size of the feature map is the same, and H _l is a complex function of the BN-ReLU-3DConv operation -
delete

Images