CN117541991A - Intelligent recognition method and system for abnormal behaviors based on security robot - Google Patents

Intelligent recognition method and system for abnormal behaviors based on security robot Download PDF

Info

Publication number
CN117541991A
CN117541991A CN202311563231.9A CN202311563231A CN117541991A CN 117541991 A CN117541991 A CN 117541991A CN 202311563231 A CN202311563231 A CN 202311563231A CN 117541991 A CN117541991 A CN 117541991A
Authority
CN
China
Prior art keywords
abnormal behavior
behavior recognition
recognition network
motion
frame difference
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202311563231.9A
Other languages
Chinese (zh)
Other versions
CN117541991B (en
Inventor
顾涛
李昭生
马松
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuxi Kebang An Intelligent Technology Co ltd
Original Assignee
Wuxi Kebang An Intelligent Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuxi Kebang An Intelligent Technology Co ltd filed Critical Wuxi Kebang An Intelligent Technology Co ltd
Priority to CN202311563231.9A priority Critical patent/CN117541991B/en
Publication of CN117541991A publication Critical patent/CN117541991A/en
Application granted granted Critical
Publication of CN117541991B publication Critical patent/CN117541991B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/30—Noise filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30232—Surveillance
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a security robot-based intelligent recognition method and system for abnormal behaviors, comprising the following steps: s1: collecting video data of the security robot, preprocessing the collected video data, and obtaining a preprocessed video image sequence; s2: extracting motion characteristics from the preprocessed image sequence based on a frame difference method; s3: constructing an abnormal behavior recognition network by using a cyclic neural network, and setting an optimization target of the abnormal behavior recognition network; s4: optimizing parameters of the abnormal behavior recognition network based on an improved random gradient descent algorithm; s5: and (5) performing fine adjustment on the abnormal behavior recognition network parameters by using a differential evolution algorithm. The method and the device can improve the accuracy and the robustness of abnormal behavior identification, thereby providing a more reliable solution for safety precaution.

Description

Intelligent recognition method and system for abnormal behaviors based on security robot
Technical Field
The invention relates to the technical field of intelligent recognition of abnormal behaviors, in particular to a method and a system for intelligent recognition of abnormal behaviors based on a security robot.
Background
With the development of society and the progress of technology, the demand in the security field is increasing. Security robots have received wide attention and application as a new type of equipment integrating advanced sensing, intelligent analysis and real-time response capabilities. The system has great potential in the aspects of monitoring, patrol, safety early warning and the like, and provides powerful support for guaranteeing public safety, protecting property and the like. However, as the application scene of the security robot becomes more and more complex and diverse, intelligent recognition of abnormal behaviors becomes an urgent need. Conventional methods are often limited by factors such as image quality, lighting conditions, etc., resulting in unsatisfactory accuracy and robustness in complex environments. Meanwhile, in the traditional abnormal behavior recognition method, the accuracy of recognition is affected due to the fact that the depth excavation of motion features is lacking in the processing of dynamic scenes and background interference is easily caused. In addition, the existing identification method has certain limitations in network structure and parameter optimization. Conventional network structures may have difficulty capturing unusual behavioral characteristics in complex scenarios, resulting in limited recognition performance. In terms of parameter optimization, the conventional algorithm may be trapped in a locally optimal solution, and the performance of the network cannot be fully exerted.
Disclosure of Invention
In view of the above, the invention provides an intelligent recognition method for abnormal behaviors based on a security robot, which aims to improve the accuracy and the robustness of abnormal behavior recognition, thereby providing a more reliable solution for security.
The intelligent recognition method for abnormal behaviors based on the security robot provided by the invention comprises the following steps of:
s1: collecting video data of the security robot, preprocessing the collected video data, and obtaining a preprocessed video image sequence;
s2: extracting motion characteristics from the preprocessed image sequence based on a frame difference method;
s3: constructing an abnormal behavior recognition network by using a cyclic neural network, and setting an optimization target of the abnormal behavior recognition network;
s4: optimizing parameters of the abnormal behavior recognition network based on an improved random gradient descent algorithm;
s5: performing fine adjustment on abnormal behavior recognition network parameters by using a differential evolution algorithm;
as a further improvement of the present invention:
optionally, in the step S1, video data of the security robot is collected, and the collected video data is preprocessed to obtain a preprocessed video image sequence, which includes:
collecting video data of a security robot, preprocessing the collected video data to obtain a preprocessed image sequence, wherein a three-dimensional denoising algorithm based on the video data is used in the preprocessing process, and the computing mode is as follows:
wherein V is denoised Representing the preprocessed video image sequence; v (V) denoised (x, y, t) is the denoised pixel value of the image of the t frame of the video at (x, y); x represents an image abscissa, x=1, 2,..x, X represents an image width, Y represents an image ordinate, y=1, 2,..y, Y represents an image height, T represents a video time frame number, t=1, 2,..; w (i, j, k) is the weight between the pixel (x+ i.y +j, t+k) and the target pixel (x, y, t); v (x+ i.y +j, t+k) is the pixel value of the video at (x+i, y+j) of the t+k frame image; the calculation mode of w (i, j, k) is as follows:
wherein V (x.y, t) is the pixel value of the image of the t frame of the video at (x, y); e represents a natural constant; sigma represents a parameter controlling the attenuation of the weight;
(i, j, k) is an element in the set N (x.y, t), N (x.y, t) representing a neighborhood position centered on (x.y, t), expressed as:
N(x.y，t)＝{(i，j，k)|i∈[-r，r]，j∈[-r，r]，k∈[-r，r]}
wherein r represents the neighborhood radius; i, j and k are integers;
optionally, the extracting motion features from the preprocessed image sequence based on the frame difference method in the step S2 includes:
s21: calculating a frame difference image:
extracting frame difference images of two adjacent frames for the preprocessed image sequence; for the t frame and the t+1st frame in the preprocessed image sequence, the calculation mode of the pixel value of the frame difference image is as follows:
wherein I is diff (x, y, t) represents a t-th frame difference image I diff (t) a value at a spatial location (x.y);
s22: gradient operation:
gradient information G of frame difference image calculated by Sobel operator 1 And G 1 ：
Wherein G is 1 (x, y, t) and G 2 (x, y, t) represents the values of the gradients of the frame difference image in the horizontal and vertical directions at the positions (x.y, t), respectively; k (K) 1 And K 2 Is the convolution kernel of the Sobel operator;
s23: motion feature extraction:
calculating the motion energy density E of the frame difference image:
E(x，y，t)＝G 1 (x，y，t) 2 +G 2 (x，y，t) 2
wherein E (x, y, t) represents the kinetic energy density value at location (x.y, t);
calculating the average motion intensity Mean of the frame difference image motion ：
Wherein, mean motion (t) represents the average motion intensity of the t-th frame difference image; num represents the number of pixels of the frame difference image of each frame;
calculating maximum motion intensity Max of frame difference image motion ：
Max motion (t)＝maximum x，y E(x，y，t)
Wherein the maximum function calculates a maximum value;
optionally, in the step S3, an abnormal behavior recognition network is constructed by using a recurrent neural network, and an optimization target of the abnormal behavior recognition network is set, including:
s31: extracting depth features of the frame difference image:
at time frame number t, the frame difference image is converted into a feature vector F (t), which is extracted by using VGG-16 network:
F(t)＝VGG16(I diff (t))
wherein I is diff (t) represents a t-th frame difference image;
s32: building an abnormal behavior recognition network external input:
combining the depth features with the motion features as external inputs to each time-step abnormal behavior recognition network:
input(t)＝[F(t)，Mean motion (t)，Max motion (t)]
s33: abnormal behavior recognition network hidden state update:
the update formula of the hidden state h (t) of the abnormal behavior recognition network at the time frame sequence number t is as follows:
wherein,and->Respectively representing a weight matrix and bias from an input layer to a hidden layer; />And->Respectively representing a weight matrix and bias from hidden layer to hidden layer; tanh represents a hyperbolic tangent function;
s34: output layer calculation:
wherein cls represents the probability of different abnormal behaviors predicted by the abnormal behavior recognition network, cls= [ cls ] 1 ，cls 2 ，...，cls M ]M is the number of abnormal behavior species;and->Respectively representing a weight matrix and bias from the hidden layer to the output layer; softmax represents the normalized exponential function;
s35: setting an optimization target of an abnormal behavior recognition network:
using the improved cross entropy loss as an optimization objective for the network, the expression is:
wherein m=1, 2,. -%, M; for the frame difference image data currently input to the abnormal behavior recognition network,is the true probability that the abnormal behavior is the m-th class; cls m Is the network prediction probability that the abnormal behavior is the m-th class; epsilon is the smoothing parameter;
optionally, optimizing parameters of the abnormal behavior recognition network based on the modified random gradient descent algorithm in the step S4 includes:
calculating an updated motion term:
wherein θ b Parameters representing the abnormal behavior recognition network at the b-th update;represented at theta b A loss function gradient below; beta is an adjustment parameter; b=1, 2, B, B represents the total number of updates;
updating parameters of the abnormal behavior recognition network:
θ b+1 ＝θ b -α·v b
wherein α is the learning rate;
the improved random gradient descent algorithm improves the updating mode of the abnormal identification network parameters, introduces the accumulation of the historical gradient, and enables the updating direction to be smoother and more consistent;
optionally, in the step S5, the tuning of the abnormal behavior recognition network parameter using a differential evolution algorithm includes:
s51: initializing a population:
in pair S4Abnormal behavior identification network parameter theta completing updating B Adding random numbers to generate P candidate solutions as initial individuals of the population;
s52: mutation operation:
delta for each individual d Three different individuals delta are randomly selected from the population d1 ，δ d2 ，δ d3 As reference vectors, d1+.d2+.d3+.d, d1, d2, d3 and d represent integers between 1 and P; structural variant delta' d ：
δ′ d ＝δ d1 +2·(δ d2 -δ d3 )
S53: crossover operation:
delta for each individual d Randomly selecting a dimension Q, q=1, 2,..q, Q representing the number of abnormal behavior recognition network parameters, constructing intersecting individuals
Wherein random represents a random number within 0 to 1;
s54: selection operation:
will beAnd delta d Calculating an improved cross entropy loss value in the step S35 as an abnormal behavior recognition network parameter, and selecting individuals with smaller loss values to remain in the population;
s55: repeating the iteration;
repeating S52-S54 until iterating 20 times;
the invention also discloses an intelligent recognition system for abnormal behaviors based on the security robot, which comprises the following steps:
and a pretreatment module: collecting video data of the security robot, and preprocessing the collected video data;
motion feature extraction module: extracting motion characteristics from the preprocessed image sequence based on a frame difference method;
abnormal behavior recognition module: constructing an abnormal behavior recognition network by using a cyclic neural network, and setting an optimization target of the abnormal behavior recognition network;
parameter optimization module: optimizing parameters of the abnormal behavior recognition network based on an improved random gradient descent algorithm;
and a fine adjustment module: performing fine adjustment on abnormal behavior recognition network parameters by using a differential evolution algorithm;
the beneficial effects are that:
the video image sequence after preprocessing is obtained by preprocessing the video data acquired by the security robot. Compared with the traditional method, the preprocessing mode can effectively reduce noise and interference in the image and improve the accuracy and reliability of subsequent feature extraction.
The invention adopts a frame difference method to extract the motion characteristics from the preprocessed image sequence. Compared with the traditional image processing method, the frame difference method can capture the motion information in the video more accurately, so that more reliable features are provided for subsequent abnormal behavior identification.
By introducing the cyclic neural network, the invention constructs the abnormal behavior recognition network. Compared with the traditional machine learning method, the deep learning can automatically learn complex features and modes, so that the accuracy and the robustness of recognition are improved. The invention introduces an improved random gradient descent algorithm and an adaptive optimization algorithm in the network optimization process. Compared with the traditional optimization algorithm, the algorithm can effectively adjust network parameters, and improves the overall performance of the system.
In combination with the optimization measures, the method comprehensively improves the accuracy, the robustness and the adaptability of the abnormal behavior identification from data preprocessing to feature extraction, network construction and parameter optimization.
Drawings
Fig. 1 is a schematic flow chart of a security robot-based abnormal behavior intelligent recognition method according to an embodiment of the invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings, without limiting the invention in any way, and any alterations or substitutions based on the teachings of the invention are intended to fall within the scope of the invention.
Example 1: an intelligent recognition method for abnormal behaviors based on a security robot is shown in fig. 1, and comprises the following steps:
s1: collecting video data of the security robot, preprocessing the collected video data, and obtaining a preprocessed video image sequence:
collecting video data of a security robot, preprocessing the collected video data to obtain a preprocessed image sequence, wherein the preprocessing uses a three-dimensional denoising algorithm based on the video data, and the computing mode is as follows:
wherein V is denoised Representing the preprocessed video image sequence; v (V) denoised (x, y, t) is the denoised pixel value of the image of the t frame of the video at (x, y); x represents an image abscissa, x=1, 2,..x, X represents an image width, Y represents an image ordinate, y=1, 2,..y, Y represents an image height, T represents a video time frame number, t=1, 2,..; w (i, j, k) is the weight between the pixel (x+ i.y +j, t+k) and the target pixel (x, y, t); v (x+ i.y +j, t+k) is the pixel value of the video at (x+i, y+j) of the t+k frame image; the calculation mode of w (i, j, k) is as follows:
wherein V (x.y, t) is the pixel value of the image of the t frame of the video at (x, y); e represents a natural constant; σ represents a parameter controlling the weight decay, 10 in this embodiment;
(i, j, k) is an element in the set N (x.y, t), N (x.y, t) representing a neighborhood position centered on (x.y, t), expressed as:
N(x.y，t)＝{(i，j，k)|i∈[-r，r]，j∈[-r，r]，k∈[-r，r]}
wherein r represents the neighborhood radius, which is 3 in this embodiment; i, j and k are integers;
by preprocessing the collected video data, noise in the video image can be effectively reduced, so that subsequent feature extraction and analysis are more reliable and accurate. The preprocessing may make critical information in the video image sequence more prominent, thereby facilitating the accuracy of subsequent feature extraction.
S2: extracting motion features from the preprocessed image sequence based on a frame difference method:
s21: calculating a frame difference image:
extracting frame difference images of two adjacent frames for the preprocessed image sequence; for the t frame and the t+1st frame in the preprocessed image sequence, the calculation mode of the pixel value of the frame difference image is as follows:
wherein I is diff (x, y, t) represents a t-th frame difference image I diff (t) a value at a spatial location (x.y);
s22: gradient operation:
gradient information G of frame difference image calculated by Sobel operator 1 And G 1 ：
Wherein G is 1 (x, y, t) and G 2 (x, y, t) represents the values of the gradients of the frame difference image in the horizontal and vertical directions at the positions (x.y, t), respectively; k (K) 1 And K 2 Is the convolution kernel of the Sobel operator;
s23: motion feature extraction:
calculating the motion energy density E of the frame difference image:
E(x，y，t)＝G 1 (x，y，t) 2 +G 2 (x，y，t) 2
wherein E (x, y, t) represents the kinetic energy density value at location (x.y, t);
calculating the average motion intensity Mean of the frame difference image motion ：
Wherein, mean motion (t) represents the average motion intensity of the t-th frame difference image; num represents the number of pixels of the frame difference image of each frame;
calculating maximum motion intensity Max of frame difference image motion ：
Max motion (t)＝maximum x，y E(x，y，t)
Wherein the maximum function calculates a maximum value;
the frame difference method is a simple but effective method that can quickly capture motion information in a sequence of images. By calculating the difference between adjacent frames, the position and trajectory of the moving object can be accurately located. By extracting the motion characteristics based on the frame difference method, the influence of the background can be effectively reduced, and the abnormal behavior recognition is more accurate. The motion features are more capable of highlighting the dynamics of the abnormal behavior than the static image processing. Compared with other complex motion feature extraction methods, the frame difference method has lower calculation cost. This enables the recognition of abnormal behavior to be performed more efficiently in a scene where the real-time requirements are high.
S3: constructing an abnormal behavior recognition network by using a cyclic neural network, and setting an optimization target of the abnormal behavior recognition network:
s31: extracting depth features of the frame difference image:
at time frame number t, the frame difference image is converted into a feature vector F (t), which is extracted by using VGG-16 network:
F(t)＝VGG16(I diff (t))
wherein I is diff (t) represents a t-th frame difference image;
s32: building an abnormal behavior recognition network external input:
combining the depth features with the motion features as external inputs to each time-step abnormal behavior recognition network:
input(t)＝[F(t)，Mean motion (t)，Max motion (t)]
s33: abnormal behavior recognition network hidden state update:
the update formula of the hidden state h (t) of the abnormal behavior recognition network at the time frame sequence number t is as follows:
wherein,and->Respectively representing a weight matrix and bias from an input layer to a hidden layer; />And->Respectively representing a weight matrix and bias from hidden layer to hidden layer; tanh represents a hyperbolic tangent function;
s34: output layer calculation:
wherein cls represents an abnormal rowTo identify the probability of different abnormal behaviour predicted by the network, cls= [ cls ] 1 ，cls 2 ，...，cls M ]M is the number of abnormal behaviors, in this embodiment, there are seven abnormal behaviors including intrusion detection, object carry-over, fire detection, smoke detection, aggregation detection, fight detection, and loiter detection;and->Respectively representing a weight matrix and bias from the hidden layer to the output layer; softmax represents a normalized exponential function expressed as:
wherein m=1, 2,. -%, M; (W) ho h(T)+b bo ) m Represents W ho h(T)+b bo The m-th element of (2);
s35: setting an optimization target of an abnormal behavior recognition network:
using the improved cross entropy loss as an optimization objective for the network, the expression is:
wherein, for the frame difference image data currently input to the abnormal behavior recognition network,is the true probability that the abnormal behavior is the m-th class; cls m Is the network prediction probability that the abnormal behavior is the m-th class; e is a smoothing parameter, 10 in this embodiment -3 ；
The recurrent neural network is suitable for processing data with time sequence correlation, and can effectively capture time information in a video sequence, so that the system can better understand and analyze dynamic changes in the video data. The cyclic neural network has strong feature learning capability, can automatically learn proper feature representation from data, and does not need to manually designate features in advance. This makes the system more adaptable and generalizable. The cyclic neural network can comprehensively consider information at each moment in the video sequence through multiple iterations, so that the dynamic evolution process of the abnormal behavior is comprehensively understood, and the accuracy of the abnormal behavior identification is improved.
S4: optimizing parameters of the abnormal behavior recognition network based on an improved random gradient descent algorithm:
calculating an updated motion term:
wherein θ b Parameters representing the abnormal behavior recognition network at the b-th update;represented at theta b A loss function gradient below; beta is an adjustment parameter, in this example 0.95; b=1, 2, B, B represents the total number of updates;
updating parameters of the abnormal behavior recognition network:
θ b+1 ＝θ b -α·v b
where α is the learning rate, which in this embodiment is 5×10 -4 ；
The improved random gradient descent algorithm improves the updating mode of the abnormal identification network parameters, introduces the accumulation of the historical gradient, and enables the updating direction to be smoother and more consistent;
the improved random gradient descent algorithm can help the system to find the globally optimal solution, and avoid the dilemma of being in the locally optimal solution. This enables the network parameters to be better optimized, improving the performance of the system. Compared with the traditional gradient descent algorithm, the improved random gradient descent algorithm is faster in convergence speed, and can obtain better optimization results in a shorter time. The optimization algorithm can avoid severe oscillation of parameters in the training process, so that stability of the optimization process is ensured.
S5: and (3) performing fine adjustment on the abnormal behavior recognition network parameters by using a differential evolution algorithm:
s51: initializing a population:
identifying network parameter theta for abnormal behavior after finishing updating in S4 B Adding random numbers to generate P candidate solutions as initial individuals of the population;
s52: mutation operation:
delta for each individual d Three different individuals delta are randomly selected from the population d1 ，δ d2 ，δ d3 As reference vectors, d1+.d2+.d3+.d, d1, d2, d3 and d represent integers between 1 and P; structural variant delta' d ：
δ′ d ＝δ d1 +2·(δ d2 -δ d3 )
S53: crossover operation:
delta for each individual d Randomly selecting a dimension Q, q=1, 2,..q, Q representing the number of abnormal behavior recognition network parameters, constructing intersecting individuals
Wherein random represents a random number within 0 to 1;
s54: selection operation:
will beAnd delta d Calculating an improved cross entropy loss value in the step S35 as an abnormal behavior recognition network parameter, and selecting individuals with smaller loss values to remain in the population;
s55: repeating the iteration;
repeating S52-S54 until iterating 20 times;
the differential evolution algorithm can conduct global search in the parameter space, is favorable for finding out a global optimal solution, avoids the dilemma of being in a local optimal solution, and improves the performance of the system. Because the differential evolution algorithm adopts a random search strategy, the convergence to a suboptimal solution in the training process can be effectively avoided, and the optimization quality of network parameters is ensured. The differential evolution algorithm can flexibly adjust the trimming amplitude of each parameter, so that the trimming process is more accurate and effective.
Example 2: the invention also discloses an intelligent recognition system for abnormal behaviors based on the security robot, which comprises the following five modules:
and a pretreatment module: collecting video data of the security robot, and preprocessing the collected video data;
motion feature extraction module: extracting motion characteristics from the preprocessed image sequence based on a frame difference method;
abnormal behavior recognition module: constructing an abnormal behavior recognition network by using a cyclic neural network, and setting an optimization target of the abnormal behavior recognition network;
parameter optimization module: optimizing parameters of the abnormal behavior recognition network based on an improved random gradient descent algorithm;
and a fine adjustment module: and (5) performing fine adjustment on the abnormal behavior recognition network parameters by using a differential evolution algorithm.
It should be noted that, the foregoing reference numerals of the embodiments of the present invention are merely for describing the embodiments, and do not represent the advantages and disadvantages of the embodiments. And the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, apparatus, article or method that comprises the element.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above, comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.

Claims (7)

1. The intelligent recognition method for abnormal behaviors based on the security robot is characterized by comprising the following steps of:
s1: collecting video data of the security robot, preprocessing the collected video data, and obtaining a preprocessed video image sequence;
s2: extracting motion characteristics from the preprocessed image sequence based on a frame difference method;
s3: constructing an abnormal behavior recognition network by using a cyclic neural network, and setting an optimization target of the abnormal behavior recognition network;
s4: optimizing parameters of the abnormal behavior recognition network based on an improved random gradient descent algorithm; calculating an updated motion item, and updating parameters of the abnormal behavior recognition network;
s5: and (5) performing fine adjustment on the abnormal behavior recognition network parameters by using a differential evolution algorithm.
2. The intelligent recognition method for abnormal behaviors based on the security robot according to claim 1, wherein a three-dimensional denoising algorithm based on video data is used in the preprocessing process in the step S1, and the calculation mode is as follows:
wherein V is denoised Representing the preprocessed video image sequence; v (V) denoised (x, y, t) is the denoised pixel value of the image of the t frame of the video at (x, y); x represents the image abscissa, x=1, 2, …, X represents the image width, Y represents the image ordinate, y=1, 2, …, Y represents the image height, T represents the video time frame number, t=1, 2, …, T represents the sequence length; w (i, j, k) is the weight between the pixel (x+ i.y +j, t+k) and the target pixel (x, y, t); v (x+ i.y +j, t+k) is the pixel value of the video at (x+i, y+j) of the t+k frame image; the calculation mode of w (i, j, k) is as follows:
wherein V (x.y, t) is the pixel value of the image of the t frame of the video at (x, y); e represents a natural constant; sigma represents a parameter controlling the attenuation of the weight;
(i, j, k) is an element in the set N (x.y, t), N (x.y, t) representing a neighborhood position centered on (x.y, t), expressed as:
N(x.y，t)＝{(i，j，k)|i∈[-r，r]，j∈[-r，r]，k∈[r，r]}
wherein r represents the neighborhood radius; i, j and k are integers.
3. The intelligent recognition method for abnormal behaviors based on the security robot according to claim 2, wherein the step S2 includes the following steps:
s21: calculating a frame difference image:
extracting frame difference images of two adjacent frames for the preprocessed image sequence; for the t frame and the t+1st frame in the preprocessed image sequence, the calculation mode of the pixel value of the frame difference image is as follows:
wherein I is diff (x, y, t) represents a t-th frame difference image I diff (t) a value at a spatial location (x.y);
s22: gradient operation:
gradient information G of frame difference image calculated by Sobel operator 1 And G 1 ：
Wherein G is 1 (x, y, t) and G 2 (x, y, t) represents the values of the gradients of the frame difference image in the horizontal and vertical directions at the positions (x.y, t), respectively; k (K) 1 And K 2 Is the convolution kernel of the Sobel operator;
s23: motion feature extraction:
calculating the motion energy density E of the frame difference image:
E(x，y，t)＝G 1 (x，y，t) 2 +G 2 (x，y，t) 2
wherein E (x, y, t) represents the kinetic energy density value at location (x.y, t);
calculating the average motion intensity Mean of the frame difference image motion ：
Wherein, mean motion (t) represents the average motion intensity of the t-th frame difference image; num represents the number of pixels of the frame difference image of each frame;
calculating maximum motion intensity Max of frame difference image motion ：
Max motion (t)＝maximum x，y E(x，y，t)
Wherein the maximum function calculates the maximum value.
4. The intelligent recognition method for abnormal behaviors based on the security robot according to claim 3, wherein the step S3 includes the following steps:
s31: extracting depth features of the frame difference image:
at time frame number t, the frame difference image is converted into a feature vector F (t), which is extracted by using VGG-16 network:
F(t)＝VGG16(I diff (t))
wherein I is diff (t) represents a t-th frame difference image;
s32: building an abnormal behavior recognition network external input:
combining the depth features with the motion features as external inputs to each time-step abnormal behavior recognition network:
input(t)＝[F(t)，Mean motion (t)，Max motion (t)]
s33: abnormal behavior recognition network hidden state update:
the update formula of the hidden state h (t) of the abnormal behavior recognition network at the time frame sequence number t is as follows:
wherein,and->Respectively representing a weight matrix and bias from an input layer to a hidden layer; />And->Respectively representing a weight matrix and bias from hidden layer to hidden layer; tanh represents a hyperbolic tangent function;
s34: output layer calculation:
where cls denotes the probability of different abnormal behavior predicted by the abnormal behavior recognition network, cls=cls 1 ,cls 2 ,…,cls M ]M is the number of abnormal behavior species;and->Respectively representing a weight matrix and bias from the hidden layer to the output layer; softmax represents the normalized exponential function;
s35: setting an optimization target of an abnormal behavior recognition network:
using the improved cross entropy loss as an optimization objective for the network, the expression is:
wherein m=1, 2, …, N; for the frame difference image data currently input to the abnormal behavior recognition network,is the true probability that the abnormal behavior is the m-th class; cls m Is the network prediction probability that the abnormal behavior is the m-th class; e is a smoothing parameter.
5. The intelligent recognition method for abnormal behaviors based on the security robot according to claim 4, wherein the step S4 includes the following steps:
calculating an updated motion term:
wherein θ b Parameters representing the abnormal behavior recognition network at the b-th update;represented at theta b A loss function gradient below; beta is an adjustment parameter; b=1, 2, …, B representing the total number of updates;
updating parameters of the abnormal behavior recognition network:
θ b+1 ＝θ b -α·v b
where α is the learning rate.
6. The intelligent recognition method for abnormal behaviors based on the security robot according to claim 5, wherein the step S5 comprises the following steps:
s51: initializing a population:
adding random numbers to the abnormal behavior recognition network parameters theta B which are updated in the step S4, and generating P candidate solutions as initial individuals of the population;
s52: mutation operation:
for each individual δd, three different individuals δd are randomly selected from the population 1 ，δd 2 ，δd 3 As reference vectors, d1+.d2+.d3+.d, d1, d2, d3 and d represent integers between 1 and P; structural variant delta' d ：
δ′ d ＝δ d1 +2·(δ d2 -δ d3 )
S53: crossover operation:
for each ofIndividual delta d Randomly selecting a dimension Q, q=1, 2, …, Q representing the number of abnormal behavior recognition network parameters, and constructing crossed individuals
Wherein random represents a random number within 0 to 1;
s54: selection operation:
will beAnd delta d Calculating an improved cross entropy loss value in the step S35 as an abnormal behavior recognition network parameter, and selecting individuals with smaller loss values to remain in the population;
s55: repeating the iteration;
S52-S54 are repeated until 20 iterations.
7. Intelligent recognition system based on security robot is to unusual action, characterized by comprising:
and a pretreatment module: collecting video data of the security robot, and preprocessing the collected video data;
motion feature extraction module: extracting motion characteristics from the preprocessed image sequence based on a frame difference method;
abnormal behavior recognition module: constructing an abnormal behavior recognition network by using a cyclic neural network, and setting an optimization target of the abnormal behavior recognition network;
parameter optimization module: optimizing parameters of the abnormal behavior recognition network based on an improved random gradient descent algorithm;
and a fine adjustment module: performing fine adjustment on abnormal behavior recognition network parameters by using a differential evolution algorithm;
to realize the intelligent recognition method of abnormal behaviors based on the security robot according to any one of claims 1-6.
CN202311563231.9A 2023-11-22 2023-11-22 Intelligent recognition method and system for abnormal behaviors based on security robot Active CN117541991B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311563231.9A CN117541991B (en) 2023-11-22 2023-11-22 Intelligent recognition method and system for abnormal behaviors based on security robot

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311563231.9A CN117541991B (en) 2023-11-22 2023-11-22 Intelligent recognition method and system for abnormal behaviors based on security robot

Publications (2)

Publication Number Publication Date
CN117541991A true CN117541991A (en) 2024-02-09
CN117541991B CN117541991B (en) 2024-06-14

Family

ID=89795402

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311563231.9A Active CN117541991B (en) 2023-11-22 2023-11-22 Intelligent recognition method and system for abnormal behaviors based on security robot

Country Status (1)

Country Link
CN (1) CN117541991B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111369666A (en) * 2020-03-02 2020-07-03 中国电子科技集团公司第五十二研究所 Dynamic target reconstruction method and device based on multiple RGBD cameras
WO2021129569A1 (en) * 2019-12-25 2021-07-01 神思电子技术股份有限公司 Human action recognition method
WO2022252272A1 (en) * 2021-06-03 2022-12-08 江苏大学 Transfer learning-based method for improved vgg16 network pig identity recognition
KR102529876B1 (en) * 2022-11-01 2023-05-09 한밭대학교 산학협력단 A Self-Supervised Sampler for Efficient Action Recognition, and Surveillance Systems with Sampler
US20230316763A1 (en) * 2022-04-01 2023-10-05 Active Intelligence Corp Few-shot anomaly detection

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021129569A1 (en) * 2019-12-25 2021-07-01 神思电子技术股份有限公司 Human action recognition method
CN111369666A (en) * 2020-03-02 2020-07-03 中国电子科技集团公司第五十二研究所 Dynamic target reconstruction method and device based on multiple RGBD cameras
WO2022252272A1 (en) * 2021-06-03 2022-12-08 江苏大学 Transfer learning-based method for improved vgg16 network pig identity recognition
US20230316763A1 (en) * 2022-04-01 2023-10-05 Active Intelligence Corp Few-shot anomaly detection
KR102529876B1 (en) * 2022-11-01 2023-05-09 한밭대학교 산학협력단 A Self-Supervised Sampler for Efficient Action Recognition, and Surveillance Systems with Sampler

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
张海民;: "深度学习下智慧社区视频监控异常识别方法", 西安工程大学学报, no. 02, 31 December 2020 (2020-12-31), pages 106 - 112 *

Also Published As

Publication number Publication date
CN117541991B (en) 2024-06-14

Similar Documents

Publication Publication Date Title
CN110660082B (en) Target tracking method based on graph convolution and trajectory convolution network learning
CN112396027B (en) Vehicle re-identification method based on graph convolution neural network
CN112767451B (en) Crowd distribution prediction method and system based on double-current convolutional neural network
CN109558811B (en) Motion recognition method based on motion foreground attention and unsupervised key frame extraction
CN111932583A (en) Space-time information integrated intelligent tracking method based on complex background
CN110322445B (en) Semantic segmentation method based on maximum prediction and inter-label correlation loss function
CN110084201B (en) Human body action recognition method based on convolutional neural network of specific target tracking in monitoring scene
Doulamis Adaptable deep learning structures for object labeling/tracking under dynamic visual environments
CN110222636B (en) Pedestrian attribute identification method based on background suppression
CN112464851A (en) Smart power grid foreign matter intrusion detection method and system based on visual perception
CN116343330A (en) Abnormal behavior identification method for infrared-visible light image fusion
CN106504273B (en) Improved method based on GMM moving object detection
CN106780560A (en) A kind of feature based merges the bionic machine fish visual tracking method of particle filter
CN108537825B (en) Target tracking method based on transfer learning regression network
CN117193121B (en) Control system of coating machine die head
CN117422717B (en) Intelligent mask stain positioning method and system
CN117854143B (en) Abnormal behavior identification method and system based on intelligent door lock
CN108280408B (en) Crowd abnormal event detection method based on hybrid tracking and generalized linear model
CN117541991B (en) Intelligent recognition method and system for abnormal behaviors based on security robot
CN117409347A (en) ESNN-based early fire detection method
ELBAŞI et al. Control charts approach for scenario recognition in video sequences
CN107564029A (en) Moving target detecting method based on the filtering of Gauss extreme value and the sparse RPCA of group
CN114943873A (en) Method and device for classifying abnormal behaviors of construction site personnel
CN114694090A (en) Campus abnormal behavior detection method based on improved PBAS algorithm and YOLOv5
CN113379802A (en) Multi-feature adaptive fusion related filtering target tracking method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant