CN112149618A - Pedestrian abnormal behavior detection method and device suitable for inspection vehicle - Google Patents

Pedestrian abnormal behavior detection method and device suitable for inspection vehicle Download PDF

Info

Publication number
CN112149618A
CN112149618A CN202011094116.8A CN202011094116A CN112149618A CN 112149618 A CN112149618 A CN 112149618A CN 202011094116 A CN202011094116 A CN 202011094116A CN 112149618 A CN112149618 A CN 112149618A
Authority
CN
China
Prior art keywords
joint
behavior
node
space
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011094116.8A
Other languages
Chinese (zh)
Other versions
CN112149618B (en
Inventor
贾立东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ziqing Zhixing Technology Beijing Co ltd
Original Assignee
Ziqing Zhixing Technology Beijing Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ziqing Zhixing Technology Beijing Co ltd filed Critical Ziqing Zhixing Technology Beijing Co ltd
Priority to CN202011094116.8A priority Critical patent/CN112149618B/en
Publication of CN112149618A publication Critical patent/CN112149618A/en
Application granted granted Critical
Publication of CN112149618B publication Critical patent/CN112149618B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Human Computer Interaction (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a pedestrian abnormal behavior detection method and device suitable for a patrol vehicle, wherein the method comprises the following steps: presetting an abnormal behavior detection network, and carrying out online detection on abnormal behaviors in video data acquired from an RGB-D camera and a monitoring camera; the abnormal behavior detection network acquisition method comprises the following steps: step 1, calibrating behavior types and attributes existing in original video data to obtain a type label A ═ a corresponding to each behavior1，a2，...，anAnd attribute label B ═ Bnor，bab}; step 2, selecting a continuous T frame image sequence as training sample data; step 3, establishing an undirected graph G ═ V, E; set of joint node positions V ═ { V ═ VtiI T1, T, i 1, N, including all joints in a sequence of consecutive T frame imagesThe position of the node, the edge set E has two subsets which are respectively a space edge subset and a time sequence edge subset; and 4, acquiring an abnormal behavior detection network by using the time-space graph convolutional network and the convolutional neural network according to the undirected graph G (V, E). The invention can accurately detect abnormal behaviors all day long, and provides basis for intelligent monitoring and early warning.

Description

Pedestrian abnormal behavior detection method and device suitable for inspection vehicle
Technical Field
The invention relates to the technical field of robots and video monitoring, in particular to a pedestrian abnormal behavior detection method and device based on an RGB-D camera and suitable for a patrol vehicle.
Background
The abnormal behavior detection means that under a video monitoring scene, monitoring personnel are replaced or supplemented based on various computer equipment to distinguish abnormal behaviors of human beings under the video monitoring scene in real time, so that the early warning purpose is achieved. The abnormal behaviors generally refer to behaviors which are obviously different from other human behaviors in a scene or are low in occurrence probability of human beings in the scene. In fact, the probability of abnormal behavior is very small compared to normal behavior, but the abnormal behavior has almost serious consequences. Therefore, it is necessary to monitor and warn abnormal behaviors.
The robot technology can gradually replace some monotonous and dangerous work scenes which need human beings, thereby greatly reducing the labor cost. At present, the mode that the control early warning generally adopted vision camera to combine artifical control carries out, but these two kinds of monitoring methods all have the weakening and the inconvenience of different degrees night, and its defect mainly manifests: the perception capability of human and ordinary visible light cameras is reduced in the night environment. There is currently no particularly good solution for this situation.
Behavior recognition is an important class of computer vision tasks that have wide application in the fields of video understanding, motion analysis, and intelligent monitoring. Kinect is a civilian RGB-D camera from microsoft, which is capable of simultaneously acquiring color images and depth images, and is equipped with an infrared device, which is capable of operating in a nighttime environment, and is a typical 3D object recognition device.
Disclosure of Invention
It is an object of the present invention to provide a method and apparatus for detecting abnormal behavior of pedestrians suitable for patrolling vehicles based on RGB-D cameras to overcome or at least alleviate at least one of the above-mentioned drawbacks of the prior art.
In order to achieve the purpose, the invention provides a pedestrian abnormal behavior detection method suitable for a patrol vehicle, which is characterized in that an abnormal behavior detection network is preset to carry out online detection on abnormal behaviors in video data collected by an RGB-D camera and a monitoring camera; the abnormal behavior detection network acquisition method comprises the following steps:
step 1, calibrating behavior types and attributes existing in original video data to obtain a type label A ═ a corresponding to each behavior1,a2,...,anAnd attribute label B ═ Bnor,bab}; wherein the raw video data comprises RGB-D camera data, anClass label representing nth behavior, bnorIndicating normal behavior, babRepresenting abnormal behavior;
step 2, selecting the continuous T frame image sequence calibrated in the step 1 as training sample data;
step 3, acquiring a human body position frame in each frame of image of training sample data, and establishing an undirected graph G (V, E);
wherein the set of joint node positions V ═ { V ═ VtiI T1, T, i 1, N includes the positions of all joint nodes in a sequence of consecutive T frame images, vtiThe position of the ith joint node in the t frame image is shown, and N joint nodes are shared;
the edge set E has two subsets, which are respectively a spatial edge subset and a temporal edge subset, wherein: the spatial edge subset ES＝{vtivtjL (i, j) belongs to H, wherein H represents the positions of all joint nodes in the single-frame image; time sequence edge subset E consisting of edges formed by connecting same joint nodes on two adjacent frames of imagesF＝{vtiv(t+1)i}，vtjIndicating the position of the jth joint node in the tth frame image, v(t+1)iRepresenting the position of the jth joint node in the t +1 th frame image;
and 4, acquiring an abnormal behavior detection network by using the time-space graph convolutional network and the convolutional neural network according to the undirected graph G (V, E).
Further, step 4 comprises:
step 41, performing space-time space position modeling by using a space-time graph convolution network, which specifically comprises:
step 411, performing a space-time graph convolution operation on the undirected graph G ═ V, E,adopting convolution kernel as K x K, output and joint node vtiDirectly connected local feature map fout(vti)：
Figure BDA0002723092670000021
In the formula (1), B (v)ti) Corresponding representation vtiSet of neighbor nodes of, w (v)ti,vtj) Is a weight function, Zti(vtj) Is a regular function term, p (v)ti,vtj) Representing a sampling function, representing the minimum distance between two points having a relationship in the joint points, and a weighting function used for representing different weights assigned to adjacent joint points;
step 412, for the local feature map f output in step 411out(vti) Adopting a formula (2) to divide the joint node space configuration to obtain the local limbs of the human body formed by the joint nodes;
Figure BDA0002723092670000022
in the formula (2), riRepresents the average distance r from the ith joint node to the center of gravity of the human body where the ith joint node is locatedjRepresents the average distance between the j-th joint node and the center of gravity of the human body where the j-th joint node is located, iti(vtj) And expressing the calculated joint node space configuration, and dividing each joint node and the surrounding neighbors thereof into a root node subset, a centripetal group subset and a centrifugal group subset, wherein the root node subset consists of the root node, the centripetal group subset consists of nodes closer to the body gravity center node than the root node in the neighbor nodes, and the centrifugal group subset consists of nodes farther away from the body key node than the root node in the neighbor nodes.
Further, step 4 further comprises:
step 42, performing and acquiring the classified behavior by using a space-time graph convolutional network, which specifically includes:
step 421, using the output of step 411 and the joint node vtiDirectly connected local feature map fout(vti) And the spatial configuration obtained by the division in the step 412, a single-layer spatiotemporal graph convolution operation provided by the formula (3) is operated on each frame of image, and skeleton action characteristic classification is obtained;
Figure BDA0002723092670000031
in the formula (3), foutRepresenting a skeletal motion feature classification, finRepresenting the node output characteristic diagram output in step 411 and the spatial configuration obtained by dividing in step 412,
Figure BDA0002723092670000032
determined by the adjacency matrix a and the identity matrix I of the single-frame image,
Figure BDA0002723092670000033
an adjacency matrix represented throughout undirected graph G ═ V, E,
Figure BDA0002723092670000034
an identity matrix representing the entire undirected graph G ═ V, E, W is a weight vector W (V, E) formed by the aforementioned multipathsti,vtj) A superimposed weight matrix;
step 422, after performing multiple layers of space-time graph convolution operations such as in steps 411, 412 and 421, finally outputting the final behavior classification result through a Softmax function.
Further, step 4 further comprises:
step 43 of outputting a behavior sequence f for the classification corresponding to the joint node position set in the previous T/2 frame image in the undirected graph G ═ V, E created in step 3outEach behavior in the sequence is coded into a unified vector xuniWill unify the vector xuniInputting into LSTM network for training, and outputting behavior prediction result
Figure BDA0002723092670000035
Step 44, will get throughOriginal behavior result STC identified by blank-map convolutional networkactionBehavior outcome LSTM predicted with LSTM networkactionPerforming fusion according to formula (4):
Figure BDA0002723092670000041
in the formula (4), if the difference between the two is smaller than the set threshold, the final result is the result after the original space-time convolution image, and if the difference between the two is larger than the set threshold, the intermediate value of the two prediction results is taken.
Further, the abnormal behavior detection network is preset in a cloud control platform, a vehicle-mounted controller and/or an intelligent terminal.
The invention also provides a pedestrian abnormal behavior detection device suitable for a patrol vehicle, which comprises an abnormal behavior detection network, wherein the abnormal behavior detection network is used for carrying out online detection on abnormal behaviors in video data acquired from an RGB-D camera and a monitoring camera, and an acquisition device of the abnormal behavior detection network comprises:
a calibration module, configured to calibrate behavior categories and attributes existing in original video data to obtain a category label a ═ a corresponding to each behavior1,a2,...,anAnd attribute label B ═ Bnor,bab}; wherein the raw video data comprises RGB-D camera data, anClass label representing nth behavior, bnorIndicating normal behavior, babRepresenting abnormal behavior;
a training sample data selecting module, configured to select the continuous T frame image sequence calibrated in step 1 as training sample data;
the undirected graph constructing device is used for acquiring a human body position frame in each frame of image of training sample data and establishing an undirected graph G (V, E);
wherein the set of joint node positions V ═ { V ═ VtiI T1, T, i 1, N includes the positions of all joint nodes in a sequence of consecutive T frame images, vtiThe position of the ith joint node in the t frame image is shown, and N joint nodes are shared;
the edge set E has two subsets, which are respectively a spatial edge subset and a temporal edge subset, wherein: the spatial edge subset ES＝{vtivtjL (i, j) belongs to H, wherein H represents the positions of all joint nodes in the single-frame image; time sequence edge subset E consisting of edges formed by connecting same joint nodes on two adjacent frames of imagesF＝{vtiv(t+1)i}，vtjIndicating the position of the jth joint node in the tth frame image, v(t+1)iRepresenting the position of the jth joint node in the t +1 th frame image;
and the abnormal behavior detection network offline training module is used for acquiring the abnormal behavior detection network by utilizing the time-space graph convolutional network and the convolutional neural network according to the undirected graph G (V, E).
Further, the abnormal behavior detection network offline training module includes:
the spatio-temporal spatial position modeling unit is used for performing spatio-temporal spatial position modeling by utilizing a spatio-temporal graph convolution network, and specifically comprises the following steps:
firstly, a space-time graph convolution operation is performed on an undirected graph G ═ V, E, a convolution kernel is adopted to be K multiplied by K, and an output and joint node VtiDirectly connected local feature map fout(vti)：
Figure BDA0002723092670000051
In the formula (1), B (v)ti) Corresponding representation vtiSet of neighbor nodes of, w (v)ti,vtj) Is a weight function, Zti(vtj) Is a regular function term, p (v)ti,vtj) Representing a sampling function, representing the minimum distance between two points having a relationship in the joint points, and a weighting function used for representing different weights assigned to adjacent joint points;
then, for the local feature map fout(vti) Adopting a formula (2) to divide the joint node space configuration to obtain the local limbs of the human body formed by the joint nodes;
Figure BDA0002723092670000052
in the formula (2), riRepresents the average distance r from the ith joint node to the center of gravity of the human body where the ith joint node is locatedjRepresents the average distance between the j-th joint node and the center of gravity of the human body where the j-th joint node is located, iti(vtj) And expressing the calculated joint node space configuration, and dividing each joint node and the surrounding neighbors thereof into a root node subset, a centripetal group subset and a centrifugal group subset, wherein the root node subset consists of the root node, the centripetal group subset consists of nodes closer to the body gravity center node than the root node in the neighbor nodes, and the centrifugal group subset consists of nodes farther away from the body key node than the root node in the neighbor nodes.
Further, the offline training module for the abnormal behavior detection network further includes:
the behavior classification unit is used for performing and acquiring classified behaviors by utilizing a space-time graph convolutional network, and specifically comprises the following steps:
firstly, the joint nodes v output by the space-time space position modeling unit are utilizedtiThe directly connected local feature graph and the space configuration obtained by division run single-layer space-time graph convolution operation on each frame image to obtain skeleton action feature classification;
then, after convolution operation of a plurality of layers of space-time space position modeling units and behavior classification units, finally, a final behavior classification result is output through a Softmax function;
a behavior prediction unit that outputs a behavior sequence f for classification corresponding to a set of joint node positions in the previous T/2-frame image in the undirected graph G ═ V, EoutEach behavior in the sequence is coded into a unified vector xuniWill unify the vector xuniInputting into LSTM network for training, and outputting behavior prediction result
Figure BDA0002723092670000061
FusionA unit for STC of original behavior result identified via space-time graph convolutional networkactionBehavior outcome LSTM predicted with LSTM networkactionPerforming fusion according to formula (4):
Figure BDA0002723092670000062
in the formula (4), if the difference between the two is smaller than the set threshold, the final result is the result after the original space-time convolution image, and if the difference between the two is larger than the set threshold, the intermediate value of the two prediction results is taken.
Further, the abnormal behavior detection network is preset in the cloud control platform, the vehicle-mounted controller and/or the intelligent terminal.
Furthermore, the intelligent terminal is a cruise machine trolley and comprises an RGB-D camera in signal connection with the abnormal behavior detection network.
Due to the adoption of the technical scheme, the invention has the following advantages: the invention can accurately detect abnormal behaviors all day long, and provides basis for intelligent monitoring and early warning.
Drawings
FIG. 1 is a block diagram of the abnormal behavior detection computation framework of the present invention.
Fig. 2 is a schematic view of a human body 18 joint point.
Fig. 3 is a schematic diagram of spatial configuration partitioning.
Detailed Description
The invention is described in detail below with reference to the figures and examples.
As shown in fig. 1, the method for detecting abnormal behavior of a pedestrian suitable for a patrol vehicle according to the embodiment of the present invention includes: presetting an abnormal behavior detection network, and carrying out online detection on abnormal behaviors in video data acquired from an RGB-D camera and a monitoring camera; the abnormal behavior detection network acquisition method comprises the following steps:
step 1, calibrating behavior types and attributes existing in original video data to obtain a type label A ═ a corresponding to each behavior1,a2,...,anAnd attribute label B ═ Bnor,bab}，anClass label representing nth behavior, bnorIndicating normal behavior, babIndicating abnormal behavior.
Wherein the raw video data comprises full-day RGB-D camera data and surveillance camera data based on the determined surveillance area.
In one embodiment, the calibration method adopted in step 1 is a method combining automatic algorithm calibration and human-aided calibration, and specifically includes:
step 11, automatic calibration: and automatically calibrating the original video data by adopting an AlphaPose algorithm, wherein the attitude confidence coefficient value range of the automatic calibration is [0,1 ]. When the attitude confidence value is 0, the automatic calibration result is completely unconfirmed or no usable attitude calibration result is generated; when the pose confidence value is 1, it indicates that the pose confidence value is confident.
Step 12, manual secondary calibration: and according to a given calibration strategy, carrying out secondary manual calibration on the automatic calibration result which does not meet one of the following conditions.
The first condition is that when the attitude confidence coefficient obtained by the automatic calibration algorithm is less than a preset threshold value Pα(such as P)α0.5), performing secondary calibration by hand;
the second condition, for night video (12 pm. to 6 am.), for night video is manually calibrated by randomly sampling 1 minute of video 1 time every 5 minutes.
Using a manual calibration method to classify the attributes of the well-calibrated behaviors into normal behaviors bnorAnd abnormal behavior babAnd constructing a behavior and attribute data set.
And 2, selecting the continuous T frame image sequence calibrated in the step 1 as training sample data.
And 3, acquiring a human body position frame in each frame of image of the training sample data, and establishing an undirected graph G (V, E). Wherein, the human body position frame can be obtained by adopting a pre-trained target detection model YOLO v 5.
UndirectionalIn (V, E), the joint node position set V ═ VtiI T1, T, i 1, N includes the positions of all joint nodes in a sequence of consecutive T frame images, vtiThe position of the ith joint in the t-th frame image is shown, and N joint nodes are shared.
In the undirected graph G ═ V, E, the set of edges E has two subsets, respectively a spatial edge subset and a temporal edge subset.
Wherein: the spatial edge subset ES＝{vtivtjAnd l (i, j) belongs to H, wherein H represents the positions of all joint nodes in the single-frame image. Time sequence edge subset E consisting of edges formed by connecting same joint nodes on two adjacent frames of imagesF＝{vtiv(t+1)i}。vtjIndicating the position of the jth joint node in the tth frame image, v(t+1)iIndicates the position of the jth joint node in the t +1 th frame image.
As shown in fig. 2, fig. 2 provides an embodiment illustrating an undirected graph G ═ V, E for a single frame image, where the total number of individual human body nodes is set to 18.
And 4, according to the undirected graph G, performing space-time space position modeling and acquiring classified behaviors by using a space-time graph convolutional network, and performing weight estimation on different behaviors by using a convolutional neural network module to judge whether the behaviors belong to abnormal behaviors, so as to acquire an abnormal behavior detection network.
In one embodiment, step 4 comprises:
step 41, performing space-time space position modeling by using a space-time graph convolution network, which specifically comprises:
step 411, performing a space-time graph convolution operation on the undirected graph G ═ V, E, and outputting the undirected graph G ═ VtiDirectly connected local feature maps.
Specifically, a feature map f is inputinIf the undirected graph G including nodes but having no connection between nodes is (V, E) (as shown in fig. 2), and the convolution kernel is K × K, the output and joint node V is obtainedtiDirectly connected local feature map fout(vti) (as shown in FIG. 3):
Figure BDA0002723092670000081
in the formula (1), B (v)ti) Corresponding representation vtiThe neighbor node set of (2); zti(vtj) Is a regular function term, here any machine learning L1 or L2 regular term can be; p (v)ti,vtj) The method comprises the steps of representing a sampling function, representing the minimum distance between two points with a relation in a joint point, and adopting any general sampling function, such as random particle filtering and the like; w (v)ti,vtj) Is a weight function for representing different weights assigned to neighboring joint points, wherein a neighboring joint point refers to a joint node directly connected or related to a certain joint point around it.
Step 412, for the joint node v output in step 411tiDirectly connected local feature map fout(vti) And (4) carrying out joint node space configuration division to obtain the human body local limb formed by the joint nodes.
Joint node spatial configuration partitioning has a variety of ways, such as uniform labeling, distance-based, and spatial configuration-based. Considering the applicable approach of the present invention, for the established joint node vtiDirectly connected local feature map fout(vti) Spatial configuration partitioning is performed as shown in fig. 3. Dividing each local joint point and its surrounding neighbors into three subsets, namely a root node subset, a centripetal group subset and a centrifugal group subset, wherein the root node subset is composed of the root node itself, the centripetal group subset is composed of nodes closer to a body gravity center node than the root node in the neighboring nodes, and the centrifugal group subset is composed of nodes farther away from the body gravity center node than the root node in the neighboring nodes. The spatial configuration division is carried out by adopting an equation (2):
Figure BDA0002723092670000082
in the formula (2), riRepresents the average distance between the ith joint node and the center of gravity of the human body (the position of the triangle in figure 3)，rjRepresents the average distance between the j-th joint node and the center of gravity of the human body where the j-th joint node is located, iti(vtj) The calculated spatial configuration of the joint nodes is shown, and as shown in fig. 2, the joint nodes 11, 12 and 13 in fig. 2 after calculation form the arms of the human body.
Step 42, performing and acquiring the classified behavior by using a space-time graph convolutional network, which specifically includes:
step 421, using the output of step 411 and the joint node vtiDirectly connected local feature map fout(vti) And step 412, the obtained spatial configuration is divided, and a single-layer space-time graph convolution operation is performed on each frame of image to obtain skeleton action feature classification.
Step 421 uses the output of step 411 and the joint node vtiDirectly connected local feature map fout(vti) The spatial configuration obtained by dividing in step 412 is represented by the following equation (3) by performing a single-layer space-time graph convolution operation on each frame of image:
Figure BDA0002723092670000091
in the formula (3), foutRepresenting a skeletal motion feature classification, finRepresenting the node output characteristic diagram output in step 411 and the spatial configuration obtained by dividing in step 412,
Figure BDA0002723092670000092
from contiguous matrices of a single frame image
Figure BDA0002723092670000093
And identity matrix
Figure BDA0002723092670000094
It is determined that,
Figure BDA0002723092670000095
an adjacency matrix represented throughout undirected graph G ═ V, E,
Figure BDA0002723092670000096
an identity matrix representing the entire undirected graph G ═ V, E, W is a weight vector W (V, E) formed by the aforementioned multipathsti,vtj) And stacking the weight matrixes.
Step 422, after performing space-time graph convolution operations in multiple layers (e.g. 5 layers) such as step 411, step 412 and step 421, finally outputting the final behavior classification result STC through the Softmax functionaction＝{act1,act2,...,actT}。
Step 43, based on the undirected graph G ═ V, E, the classified output behavior sequence f corresponding to the joint node position set in the previous T/2 frame image in the undirected graph G ═ V, E established in step 3 is outputoutEach behavior in the sequence is coded into a unified vector xuniWill unify the vector xuniInputting into LSTM (English is called Long Short-Term Memory, Chinese is called Long-time Memory network) network for training, and outputting behavior prediction result
Figure BDA0002723092670000097
Through step 43, the LSTM can model the long-term dependence in the obtained action sequence information, and can be used to solve the problem of behavior jump in the foregoing behavior recognition, and improve the robustness of the abnormal behavior detection network.
Step 44, identifying the original behavior result STC identified by the space-time graph convolution networkactionBehavior outcome with LSTM network prediction
Figure BDA0002723092670000098
Performing fusion according to formula (4):
Figure BDA0002723092670000101
in the formula (4), if the difference between the two is less than the set threshold value HthreIf the difference result between the two results is greater than the set threshold value, the two prediction results are takenAn intermediate value. Wherein the threshold value HthreThe normalized average behavior values for all normal behaviors in the calibrated behavior library.
The original behavior recognition result can be compared with the behavior prediction result through the step 44, optimization and enhancement are performed, and the problem of behavior jump easily occurring in behavior recognition is solved.
And step 45, classifying the normal behavior and the abnormal behavior of the final behavior recognition result.
And classifying the identified action sequence into normal behavior and abnormal behavior through a shallow convolutional neural network module according to a set rule, and giving an alarm if the abnormal behavior is output.
In one embodiment, the present invention further comprises:
step 5, the model pruning method specifically comprises the following steps:
step 51, evaluating the importance of the neuron parameters in the proposed abnormal behavior detection network, removing the neuron with the lowest importance, and then fine-tuning the abnormal behavior detection network.
And step 52, performing model pruning by adopting a group convolution pruning method according to the pre-trained YOLO v5 network, and then performing reasoning deployment on a cloud control platform, a vehicle-mounted controller and/or an intelligent terminal (cruise trolley).
The method provided by the embodiment of the invention firstly utilizes a convolution network to detect the human body area, then models the human skeleton key points into a graph convolution network, utilizes the identified skeleton key points to identify the human body behavior and detects the abnormal behavior. The matched LSTM enhancement algorithm can utilize the continuity of human behaviors to enhance the behavior identification accuracy.
In one embodiment, the pedestrian abnormal behavior detection method suitable for the patrol vehicle comprises the following steps: an abnormal behavior detection network is preset in the cloud control platform, the vehicle-mounted controller and/or the intelligent terminal (cruise trolley).
Taking a cruise trolley configured with an abnormal behavior detection network as an example, the method for detecting the abnormal behavior on line by the abnormal behavior detection network is described as follows:
the cruise trolley is used for routing inspection according to a specified route, and is provided with an RGB-D enhanced camera, behavior detection can be carried out by using RGB images and depth images generated in the daytime environment, and the behavior detection can be realized by using an infrared camera and a depth camera at night, so that abnormal behavior detection in all-day video data can be realized. When abnormal behaviors are detected, an audible and visual alarm mode can be adopted for early warning.
The invention also provides a pedestrian abnormal behavior detection device suitable for the inspection vehicle, which comprises an abnormal behavior detection network used for carrying out online detection on abnormal behaviors in video data collected by the RGB-D camera and the monitoring camera. The device for acquiring the abnormal behavior detection network comprises a calibration module, a training sample data selection module, an undirected graph construction device and an abnormal behavior detection network offline training module, wherein:
the calibration module is used for calibrating the behavior categories and attributes existing in the original video data to obtain the category label A ═ a corresponding to each behavior1,a2,...,anAnd attribute label B ═ Bnor,bab}; wherein the raw video data comprises RGB-D camera data, anClass label representing nth behavior, bnorIndicating normal behavior, babIndicating abnormal behavior.
And the training sample data selection module is used for selecting the continuous T frame image sequence calibrated in the step 1 as training sample data.
The undirected graph construction device is used for acquiring a human body position frame in each frame of image of training sample data and establishing an undirected graph G (V, E).
Wherein the set of joint node positions V ═ { V ═ VtiI T1, T, i 1, N includes the positions of all joint nodes in a sequence of consecutive T frame images, vtiThe position of the ith joint in the t-th frame image is shown, and N joint nodes are shared.
The edge set E has two subsets, which are respectively a spatial edge subset and a temporal edge subset, wherein: the spatial edge subset ES＝{vtivtjL (i, j) belongs to H, wherein H represents the positions of all joint nodes in the single-frame image; the same joint node on two adjacent frames of images is connectedTime-sequential edge subset E of formed edgesF＝{vtiv(t+1)i}，vtjIndicating the position of the jth joint node in the tth frame image, v(t+1)iIndicates the position of the jth joint node in the t +1 th frame image.
And the abnormal behavior detection network offline training module is used for acquiring the abnormal behavior detection network by utilizing the space-time graph convolutional network and the convolutional neural network according to the undirected graph G (V, E).
In one embodiment, the abnormal behavior detection network offline training module comprises a space-time space position modeling unit, a behavior classification unit, a behavior prediction unit and a fusion unit:
the space-time space position modeling unit is used for performing space-time space position modeling by utilizing a space-time graph convolution network, and specifically comprises the following steps:
first, a space-time graph convolution operation is performed on the undirected graph G ═ V, E, and a node feature graph is output.
And then, carrying out joint node space configuration division on the node characteristic diagram to obtain a human body local part formed by joint nodes.
The method for dividing the spatial configuration comprises the following steps:
dividing each joint node and its surrounding neighbors into a root node subset, a centripetal group subset and a centrifugal group subset, wherein the root node subset consists of the root node itself, the centripetal group subset consists of nodes closer to a body gravity center node than the root node in the neighbor nodes, and the centrifugal group subset consists of nodes farther from the body gravity center node than the root node in the neighbor nodes;
and a behavior classification unit. The method is used for performing and acquiring the classified behaviors by utilizing a space-time graph convolutional network, and specifically comprises the following steps:
firstly, a single-layer spatiotemporal graph convolution operation is operated on each frame image by utilizing the node output characteristic graph output by the spatiotemporal position modeling unit and the space configuration obtained by division to obtain skeleton action characteristic classification.
And finally, outputting a final behavior classification result through a Softmax function after convolution operation of a plurality of layers of space-time space position modeling units and behavior classification units.
A behavior prediction unit for 4 outputting a behavior sequence f for classification corresponding to the joint node position set in the previous T/2 frame image in the undirected graph G ═ V, EoutEach behavior in the sequence is coded into a unified vector xuniWill unify the vector xuniInputting into LSTM network for training, and outputting behavior prediction result
Figure BDA0002723092670000121
A fusion unit for STC of original behavior result identified via space-time graph convolution networkactionBehavior outcome LSTM predicted with LSTM networkactionPerforming fusion according to formula (4):
Figure BDA0002723092670000122
in the formula (4), if the difference between the two is smaller than the set threshold, the final result is the result after the original space-time convolution image, and if the difference between the two is larger than the set threshold, the intermediate value of the two prediction results is taken.
In one embodiment, the abnormal behavior detection network is preset in the cloud control platform, the vehicle-mounted controller and/or the intelligent terminal, and is used for performing online detection on abnormal behaviors existing in video data acquired by the RGB-D camera and the monitoring camera.
In one embodiment, the intelligent terminal is a cruise machine trolley and comprises an RGB-D camera and an abnormal behavior detection network in signal connection with the RGB-D camera. The abnormal behaviors in the patrol environment can be detected all day long through the RGB-D camera in the cruise machine trolley and the abnormal behavior detection network.
Finally, it should be pointed out that: the above examples are only for illustrating the technical solutions of the present invention, and are not limited thereto. Those of ordinary skill in the art will understand that: modifications can be made to the technical solutions described in the foregoing embodiments, or some technical features may be equivalently replaced; such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. A pedestrian abnormal behavior detection method suitable for a patrol vehicle is characterized in that an abnormal behavior detection network is preset to carry out online detection on abnormal behaviors existing in video data collected by an RGB-D camera and a monitoring camera; the abnormal behavior detection network acquisition method comprises the following steps:
step 1, calibrating behavior types and attributes existing in original video data to obtain a type label A ═ a corresponding to each behavior1,a2,...,anAnd attribute label B ═ Bnor,bab}; wherein the raw video data comprises RGB-D camera data, anClass label representing nth behavior, bnorIndicating normal behavior, babRepresenting abnormal behavior;
step 2, selecting the continuous T frame image sequence calibrated in the step 1 as training sample data;
step 3, acquiring a human body position frame in each frame of image of training sample data, and establishing an undirected graph G (V, E);
wherein the set of joint node positions V ═ { V ═ VtiI T1, T, i 1, N includes the positions of all joint nodes in a sequence of consecutive T frame images, vtiThe position of the ith joint node in the t frame image is shown, and N joint nodes are shared;
the edge set E has two subsets, which are respectively a spatial edge subset and a temporal edge subset, wherein: the spatial edge subset ES＝{vtivtjL (i, j) belongs to H, wherein H represents the positions of all joint nodes in the single-frame image; time sequence edge subset E consisting of edges formed by connecting same joint nodes on two adjacent frames of imagesF＝{vtiv(t+1)i}，vtjIndicating the position of the jth joint node in the tth frame image, v(t+1)iRepresenting the position of the jth joint node in the t +1 th frame image;
and 4, acquiring an abnormal behavior detection network by using the time-space graph convolutional network and the convolutional neural network according to the undirected graph G (V, E).
2. The pedestrian abnormal behavior detection method suitable for a patrol vehicle according to claim 1, wherein step 4 includes:
step 41, performing space-time space position modeling by using a space-time graph convolution network, which specifically comprises:
step 411, performing a space-time graph convolution operation on the undirected graph G ═ V, E, with a convolution kernel K × K, and outputting the undirected graph and the joint node VtiDirectly connected local feature map fout(vti)：
Figure FDA0002723092660000011
In the formula (1), B (v)ti) Corresponding representation vtiSet of neighbor nodes of, w (v)ti,vtj) Is a weight function, Zti(vtj) Is a regular function term, p (v)ti,vtj) Representing a sampling function, representing the minimum distance between two points having a relationship in the joint points, and a weighting function used for representing different weights assigned to adjacent joint points;
step 412, for the local feature map f output in step 411out(vti) Adopting a formula (2) to divide the joint node space configuration to obtain the local limbs of the human body formed by the joint nodes;
Figure FDA0002723092660000021
in the formula (2), riRepresents the average distance r from the ith joint node to the center of gravity of the human body where the ith joint node is locatedjRepresents the average distance between the j-th joint node and the center of gravity of the human body where the j-th joint node is located, iti(vtj) Representing the space configuration of the joint nodes obtained by calculation, and dividing each joint node and the surrounding neighbors thereof into a root node subset and a centripetal group subsetAnd a centrifugal group subset, wherein the root node subset consists of the root node itself, the centripetal group subset consists of nodes closer to the body gravity center node than the root node among the neighbor nodes, and the centrifugal group subset consists of nodes farther from the body gravity center node than the root node among the neighbor nodes.
3. The method for detecting the abnormal behavior of the pedestrian adapted to the patrol vehicle according to claim 2, wherein the step 4 further comprises:
step 42, performing and acquiring the classified behavior by using a space-time graph convolutional network, which specifically includes:
step 421, using the output of step 411 and the joint node vtiDirectly connected local feature map fout(vti) And the spatial configuration obtained by the division in the step 412, a single-layer spatiotemporal graph convolution operation provided by the formula (3) is operated on each frame of image, and skeleton action characteristic classification is obtained;
Figure FDA0002723092660000022
in the formula (3), foutRepresenting a skeletal motion feature classification, finRepresenting the node output characteristic diagram output in step 411 and the spatial configuration obtained by dividing in step 412,
Figure FDA0002723092660000023
determined by the adjacency matrix a and the identity matrix I of the single-frame image,
Figure FDA0002723092660000024
an adjacency matrix represented throughout undirected graph G ═ V, E,
Figure FDA0002723092660000025
an identity matrix representing the entire undirected graph G ═ V, E, W is a weight vector W (V, E) formed by the aforementioned multipathsti,vtj) A superimposed weight matrix;
step 422, after performing multiple layers of space-time graph convolution operations such as in steps 411, 412 and 421, finally outputting the final behavior classification result through a Softmax function.
4. The pedestrian abnormal behavior detection method suitable for the patrol vehicle according to claim 2 or 3, wherein the step 4 further includes:
step 43 of outputting a behavior sequence f for the classification corresponding to the joint node position set in the previous T/2 frame image in the undirected graph G ═ V, E created in step 3outEach behavior in the sequence is coded into a unified vector xuniWill unify the vector xuniInputting into LSTM network for training, and outputting behavior prediction result
Figure FDA0002723092660000031
Step 44, identifying the original behavior result STC identified by the space-time graph convolution networkactionBehavior outcome LSTM predicted with LSTM networkactionPerforming fusion according to formula (4):
Figure FDA0002723092660000032
in the formula (4), if the difference between the two is smaller than the set threshold, the final result is the result after the original space-time convolution image, and if the difference between the two is larger than the set threshold, the intermediate value of the two prediction results is taken.
5. The pedestrian abnormal behavior detection method suitable for the patrol vehicle according to claim 4, wherein the abnormal behavior detection network is preset in a cloud control platform, a vehicle-mounted controller and/or an intelligent terminal.
6. The utility model provides a pedestrian's unusual action detection device suitable for patrol and examine car which characterized in that, includes unusual action detection network for carry out online detection to the unusual action that exists in the video data of gathering in RGB-D camera and the surveillance camera, the acquisition device of unusual action detection network includes:
a calibration module, configured to calibrate behavior categories and attributes existing in original video data to obtain a category label a ═ a corresponding to each behavior1,a2,...,anAnd attribute label B ═ Bnor,bab}; wherein the raw video data comprises RGB-D camera data, anClass label representing nth behavior, bnorIndicating normal behavior, babRepresenting abnormal behavior;
a training sample data selecting module, configured to select the continuous T frame image sequence calibrated in step 1 as training sample data;
the undirected graph constructing device is used for acquiring a human body position frame in each frame of image of training sample data and establishing an undirected graph G (V, E);
wherein the set of joint node positions V ═ { V ═ VtiI T1, T, i 1, N includes the positions of all joint nodes in a sequence of consecutive T frame images, vtiThe position of the ith joint node in the t frame image is shown, and N joint nodes are shared;
the edge set E has two subsets, which are respectively a spatial edge subset and a temporal edge subset, wherein: the spatial edge subset ES＝{vtivtjL (i, j) belongs to H, wherein H represents the positions of all joint nodes in the single-frame image; time sequence edge subset E consisting of edges formed by connecting same joint nodes on two adjacent frames of imagesF＝{vtiv(t+1)i}，vtjIndicating the position of the jth joint node in the tth frame image, v(t+1)iRepresenting the position of the jth joint node in the t +1 th frame image;
and the abnormal behavior detection network offline training module is used for acquiring the abnormal behavior detection network by utilizing the time-space graph convolutional network and the convolutional neural network according to the undirected graph G (V, E).
7. The abnormal behavior detection device for pedestrians suitable for patrolling vehicles according to claim 6, wherein the abnormal behavior detection network offline training module comprises:
the spatio-temporal spatial position modeling unit is used for performing spatio-temporal spatial position modeling by utilizing a spatio-temporal graph convolution network, and specifically comprises the following steps:
firstly, a space-time graph convolution operation is performed on an undirected graph G ═ V, E, a convolution kernel is adopted to be K multiplied by K, and an output and joint node VtiDirectly connected local feature map fout(vti)：
Figure FDA0002723092660000041
In the formula (1), B (v)ti) Corresponding representation vtiSet of neighbor nodes of, w (v)ti,vtj) Is a weight function, Zti(vtj) Is a regular function term, p (v)ti,vtj) Representing a sampling function, representing the minimum distance between two points having a relationship in the joint points, and a weighting function used for representing different weights assigned to adjacent joint points;
then, for the local feature map fout(vti) Adopting a formula (2) to divide the joint node space configuration to obtain the local limbs of the human body formed by the joint nodes;
Figure FDA0002723092660000042
in the formula (2), riRepresents the average distance r from the ith joint node to the center of gravity of the human body where the ith joint node is locatedjRepresents the average distance between the j-th joint node and the center of gravity of the human body where the j-th joint node is located, iti(vtj) And expressing the calculated joint node space configuration, and dividing each joint node and the surrounding neighbors thereof into a root node subset, a centripetal group subset and a centrifugal group subset, wherein the root node subset consists of the root node, the centripetal group subset consists of nodes closer to the body gravity center node than the root node in the neighbor nodes, and the centrifugal group subset consists of nodes farther away from the body key node than the root node in the neighbor nodes.
8. The abnormal behavior detection device for pedestrians suitable for use in a patrol vehicle as claimed in claim 7, wherein the abnormal behavior detection network offline training module further comprises:
the behavior classification unit is used for performing and acquiring classified behaviors by utilizing a space-time graph convolutional network, and specifically comprises the following steps:
firstly, the joint nodes v output by the space-time space position modeling unit are utilizedtiThe directly connected local feature graph and the space configuration obtained by division run single-layer space-time graph convolution operation on each frame image to obtain skeleton action feature classification;
then, after convolution operation of a plurality of layers of space-time space position modeling units and behavior classification units, finally, a final behavior classification result is output through a Softmax function;
a behavior prediction unit that outputs a behavior sequence f for classification corresponding to a set of joint node positions in the previous T/2-frame image in the undirected graph G ═ V, EoutEach behavior in the sequence is coded into a unified vector xuniWill unify the vector xuniInputting into LSTM network for training, and outputting behavior prediction result
Figure FDA0002723092660000051
A fusion unit for STC of original behavior result identified via space-time graph convolution networkactionBehavior outcome LSTM predicted with LSTM networkactionPerforming fusion according to formula (4):
Figure FDA0002723092660000052
in the formula (4), if the difference between the two is smaller than the set threshold, the final result is the result after the original space-time convolution image, and if the difference between the two is larger than the set threshold, the intermediate value of the two prediction results is taken.
9. The method for detecting the abnormal behavior of the pedestrian, which is suitable for the patrol vehicle, according to claim 8, wherein the abnormal behavior detection network is preset in a cloud control platform, a vehicle-mounted controller and/or an intelligent terminal.
10. The device for detecting the abnormal behavior of the pedestrian according to claim 9, wherein the intelligent terminal is a cruise machine car, and comprises an RGB-D camera in signal connection with the abnormal behavior detection network.
CN202011094116.8A 2020-10-14 2020-10-14 Pedestrian abnormal behavior detection method and device suitable for inspection vehicle Active CN112149618B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011094116.8A CN112149618B (en) 2020-10-14 2020-10-14 Pedestrian abnormal behavior detection method and device suitable for inspection vehicle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011094116.8A CN112149618B (en) 2020-10-14 2020-10-14 Pedestrian abnormal behavior detection method and device suitable for inspection vehicle

Publications (2)

Publication Number Publication Date
CN112149618A true CN112149618A (en) 2020-12-29
CN112149618B CN112149618B (en) 2022-09-09

Family

ID=73951779

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011094116.8A Active CN112149618B (en) 2020-10-14 2020-10-14 Pedestrian abnormal behavior detection method and device suitable for inspection vehicle

Country Status (1)

Country Link
CN (1) CN112149618B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112801061A (en) * 2021-04-07 2021-05-14 南京百伦斯智能科技有限公司 Posture recognition method and system
CN113837306A (en) * 2021-09-29 2021-12-24 南京邮电大学 Abnormal behavior detection method based on human body key point space-time diagram model
CN115984787A (en) * 2023-03-20 2023-04-18 齐鲁云商数字科技股份有限公司 Intelligent vehicle-mounted real-time alarm method for industrial brain public transport

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1963824A (en) * 2006-11-17 2007-05-16 东华大学 Method for forecasting diameter of fibre of unwoven cloth based on extrusion method of polymer of NN
US20120314064A1 (en) * 2011-06-13 2012-12-13 Sony Corporation Abnormal behavior detecting apparatus and method thereof, and video monitoring system
CN110135319A (en) * 2019-05-09 2019-08-16 广州大学 A kind of anomaly detection method and its system
CN110796110A (en) * 2019-11-05 2020-02-14 西安电子科技大学 Human behavior identification method and system based on graph convolution network
CN110852152A (en) * 2019-09-27 2020-02-28 中山大学 Deep hash pedestrian re-identification method based on data enhancement
CN111597929A (en) * 2020-04-30 2020-08-28 青岛科技大学 Group behavior identification method based on channel information fusion and group relation space structured modeling
CN111723779A (en) * 2020-07-20 2020-09-29 浙江大学 Chinese sign language recognition system based on deep learning
CN111723729A (en) * 2020-06-18 2020-09-29 成都颜禾曦科技有限公司 Intelligent identification method for dog posture and behavior of surveillance video based on knowledge graph

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1963824A (en) * 2006-11-17 2007-05-16 东华大学 Method for forecasting diameter of fibre of unwoven cloth based on extrusion method of polymer of NN
US20120314064A1 (en) * 2011-06-13 2012-12-13 Sony Corporation Abnormal behavior detecting apparatus and method thereof, and video monitoring system
CN110135319A (en) * 2019-05-09 2019-08-16 广州大学 A kind of anomaly detection method and its system
CN110852152A (en) * 2019-09-27 2020-02-28 中山大学 Deep hash pedestrian re-identification method based on data enhancement
CN110796110A (en) * 2019-11-05 2020-02-14 西安电子科技大学 Human behavior identification method and system based on graph convolution network
CN111597929A (en) * 2020-04-30 2020-08-28 青岛科技大学 Group behavior identification method based on channel information fusion and group relation space structured modeling
CN111723729A (en) * 2020-06-18 2020-09-29 成都颜禾曦科技有限公司 Intelligent identification method for dog posture and behavior of surveillance video based on knowledge graph
CN111723779A (en) * 2020-07-20 2020-09-29 浙江大学 Chinese sign language recognition system based on deep learning

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
SIJIE YAN等: "Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition", 《ARXIV:1801.07455V2》 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112801061A (en) * 2021-04-07 2021-05-14 南京百伦斯智能科技有限公司 Posture recognition method and system
CN113837306A (en) * 2021-09-29 2021-12-24 南京邮电大学 Abnormal behavior detection method based on human body key point space-time diagram model
CN113837306B (en) * 2021-09-29 2024-04-12 南京邮电大学 Abnormal behavior detection method based on human body key point space-time diagram model
CN115984787A (en) * 2023-03-20 2023-04-18 齐鲁云商数字科技股份有限公司 Intelligent vehicle-mounted real-time alarm method for industrial brain public transport

Also Published As

Publication number Publication date
CN112149618B (en) 2022-09-09

Similar Documents

Publication Publication Date Title
CN112149618B (en) Pedestrian abnormal behavior detection method and device suitable for inspection vehicle
US11037005B2 (en) Method and apparatus for identifying traffic light
KR102129893B1 (en) Ship tracking method and system based on deep learning network and average movement
KR101607224B1 (en) Dynamic object classification
Chen et al. A UAV-based forest fire detection algorithm using convolutional neural network
Qu et al. Moving vehicle detection with convolutional networks in UAV videos
CN111491093B (en) Method and device for adjusting field angle of camera
CN113228043A (en) System and method for obstacle detection and association of mobile platform based on neural network
CN110852222A (en) Campus corridor scene intelligent monitoring method based on target detection
Zhao et al. Fusion of Velodyne and camera data for scene parsing
CN115761537B (en) Power transmission line foreign matter intrusion identification method oriented to dynamic feature supplementing mechanism
CN114898322A (en) Driving environment identification method and device, vehicle and storage medium
CN115147745A (en) Small target detection method based on urban unmanned aerial vehicle image
CN116052082A (en) Power distribution station room anomaly detection method and device based on deep learning algorithm
CN116846059A (en) Edge detection system for power grid inspection and monitoring
CN115546742A (en) Rail foreign matter identification method and system based on monocular thermal infrared camera
CN113378638B (en) Method for identifying abnormal behavior of turbine operator based on human body joint point detection and D-GRU network
CN109740527A (en) Image processing method in a kind of video frame
KR20220153247A (en) System for monitoring black ice
CN117115752A (en) Expressway video monitoring method and system
CN110334703B (en) Ship detection and identification method in day and night image
CN117218380A (en) Dynamic target detection tracking method for unmanned ship remote sensing image
CN114821042A (en) R-FCN disconnecting link detection method combining local features and global features
KR20220067271A (en) Image acquisition apparatus and image acquisition method
Lulio et al. JSEG-based image segmentation in computer vision for agricultural mobile robot navigation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant