CN114140832A - Method and device for detecting pedestrian boundary crossing risk in well, electronic equipment and storage medium - Google Patents
Method and device for detecting pedestrian boundary crossing risk in well, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114140832A CN114140832A CN202210113490.0A CN202210113490A CN114140832A CN 114140832 A CN114140832 A CN 114140832A CN 202210113490 A CN202210113490 A CN 202210113490A CN 114140832 A CN114140832 A CN 114140832A
- Authority
- CN
- China
- Prior art keywords
- pedestrian
- human body
- body posture
- algorithm
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20076—Probabilistic image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method and a device for detecting the crossing risk of underground pedestrians, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring an underground pedestrian monitoring video image in real time; performing frame cutting processing on the video image to obtain an image to be processed, wherein the image to be processed comprises: a plurality of pedestrian images; identifying the plurality of pedestrian images based on a human body posture estimation algorithm to generate human body posture information of each pedestrian; tracking the multiple pedestrian images based on a multi-target pedestrian tracking algorithm to generate a motion track of each pedestrian; and performing coordinate comparison operation according to the human body posture information, the motion trail and the preset dangerous area of each pedestrian, and determining whether the pedestrian has the boundary crossing risk according to the coordinate comparison operation result. By the method and the device, the underground pedestrian boundary crossing detection can be effectively carried out, real-time accurate early warning can be provided for the boundary crossing detection of the pedestrian danger area in the fully mechanized mining face, and the working safety of coal mine workers can be better ensured.
Description
Technical Field
The invention relates to the technical field of mine safety, in particular to a method and a device for detecting pedestrian boundary crossing risks in a mine, electronic equipment and a storage medium.
Background
With the continuous improvement of the production automation degree of a mining area, more and more video monitoring systems are deployed to key safety areas in the underground, such as a coal mining area of a working face, a water pump room, a support big foot and the like. The high-definition video monitoring system can provide abundant reporting reference information after an accident occurs. However, the passive method of obtaining evidence afterwards does not fully exert the active recognition and alarm effect of the video monitoring system, only records the video information, and does not further process and analyze the video information, and the underground automatic monitoring and management level is still weak.
In order to better utilize the existing hardware deployment to obtain rich monitoring management information, an underground intelligent monitoring and mine pedestrian crossing detection method based on video analysis is developed. Video-based pedestrian target detection methods can be currently divided into two major categories: one based on background modeling and the other based on statistical learning. The background modeling method is based on the existing public pedestrian sample library by utilizing a foreground-background segmentation technology, is simple and convenient to calculate, has an intuitive effect, and is easier to realize real-time processing, and the method is typically Gaussian mixture background modeling. However, in a mine environment, the problems of much noise, low angle position of a camera, dark overall illumination, complex light change and the like often exist, so that the existing public pedestrian sample library cannot be directly applied to underground pedestrians, and an insufficient amount of underground pedestrian samples do not exist. The statistical learning method is good in accuracy, and the key point is that the selection of features and the sample training of a classifier can be efficiently described, but due to the fact that the calculation amount of feature extraction and image sliding window traversal is too large, the real-time requirement is difficult to guarantee.
That is, there is no effective detection scheme for the cross-border risk of the pedestrian in the well, which is a problem to be solved urgently.
Disclosure of Invention
In view of the above, the present invention provides a method, an apparatus, an electronic device and a storage medium for detecting a pedestrian boundary crossing risk in a well, so as to solve at least one of the above-mentioned problems.
According to a first aspect of the invention, there is provided a method of downhole pedestrian out-of-range risk detection, the method comprising:
acquiring an underground pedestrian monitoring video image in real time;
performing frame cutting processing on the video image to obtain an image to be processed, wherein the image to be processed comprises: a plurality of pedestrian images;
identifying the plurality of pedestrian images based on a human body posture estimation algorithm to generate human body posture information of each pedestrian;
tracking the multiple pedestrian images based on a multi-target pedestrian tracking algorithm to generate a motion track of each pedestrian;
and performing coordinate comparison operation according to the human body posture information, the motion trail and the preset dangerous area of each pedestrian, and determining whether the pedestrian has the boundary crossing risk according to the coordinate comparison operation result.
According to a second aspect of the present invention, there is provided a downhole pedestrian crossing risk detection apparatus, the apparatus comprising:
the video acquisition unit is used for acquiring a pedestrian monitoring video image in the well in real time;
a to-be-processed image generating unit, configured to perform frame cutting processing on the video image to obtain a to-be-processed image, where the to-be-processed image includes: a plurality of pedestrian images;
a human body posture information generating unit for performing recognition processing on the plurality of pedestrian images based on a human body posture estimation algorithm to generate human body posture information of each pedestrian;
the motion trail generation unit is used for tracking the multiple pedestrian images based on a multi-target pedestrian tracking algorithm so as to generate motion trails of all pedestrians;
and the judging unit is used for carrying out coordinate comparison operation according to the human body posture information, the motion trail and the preset danger area of each pedestrian and determining whether the pedestrian has the boundary crossing risk or not according to the coordinate comparison operation result.
According to a third aspect of the present invention, there is provided an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method when executing the program.
According to a fourth aspect of the invention, a computer-readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method.
According to the technical scheme, the obtained underground pedestrian monitoring video image is subjected to frame cutting processing to obtain an image to be processed, then a plurality of pedestrian images in the image to be processed are identified based on a human body posture estimation algorithm to generate human body posture information of each pedestrian, meanwhile a plurality of pedestrian images are tracked based on a multi-target pedestrian tracking algorithm to generate a motion track of each pedestrian, then coordinate comparison operation is carried out according to the human body posture information, the motion track and a preset danger area of each pedestrian, whether the pedestrian has boundary crossing risks or not is determined according to the coordinate comparison operation result, therefore, underground pedestrian boundary crossing detection can be effectively carried out, real-time accurate early warning can be provided for the boundary crossing detection of the pedestrian danger area in the fully mechanized mining working face, and the working safety of coal mine workers can be well ensured.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a flow chart of a method of downhole pedestrian out-of-bounds risk detection according to an embodiment of the invention;
FIG. 2 is a detailed flow chart of a method of downhole pedestrian boundary crossing risk detection according to an embodiment of the present invention;
fig. 3(1) -fig. 3(5) are exemplary diagrams of human body recognition according to an embodiment of the present invention;
FIG. 4 is a block diagram of a downhole pedestrian crossing risk detection device according to an embodiment of the invention;
fig. 5 is a block diagram of the structure of the human body posture information generating unit 3 according to the embodiment of the present invention;
fig. 6 is a block diagram of the structure of the motion trajectory generation unit 4 according to the embodiment of the present invention;
fig. 7 is a block diagram of the structure of the judgment unit 5 according to the embodiment of the present invention;
fig. 8 is a schematic block diagram of a system configuration of an electronic apparatus 600 according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In carrying out the present application, the applicant has found the following related art:
the first technology is as follows: based on a deployed high-definition network camera in the well, video streams are acquired in a centralized mode, moving pedestrian targets in the video streams are identified through a pedestrian boundary crossing detection algorithm, a boundary crossing trend is calculated through state buffering processing on the basis of the identification of the targets, and the boundary crossing direction is judged. Specifically, configuration parameters such as a hotspot region of interest, a detection mode (line crossing mode: setting detection line; region mode: setting quadrangle) and the like are selectively set. After an input image is initialized, preprocessing and foreground extraction are carried out, a main detection module can identify and mark a pedestrian target, and finally, cross-line or intrusion judgment is carried out on the pedestrian target.
However, the technology does not automatically identify the dangerous area, and still needs to manually configure parameters and draw an electronic fence to determine the dangerous area. Secondly, the pedestrian target detection result is output in a rectangular frame mode, and if a worker only enables arms to cross the cable slot and stands outside the dangerous area, the rectangular frame of the detection result is overlapped with the dangerous area, so that misjudgment is caused. Finally, the foreground and background extraction method is difficult to extract pedestrian targets when multi-target motion is mutually shielded.
The second technology is as follows: differentiating an original video into image frames, and selecting one frame as a background frame; the attention mechanism and the target detection algorithm based on deep learning detect workers in each frame of image, form a frame at the position of the workers, cut the frame and store the frame as a picture, and then store three items of the rectangular frame picture of the cut workers, positioning information and processed frame information as a group in a linked list to complete the compression of all the frame images. And sequentially taking out the cut rectangular frame, the position and the number of the image frame of the worker from the linked list as a group, covering each group into the background frame, and finishing the decoding and restoring of all the frame images. The target detection algorithm is a convolutional neural network based on regions, and comprises FAST-RCNN, SSD, YOLO and the like, and is used for detecting the types of objects and positioning the positions of the objects.
However, the target detection algorithm in the technology is used for pedestrian recognition, and the positioning object position is only limited to a rectangular frame, and the method is not accurate enough for line-crossing detection scenes. In addition, background frame selection by difference is not flexible enough, and model effect is difficult to guarantee.
That is to say, no effective underground pedestrian boundary crossing risk detection scheme exists at present, and based on the scheme, the embodiment of the invention provides the underground pedestrian boundary crossing risk detection scheme, and the scheme can provide real-time accurate early warning for the boundary crossing detection of the pedestrian dangerous area in the fully mechanized mining working face through the human posture key point skeleton detection and identification and the multi-target pedestrian tracking algorithm, can better perform the underground pedestrian boundary crossing detection, ensure the working safety of coal mine workers, and make important contribution to reducing the occurrence of safety accidents.
It should be noted that, in the technical solution of the present application, the acquisition, storage, use, processing, etc. of data all conform to the relevant regulations of the national laws and regulations. Embodiments of the present invention are described in detail below with reference to the accompanying drawings.
Fig. 1 is a flowchart of a method for detecting a pedestrian crossing risk in a well, according to an embodiment of the present invention, as shown in fig. 1, the method includes:
102, performing frame cutting processing on the video image to obtain an image to be processed, wherein the image to be processed comprises: a plurality of pedestrian images.
And 103, identifying the multiple pedestrian images based on a human body posture estimation algorithm to generate human body posture information of each pedestrian.
And 104, tracking the multiple pedestrian images based on a multi-target pedestrian tracking algorithm to generate the motion trail of each pedestrian.
And 105, performing coordinate comparison operation according to the human body posture information, the motion track and the preset dangerous area of each pedestrian, and determining whether the pedestrian has the boundary crossing risk according to the coordinate comparison operation result.
The method comprises the steps of performing frame cutting processing on an obtained underground pedestrian monitoring video image to obtain an image to be processed, then identifying and processing a plurality of pedestrian images in the image to be processed based on a human body posture estimation algorithm to generate human body posture information of each pedestrian, simultaneously tracking and processing the plurality of pedestrian images based on a multi-target pedestrian tracking algorithm to generate a motion track of each pedestrian, then performing coordinate comparison operation according to the human body posture information, the motion track and a preset danger area of each pedestrian, and determining whether the pedestrian has boundary crossing risks or not according to the coordinate comparison operation result.
Preferably, the human posture estimation algorithm may be an openpos (human posture recognition) algorithm. The step 103 specifically includes: identifying preset joint points of all people in the image based on the OpenPose algorithm; and performing joint point connection operation on each pedestrian according to the identified joint points and the connection information among the joint points to generate human body posture information of each pedestrian.
The predetermined joint points include at least foot joint points, and other joint points may be determined according to actual conditions.
Human body posture information is generated through an OpenPose algorithm, and the coordinates of the lower limbs and the feet of the skeleton can be compared with the coordinates of the dangerous area, so that the pedestrian crossing risk is judged.
Preferably, the multi-target pedestrian tracking algorithm may be a deep sort (multi-target tracking) algorithm. The step 104 specifically includes: detecting current position information of each pedestrian in the image based on the Deepsort algorithm; predicting the current position information of each detected pedestrian based on a Kalman filtering prediction algorithm to predict the position information of each pedestrian at the next moment; and generating the motion trail of each pedestrian according to the current position information and the position information of the pedestrian at the next moment.
The motion trail of the pedestrian is generated through the deep sort algorithm, the problem that the identification of the motion trail of the pedestrian is inaccurate due to shielding can be avoided, and the accuracy of pedestrian boundary crossing risk judgment is further improved.
The embodiment of the invention adopts mine working face monitoring video, and provides a video-based underground multi-target tracking human body posture identification line-crossing early warning scheme by taking the characteristics and requirements of pedestrian line-crossing detection early warning as core targets of improving early warning accuracy, wherein the scheme mainly comprises two parts: firstly, recognizing a human body posture skeleton based on video analysis; and secondly, a visual multi-target tracking algorithm based on deep learning. These two parts are described in detail below in conjunction with the downhole pedestrian crossing risk detection flow shown in fig. 2.
Human body posture skeleton recognition based on video analysis
In practice, the improved openpos algorithm may be used to recognize the human gesture skeleton. Specifically, the embodiment of the present invention adopts an improved openpos algorithm, and the identification process specifically includes the following four steps:
1. the neural network prediction is characterized in that a characteristic backbone network replaces the original VGG-19 by Mobile Net, wherein the Mobile Net and the VGG-19 are terms specific to the CV field and can be understood as a neural network with a specific structure.
2. A joint is determined.
3. Limb connection is determined.
4. The limbs are assembled to form the human skeleton.
In the neural network structure of the original openpos algorithm, feature extraction is performed through the VGG-19. In the embodiment of the invention, the Mobile Net structure with the cavity convolution is used for replacing the original VGG structure, because the Mobile Net has a deeper network structure, a larger perception visual field and a better effect than the VGG-19.
The original openpos algorithm is a multi-stage CNN with two branches, the first branch to predict confidence map (confidence map, denoted S) and the second branch to predict Par Affinity Fields (limb 2D vector, denoted L). After each stage, the predictions from the two branches and the image features are concatenated for the next stage.
The original openpos algorithm has two prediction branches, which have the same structure and obtain different numbers of output results only in the output stage. In the embodiment of the present invention, two branches are merged into one, and only in the output stage, the two branches are branched out using the convolution of 11, so as to obtain two result outputs, where:
s = (S1, S2 … … Sj) represents heat map (thermodynamic diagram), and j represents the number of joints to be detected.
L = (L1, L2 … … Lc) denotes a vector map (vector diagram), and C denotes the number of pairs of joints to be detected.
The recognition of the human body gesture based on the improved openpos algorithm is described in detail below with reference to the human body recognition examples shown in fig. 3(1) -fig. 3 (5).
First, an initial image shown in fig. 3(1) is input, and a heatmap of each joint of the pedestrian is obtained.
Using convolutional neural network cnn (mobile net), 19 key points (i.e., 19 joint points) are extracted from the pedestrian image, specifically, the 19 key point categories include 18 human body mark points and a background class, and the 18 human body mark points may be: nose, neck, right shoulder, right elbow, right wrist, left shoulder, left elbow, left wrist, right hip, right knee, right ankle, left hip, left knee, left ankle, left eye, right eye, left ear, right ear, the background class may be denoted pt 19. Specifically, reference may be made to the mark points shown in fig. 3(5), where it should be noted that the mark numbers therein do not have a corresponding relationship with the description herein.
A heatmap of 19 channels is generated, each channel being the heatmap of a certain joint, while vectormap (vector map) of 192 channels, which is twice the heatmap by 38 (19 x 2), is generated, because there are 19 keypoint connecting lines, each represented by a vector, having the map of the x-axis, and the map of the y-axis, respectively.
Second, the specific position of the joint is extracted from the heatmap, as shown in fig. 3 (2).
A non-maximum suppression (NMS) algorithm may be used to obtain the peak in heatmap, with the value being the confidence, as score for the part. The output is:
all _ peaks = [ [ ((h0, w0, s0,0), (h1, w1, s1,1) ]. ] \ all values of the first part
[ ((hi, wi, si, i), (hi +1, wi +1, si +1, i +1) ]. ] all values of the second part
…
]
Therefore, joint information (position, fraction) can be obtained, wherein part refers to a human body key point, and h and w respectively represent the height and width pixel values of the key point in the whole picture and represent the position of the key point. s denotes that confidence can be understood as a score value.
In a third step, see fig. 3(3), joint information and paf are used to obtain a limb connection, such as the right lower leg connection shown in fig. 3 (3). Wherein paf is the affinity field of key points, which is used to describe the affinity between different key points. Different joints belonging to the same person have high affinity, and joints among different persons have low affinity.
The model comprises 19 limbs, two parts and paf corresponding to each limb are determined, and the result obtained by integrating paf information between the two parts is used as the confidence coefficient of the limb. All connection information is available, each connection can be considered as a limb.
And fourthly, assembling limbs, as shown in fig. 3 and 4, and completing the splicing of the whole human body. The joint points of the pedestrians are connected, and due to the vector property of paf, the generated even matching is correct, and finally the even matching is combined into the whole skeleton of one person.
Visual multi-target tracking algorithm based on deep learning
The embodiment of the invention adopts an improved deep sort algorithm to track multiple targets of pedestrians, and the method comprises the following specific steps:
1. the target detection module uses a YOLOv5x (a detector name) detector to obtain a Bbox (Bounding-box) and generate detection frames detections.
Specifically, the target detection module detects the bbox in the current frame by using the YOLOv5x framework, and then converts the detected bbox into detections, mainly aiming at detecting the position information of the pedestrian in each frame.
2. And predicting by Kalman filtering.
Specifically, kalman filtering is used to predict the state of tracks (tracking box) in the previous frame at the current frame, which is used to predict the position at the current time based on the position at the previous time of the target. That is, the new position = current position + displacement amount, displacement amount = time × speed.
3. And (4) cascading matching.
Specifically, the cascade matching is to calculate a cost matrix of tracks and detectors for mahalanobis distance based on appearance information, then perform cascade matching successively, add an IOU (interaction over Union, a standard for measuring the accuracy of detecting corresponding objects in a specific data set) constraint on the basis of original matching, avoid the problem of disappearance of a tracking target under the shielding condition, and finally obtain all matching pairs, unmatched tracks and unmatched detectors of the current frame.
4. And updating Kalman filtering.
And updating each successfully matched track by using the corresponding detection, and setting the unsuccessfully matched tracks and the detections as new detection targets, wherein the updating is mainly used for circularly updating the detection information of the successfully matched tracks.
And then, referring to fig. 2, storing the human body posture information obtained based on the openpos algorithm and the pedestrian motion trajectory obtained based on the DeepSort algorithm, performing an offline intrusion judgment, and performing an early warning operation to remind pedestrians of danger when the judgment result is the offline intrusion.
According to the embodiment of the invention, real-time accurate early warning is provided for the line crossing detection of the pedestrian danger area in the fully mechanized mining working face through the detection and identification of the human body posture key point skeleton and the multi-target pedestrian tracking algorithm, the working safety of coal miners can be well ensured, and meanwhile, important contribution is made to reducing the occurrence of safety accidents.
According to the embodiment of the invention, the key parts of the pedestrians are positioned by adopting a human body posture estimation algorithm instead of adopting a target detection mode to identify the pedestrians, so that the crossing problem of the dangerous area can be judged more accurately. Meanwhile, the embodiment of the invention integrates the pedestrian tracking algorithm of the Deepsort multi-target, can better judge the pedestrian track, avoids the false recognition caused by shielding, improves the stability and the accuracy of the pedestrian recognition, and reduces the false alarm rate of the off-line alarm.
It should be noted that, in the embodiment of the present invention, the openpos algorithm is used to identify key points of a human body, but the protection scope of the present invention is not limited to this, and other human body posture estimation algorithms may be used instead, so as to also achieve the purpose of the present invention, and the present invention should also be covered in the protection scope of the present invention. Meanwhile, the deep sort multi-target pedestrian tracking algorithm adopted in the embodiment of the invention can accurately track a plurality of pedestrians in the camera area, but the protection scope of the invention is not limited to this, for example, other deep learning-based target tracking algorithms or traditional tracking algorithms can be adopted for substitution, the invention purpose can also be achieved, and the invention scope also shall be covered in the protection scope of the invention.
Based on similar inventive concepts, the embodiment of the invention also provides a device for detecting the pedestrian boundary crossing risk in the well, and the device can be preferably used for realizing the flow of the method embodiment.
Fig. 4 is a block diagram of the structure of the underground pedestrian boundary crossing risk detection device, as shown in fig. 4, the device comprises: the device comprises a video acquisition unit 1, an image to be processed generation unit 2, a human body posture information generation unit 3, a motion trail generation unit 4 and a judgment unit 5, wherein:
the video acquisition unit 1 is used for acquiring underground pedestrian monitoring video images in real time;
a to-be-processed image generating unit 2, configured to perform frame cutting processing on the video image to obtain a to-be-processed image, where the to-be-processed image includes: a plurality of pedestrian images;
a human body posture information generating unit 3 for performing recognition processing on the plurality of pedestrian images based on a human body posture estimation algorithm to generate human body posture information of each pedestrian;
a motion trajectory generation unit 4 configured to perform tracking processing on the plurality of pedestrian images based on a multi-target pedestrian tracking algorithm to generate a motion trajectory of each pedestrian;
and the judging unit 5 is used for performing coordinate comparison operation according to the human body posture information, the motion trail and the preset dangerous area of each pedestrian and determining whether the pedestrian has the boundary crossing risk according to the coordinate comparison operation result.
The underground pedestrian detection method comprises the steps of performing frame cutting processing on underground pedestrian monitoring video images acquired by a video acquisition unit 1 through an image generation unit 2 to be processed to obtain images to be processed, then performing identification processing on a plurality of pedestrian images in the images to be processed through a human posture information generation unit 3 based on a human posture estimation algorithm to generate human posture information of all pedestrians, simultaneously performing tracking processing on the plurality of pedestrian images through a motion track generation unit 4 based on a multi-target pedestrian tracking algorithm to generate motion tracks of all pedestrians, then performing coordinate comparison operation on the human posture information, the motion tracks and preset danger areas of all pedestrians through a judgment unit 5, and determining whether the pedestrians have boundary crossing risks according to coordinate comparison operation results, so that underground pedestrian boundary crossing detection can be effectively performed, and real-time accurate early warning can be provided for pedestrian danger area boundary crossing detection in a fully mechanized working face, the working safety of coal miners can be better ensured.
Preferably, the human body posture estimation algorithm used by the human body posture information generation unit 3 may be an openpos algorithm.
As shown in fig. 5, the human posture information generating unit 3 specifically includes: a joint point recognition module 31 and a human posture information generation module 32, wherein:
a joint point identification module 31, configured to perform identification of predetermined joint points for each person in the image based on the openpos algorithm;
and a human body posture information generating module 32, configured to perform joint point connection operation on each pedestrian according to the identified joint points and connection information between the joint points, so as to generate human body posture information of each pedestrian.
In practical operation, the human posture information generating unit 3 may also use a modified openpos algorithm, and the specific identification process may be as described in the above method embodiment.
Preferably, the multi-target pedestrian tracking algorithm used in the motion trajectory generating unit 4 may be a deep sort algorithm.
As shown in fig. 6, the motion trajectory generating unit 4 specifically includes: a current position information detection module 41, a position information prediction module 42, and a motion trajectory generation module 43, wherein:
a current position information detection module 41, configured to perform a current position information detection operation on each pedestrian in the image based on the DeepSort algorithm;
the position information prediction module 42 is configured to perform prediction operation on the detected current position information of each pedestrian based on a kalman filtering prediction algorithm to predict the position information of each pedestrian at the next time;
and a motion trajectory generating module 43, configured to generate a motion trajectory of each pedestrian according to the current position information and the position information of the next moment of each pedestrian.
Specifically, as shown in fig. 7, the above-described judging unit 5 includes: a coordinate comparison module 51 and an out-of-range risk determination module 52, wherein:
the coordinate comparison module 51 is used for performing coordinate comparison operation according to the human body posture information, the motion trail and the preset danger area of each pedestrian;
and the boundary crossing risk determining module 52 is used for determining that the pedestrian has a boundary crossing risk in response to the coordinate comparison operation result that the human body posture information and the motion trail of the pedestrian cross the preset dangerous area.
For specific execution processes of the units and the modules, reference may be made to the description in the foregoing method embodiments, and details are not described here again.
In practical operation, the units and the modules may be combined or may be singly arranged, and the present invention is not limited thereto.
The present embodiment also provides an electronic device, which may be a desktop computer, a tablet computer, a mobile terminal, and the like, but is not limited thereto. In this embodiment, the electronic device may be implemented by referring to the above method embodiment and the embodiment of the underground pedestrian crossing risk detection device, which are incorporated herein, and repeated descriptions are omitted.
Fig. 8 is a schematic block diagram of a system configuration of an electronic apparatus 600 according to an embodiment of the present invention. As shown in fig. 8, the electronic device 600 may include a central processor 100 and a memory 140; the memory 140 is coupled to the central processor 100. Notably, this diagram is exemplary; other types of structures may also be used in addition to or in place of the structure to implement telecommunications or other functions.
In one embodiment, the downhole pedestrian crossing risk detection function may be integrated into the central processor 100. The central processor 100 may be configured to control as follows:
acquiring an underground pedestrian monitoring video image in real time;
performing frame cutting processing on the video image to obtain an image to be processed, wherein the image to be processed comprises: a plurality of pedestrian images;
identifying the plurality of pedestrian images based on a human body posture estimation algorithm to generate human body posture information of each pedestrian;
tracking the multiple pedestrian images based on a multi-target pedestrian tracking algorithm to generate a motion track of each pedestrian;
and performing coordinate comparison operation according to the human body posture information, the motion trail and the preset dangerous area of each pedestrian, and determining whether the pedestrian has the boundary crossing risk according to the coordinate comparison operation result.
As can be seen from the above description, the electronic device provided in the embodiment of the present application obtains an image to be processed by performing frame cutting on an acquired video image of a downhole pedestrian monitor, then performs recognition processing on a plurality of pedestrian images in the image to be processed based on a human body posture estimation algorithm to generate human body posture information of each pedestrian, meanwhile, a plurality of pedestrian images are tracked based on a multi-target pedestrian tracking algorithm to generate the motion trail of each pedestrian, then coordinate comparison operation is carried out according to the human body posture information, the motion trail and a preset danger area of each pedestrian, whether the pedestrian has out-of-range risk or not is determined according to the coordinate comparison operation result, and thus, the underground pedestrian crossing detection can be effectively carried out, real-time accurate early warning can be provided for the pedestrian danger area crossing detection in the fully mechanized mining working face, and the working safety of coal mine workers can be well ensured.
In another embodiment, the downhole pedestrian crossing risk detection device may be configured separately from the central processing unit 100, for example, the downhole pedestrian crossing risk detection device may be configured as a chip connected to the central processing unit 100, and the downhole pedestrian crossing risk detection function is realized by the control of the central processing unit.
As shown in fig. 8, the electronic device 600 may further include: communication module 110, input unit 120, audio processing unit 130, display 160, power supply 170. It is noted that the electronic device 600 does not necessarily include all of the components shown in FIG. 8; furthermore, the electronic device 600 may also comprise components not shown in fig. 8, which may be referred to in the prior art.
As shown in fig. 8, the central processor 100, sometimes referred to as a controller or operational control, may include a microprocessor or other processor device and/or logic device, the central processor 100 receiving input and controlling the operation of the various components of the electronic device 600.
The memory 140 may be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. The information relating to the failure may be stored, and a program for executing the information may be stored. And the central processing unit 100 may execute the program stored in the memory 140 to realize information storage or processing, etc.
The input unit 120 provides input to the cpu 100. The input unit 120 is, for example, a key or a touch input device. The power supply 170 is used to provide power to the electronic device 600. The display 160 is used to display an object to be displayed, such as an image or a character. The display may be, for example, an LCD display, but is not limited thereto.
The memory 140 may be a solid state memory such as Read Only Memory (ROM), Random Access Memory (RAM), a SIM card, or the like. There may also be a memory that holds information even when power is off, can be selectively erased, and is provided with more data, an example of which is sometimes called an EPROM or the like. The memory 140 may also be some other type of device. Memory 140 includes buffer memory 141 (sometimes referred to as a buffer). The memory 140 may include an application/function 142 for storing application programs and function programs or a flow for executing the operation of the electronic device 600 by the central processor 100.
The communication module 110 is a transmitter/receiver 110 that transmits and receives signals via an antenna 111. The communication module (transmitter/receiver) 110 is coupled to the central processor 100 to provide an input signal and receive an output signal, which may be the same as in the case of a conventional mobile communication terminal.
Based on different communication technologies, a plurality of communication modules 110, such as a cellular network module, a bluetooth module, and/or a wireless local area network module, may be provided in the same electronic device. The communication module (transmitter/receiver) 110 is also coupled to a speaker 131 and a microphone 132 via an audio processor 130 to provide audio output via the speaker 131 and receive audio input from the microphone 132 to implement general telecommunications functions. Audio processor 130 may include any suitable buffers, decoders, amplifiers and so forth. In addition, an audio processor 130 is also coupled to the central processor 100, so that recording on the local can be enabled through a microphone 132, and so that sound stored on the local can be played through a speaker 131.
Embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the steps of the method for detecting a pedestrian boundary crossing risk in a well.
In summary, in order to solve the problem that the human body identification is inaccurate in the pedestrian crossing detection scene, the embodiment of the invention adopts the human body posture estimation and multi-target tracking algorithm of transfer learning to realize the real-time identification of the two-dimensional multi-person key points, and the problem of inaccurate identification caused by shielding can be avoided by comparing the foot coordinates of the lower limbs of the framework with the coordinates of the dangerous area and simultaneously carrying out multi-target example tracking, so that whether the pedestrian crosses the boundary can be more accurately judged.
The preferred embodiments of the present invention have been described above with reference to the accompanying drawings. The many features and advantages of the embodiments are apparent from the detailed specification, and thus, it is intended by the appended claims to cover all such features and advantages of the embodiments which fall within the true spirit and scope thereof. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the embodiments of the invention to the exact construction and operation illustrated and described, and accordingly, all suitable modifications and equivalents may be resorted to, falling within the scope thereof.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The principle and the implementation mode of the invention are explained by applying specific embodiments in the invention, and the description of the embodiments is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. A method of downhole pedestrian out-of-range risk detection, the method comprising:
acquiring an underground pedestrian monitoring video image in real time;
performing frame cutting processing on the video image to obtain an image to be processed, wherein the image to be processed comprises: a plurality of pedestrian images;
identifying the plurality of pedestrian images based on a human body posture estimation algorithm to generate human body posture information of each pedestrian;
tracking the multiple pedestrian images based on a multi-target pedestrian tracking algorithm to generate a motion track of each pedestrian;
and performing coordinate comparison operation according to the human body posture information, the motion trail and the preset dangerous area of each pedestrian, and determining whether the pedestrian has the boundary crossing risk according to the coordinate comparison operation result.
2. The method according to claim 1, wherein the human body posture estimation algorithm is an openpos algorithm, and the identifying the plurality of pedestrian images based on the human body posture estimation algorithm to generate the human body posture information of each pedestrian comprises:
identifying preset joint points of all people in the image based on the OpenPose algorithm;
and performing joint point connection operation on each pedestrian according to the identified joint points and the connection information among the joint points to generate human body posture information of each pedestrian.
3. The method according to claim 1, wherein the multi-target pedestrian tracking algorithm is a deep sort algorithm, and the tracking processing of the multiple pedestrian images based on the multi-target pedestrian tracking algorithm to generate the motion trail of each pedestrian comprises:
detecting current position information of each pedestrian in the image based on the Deepsort algorithm;
predicting the current position information of each detected pedestrian based on a Kalman filtering prediction algorithm to predict the position information of each pedestrian at the next moment;
and generating the motion trail of each pedestrian according to the current position information and the position information of the pedestrian at the next moment.
4. The method for detecting the underground pedestrian boundary crossing risk according to claim 1, wherein the step of performing coordinate comparison operation according to the human body posture information, the motion trail and the preset danger area of each pedestrian, and the step of determining whether the pedestrian has the boundary crossing risk according to the coordinate comparison operation result comprises the steps of:
performing coordinate comparison operation according to the human body posture information, the motion trail and the preset danger area of each pedestrian;
and determining that the pedestrian has the out-of-range risk in response to the coordinate comparison operation result that the human body posture information and the motion trail of the pedestrian cross the preset dangerous area.
5. A downhole pedestrian out-of-range risk detection device, the device comprising:
the video acquisition unit is used for acquiring a pedestrian monitoring video image in the well in real time;
a to-be-processed image generating unit, configured to perform frame cutting processing on the video image to obtain a to-be-processed image, where the to-be-processed image includes: a plurality of pedestrian images;
a human body posture information generating unit for performing recognition processing on the plurality of pedestrian images based on a human body posture estimation algorithm to generate human body posture information of each pedestrian;
the motion trail generation unit is used for tracking the multiple pedestrian images based on a multi-target pedestrian tracking algorithm so as to generate motion trails of all pedestrians;
and the judging unit is used for carrying out coordinate comparison operation according to the human body posture information, the motion trail and the preset danger area of each pedestrian and determining whether the pedestrian has the boundary crossing risk or not according to the coordinate comparison operation result.
6. The device according to claim 5, wherein the human body posture estimation algorithm is an OpenPose algorithm, and the human body posture information generation unit includes:
the joint point identification module is used for identifying preset joint points of each person in the image based on the OpenPose algorithm;
and the human body posture information generating module is used for performing joint point connection operation on each pedestrian according to the identified joint points and the connection information among the joint points so as to generate the human body posture information of each pedestrian.
7. The device according to claim 5, wherein the multi-target pedestrian tracking algorithm is a Deepsort algorithm, and the motion trajectory generation unit comprises:
the current position information detection module is used for detecting current position information of each pedestrian in the image based on the Deepsort algorithm;
the position information prediction module is used for performing prediction operation on the detected current position information of each pedestrian based on a Kalman filtering prediction algorithm so as to predict the position information of each pedestrian at the next moment;
and the motion trail generation module is used for generating the motion trail of each pedestrian according to the current position information and the position information of the next moment of each pedestrian.
8. The downhole pedestrian crossing risk detecting device according to claim 5, wherein the judging unit includes:
the coordinate comparison module is used for carrying out coordinate comparison operation according to the human body posture information, the motion trail and the preset danger area of each pedestrian;
and the boundary crossing risk determining unit is used for responding to the coordinate comparison operation result that the human body posture information and the motion trail of the pedestrian cross a preset dangerous area, and then determining that the pedestrian has the boundary crossing risk.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the steps of the method of detecting a pedestrian crossing risk in a well according to any one of claims 1 to 4.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method for detecting a pedestrian crossing risk downhole according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210113490.0A CN114140832A (en) | 2022-01-30 | 2022-01-30 | Method and device for detecting pedestrian boundary crossing risk in well, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210113490.0A CN114140832A (en) | 2022-01-30 | 2022-01-30 | Method and device for detecting pedestrian boundary crossing risk in well, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114140832A true CN114140832A (en) | 2022-03-04 |
Family
ID=80381930
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210113490.0A Pending CN114140832A (en) | 2022-01-30 | 2022-01-30 | Method and device for detecting pedestrian boundary crossing risk in well, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114140832A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115866334A (en) * | 2023-02-27 | 2023-03-28 | 成都华域天府数字科技有限公司 | Data processing method for cutting and associating content in video flow |
| CN115953872A (en) * | 2022-12-28 | 2023-04-11 | 广东悦视智能科技有限公司 | Pedestrian border crossing early warning method and device based on video monitoring and storage medium |
| CN116502816A (en) * | 2022-12-26 | 2023-07-28 | 北京北投生态环境有限公司 | Land parcel monitoring well layout method suitable for pollutants |
| CN116993776A (en) * | 2023-06-30 | 2023-11-03 | 中信重工开诚智能装备有限公司 | Personnel track tracking method |
| CN117115783A (en) * | 2023-09-06 | 2023-11-24 | 江苏泰坦智慧科技有限公司 | Assembly line work behavior recognition method based on machine vision |
| CN117152689A (en) * | 2023-10-31 | 2023-12-01 | 易启科技(吉林省)有限公司 | River channel target detection method and system based on vision |
| CN117457193A (en) * | 2023-12-22 | 2024-01-26 | 之江实验室 | Physical health monitoring method and system based on human body key point detection |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112257502A (en) * | 2020-09-16 | 2021-01-22 | 深圳微步信息股份有限公司 | Pedestrian identification and tracking method and device for surveillance video and storage medium |
| CN112950668A (en) * | 2021-02-26 | 2021-06-11 | 北斗景踪技术(山东)有限公司 | Intelligent monitoring method and system based on mold position measurement |
| CN113192105A (en) * | 2021-04-16 | 2021-07-30 | 嘉联支付有限公司 | Method and device for tracking multiple persons and estimating postures indoors |
| WO2021174697A1 (en) * | 2020-03-06 | 2021-09-10 | 平安科技（深圳）有限公司 | Human body posture evaluation method and apparatus, computer device, and storage medium |
| CN113888824A (en) * | 2021-06-30 | 2022-01-04 | 华北理工大学 | Mine violation border crossing behavior identification and alarm method and device based on topological spatial relationship |
-
2022
- 2022-01-30 CN CN202210113490.0A patent/CN114140832A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021174697A1 (en) * | 2020-03-06 | 2021-09-10 | 平安科技（深圳）有限公司 | Human body posture evaluation method and apparatus, computer device, and storage medium |
| CN112257502A (en) * | 2020-09-16 | 2021-01-22 | 深圳微步信息股份有限公司 | Pedestrian identification and tracking method and device for surveillance video and storage medium |
| CN112950668A (en) * | 2021-02-26 | 2021-06-11 | 北斗景踪技术(山东)有限公司 | Intelligent monitoring method and system based on mold position measurement |
| CN113192105A (en) * | 2021-04-16 | 2021-07-30 | 嘉联支付有限公司 | Method and device for tracking multiple persons and estimating postures indoors |
| CN113888824A (en) * | 2021-06-30 | 2022-01-04 | 华北理工大学 | Mine violation border crossing behavior identification and alarm method and device based on topological spatial relationship |
Non-Patent Citations (1)
| Title |
|---|
| 徐涛等: "基于深度学习的行人多目标跟踪方法", 《吉林大学学报（工学版）》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116502816A (en) * | 2022-12-26 | 2023-07-28 | 北京北投生态环境有限公司 | Land parcel monitoring well layout method suitable for pollutants |
| CN116502816B (en) * | 2022-12-26 | 2024-01-02 | 北京北投生态环境有限公司 | Land parcel monitoring well layout method suitable for pollutants |
| CN115953872A (en) * | 2022-12-28 | 2023-04-11 | 广东悦视智能科技有限公司 | Pedestrian border crossing early warning method and device based on video monitoring and storage medium |
| CN115866334A (en) * | 2023-02-27 | 2023-03-28 | 成都华域天府数字科技有限公司 | Data processing method for cutting and associating content in video flow |
| CN116993776A (en) * | 2023-06-30 | 2023-11-03 | 中信重工开诚智能装备有限公司 | Personnel track tracking method |
| CN116993776B (en) * | 2023-06-30 | 2024-02-13 | 中信重工开诚智能装备有限公司 | Personnel track tracking method |
| CN117115783A (en) * | 2023-09-06 | 2023-11-24 | 江苏泰坦智慧科技有限公司 | Assembly line work behavior recognition method based on machine vision |
| CN117115783B (en) * | 2023-09-06 | 2024-03-26 | 江苏泰坦智慧科技有限公司 | Assembly line work behavior recognition method based on machine vision |
| CN117152689A (en) * | 2023-10-31 | 2023-12-01 | 易启科技(吉林省)有限公司 | River channel target detection method and system based on vision |
| CN117152689B (en) * | 2023-10-31 | 2024-01-19 | 易启科技(吉林省)有限公司 | River channel target detection method and system based on vision |
| CN117457193A (en) * | 2023-12-22 | 2024-01-26 | 之江实验室 | Physical health monitoring method and system based on human body key point detection |
| CN117457193B (en) * | 2023-12-22 | 2024-04-02 | 之江实验室 | Physical health monitoring method and system based on human body key point detection |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114140832A (en) | Method and device for detecting pedestrian boundary crossing risk in well, electronic equipment and storage medium | |
| US8243987B2 (en) | Object tracking using color histogram and object size | |
| WO2020215552A1 (en) | Multi-target tracking method, apparatus, computer device, and storage medium | |
| CN113076809A (en) | High-altitude falling object detection method based on visual Transformer | |
| US20130177203A1 (en) | Object tracking and processing | |
| US9183431B2 (en) | Apparatus and method for providing activity recognition based application service | |
| CN111259751A (en) | Video-based human behavior recognition method, device, equipment and storage medium | |
| CN104361327A (en) | Pedestrian detection method and system | |
| CN110889334A (en) | Personnel intrusion identification method and device | |
| CN111178323B (en) | Group behavior recognition method, device, equipment and storage medium based on video | |
| KR101062225B1 (en) | Intelligent video retrieval method and system using surveillance camera | |
| EP3973446A1 (en) | Forensic video exploitation and analysis tools | |
| CN111881853A (en) | Method and device for identifying abnormal behaviors in oversized bridge and tunnel | |
| CN109754034A (en) | A kind of terminal device localization method and device based on two dimensional code | |
| CN112861673A (en) | False alarm removal early warning method and system for multi-target detection of surveillance video | |
| CN113326773A (en) | Recognition model training method, recognition method, device, equipment and storage medium | |
| CN114241379A (en) | Passenger abnormal behavior identification method, device and equipment and passenger monitoring system | |
| CN107480580B (en) | Image recognition method and image recognition device | |
| CN113910224A (en) | Robot following method and device and electronic equipment | |
| JP2022526071A (en) | Situational awareness monitoring | |
| US11715372B2 (en) | Signal lamp recognition method, device, and storage medium | |
| CN106803937B (en) | Double-camera video monitoring method, system and monitoring device with text log | |
| Satybaldina et al. | Development of an algorithm for abnormal human behavior detection in intelligent video surveillance system | |
| KR20010035100A (en) | An Effective Object Tracking Method and for Apparatus for Interactive HyperLink Video | |
| JP2020013480A (en) | Vehicle recognition system and vehicle recognition method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220304 |