CN112541424A - Real-time detection method for pedestrian falling under complex environment - Google Patents
Real-time detection method for pedestrian falling under complex environment Download PDFInfo
- Publication number
- CN112541424A CN112541424A CN202011427824.9A CN202011427824A CN112541424A CN 112541424 A CN112541424 A CN 112541424A CN 202011427824 A CN202011427824 A CN 202011427824A CN 112541424 A CN112541424 A CN 112541424A
- Authority
- CN
- China
- Prior art keywords
- pedestrian
- target
- frame
- detection
- falling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 28
- 238000011897 real-time detection Methods 0.000 title claims abstract description 9
- 238000001514 detection method Methods 0.000 claims abstract description 63
- 230000008859 change Effects 0.000 claims abstract description 8
- 238000007781 pre-processing Methods 0.000 claims abstract description 4
- 238000012549 training Methods 0.000 claims description 21
- 238000000605 extraction Methods 0.000 claims description 14
- 239000011159 matrix material Substances 0.000 claims description 12
- 230000000694 effects Effects 0.000 claims description 10
- 238000005259 measurement Methods 0.000 claims description 10
- 238000012360 testing method Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000012937 correction Methods 0.000 claims description 5
- 230000001133 acceleration Effects 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 4
- 230000004927 fusion Effects 0.000 claims description 4
- 238000005070 sampling Methods 0.000 claims description 4
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 238000012706 support-vector machine Methods 0.000 claims description 3
- 230000017105 transposition Effects 0.000 claims description 3
- 230000003190 augmentative effect Effects 0.000 claims description 2
- 238000012544 monitoring process Methods 0.000 abstract description 2
- 238000012545 processing Methods 0.000 abstract description 2
- 238000010606 normalization Methods 0.000 abstract 1
- 230000006870 function Effects 0.000 description 9
- 230000009466 transformation Effects 0.000 description 5
- 230000033001 locomotion Effects 0.000 description 4
- 239000013598 vector Substances 0.000 description 4
- 238000005286 illumination Methods 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 206010063746 Accidental death Diseases 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Image Analysis (AREA)
Abstract
A real-time detection method for pedestrian falling under a complex environment relates to a detection method for monitoring the state of a pedestrian. The method comprises the following steps: preprocessing the acquired video, converting each frame of the video stream into a picture, and performing normalization processing; detecting the pedestrian, namely adjusting the size of a detection frame according to the change of the distance between the pedestrian and the camera; tracking the target, namely tracking the target, extracting features and calculating similarity by adopting a sort algorithm; predicting the current position through a Kalman filter, and associating a detection box and a target position by using a Hungarian algorithm; and (4) judging whether the pedestrian falls down in the target area, wherein the aspect ratio of the pedestrian is less than or equal to 0.4 when the pedestrian stands, and the aspect ratio is increased to 0.7-1.2 when the pedestrian falls down. The invention aims to solve the problems in the task of detecting and judging the falling of pedestrians, and constructs a real-time detection method for the falling of the pedestrians, which has a simple and clear structure, various application scenes, high precision and high robustness.
Description
Technical Field
The invention relates to a detection method for monitoring states of pedestrians, in particular to the technical field of real-time detection of falling of pedestrians in a complex environment.
Background
The probability of accidental death caused by the falling of the old people in the society is increased continuously nowadays, which arouses wide attention of people and develops a great deal of research work on the falling detection of the old people. The current automatic detection system for the falling of the pedestrian is mainly divided into three types: scene device based, wearable device based and computer vision based fall automatic detection systems.
The automatic falling detection system based on the scene equipment analyzes and judges the data of human body motion collected by the sensor equipment under a specific scene, and although the normal life can not be influenced and the accuracy is high, the equipment cost is higher, so that the automatic falling detection system is difficult to popularize and use. The falling detection system based on the wearable device detects and judges whether the old man falls or not through the equipment worn with the pressure sensor, the acceleration sensor and the like, the equipment is easy to damage and interfere, a false alarm event occurs, and the equipment is worn all the time to influence normal activities.
The falling detection system based on computer vision analyzes the shape characteristics and the movement of a human body to detect falling events by transmitting a shot image to a computing terminal, applying a specially designed image processing algorithm to analyze a scene in real time, and carrying out processes such as pedestrian detection, target tracking, image segmentation and the like. The method has the advantages of low cost, small interference, good real-time performance, high accuracy, no influence on human activities and the like.
However, the current fall detection algorithm proposed based on computer vision still has some defects and shortcomings, including: 1. the traditional pedestrian detection and tracking method based on computer vision mainly adopts the manual characteristic values to carry out global characteristic detection, the characteristics are easily influenced by complex environments such as illumination change, scene change, shelter shielding and the like, the robustness of a detection and tracking algorithm is poor, and an ideal effect cannot be obtained. 2. The robustness of detecting a moving object in a complex environment can be improved by adopting a target tracking algorithm of related filtering, but the algorithm can cause the tracking drift problem due to the self scale transformation of the target object.
Disclosure of Invention
The invention aims to solve the problems in the task of detecting and judging the falling of the pedestrian by using the latest deep learning computer vision technology and construct a real-time detection method for the falling of the pedestrian, which has the advantages of simple and clear structure, various application scenes, high precision and high robustness.
The real-time detection method for pedestrian falling in the complex environment comprises the following steps:
the method comprises the following steps: preprocessing the acquired video, converting each frame of the video stream into a picture, normalizing the picture, and outputting the picture with the resolution of 416 x 416;
step two: pedestrian detection, namely taking Darknet-53 as a main network for feature extraction, carrying out FPN up-sampling on an input picture by utilizing the idea of a feature pyramid network to obtain three scale feature values and carrying out fusion, further enabling the network to adjust the priorbox of the receptive field convolution layer according to the actual size of an object groudtuth, and calculating the intersection ratio between bounding boxes to IoU:
in the formula, area (a) is the area of the original mark frame area, area (b) is the area of the candidate frame area, if the overlapping degree of the area and the candidate frame area is higher, the result tends to 1, so as to obtain the best matching image Box, and finally, the size of the detection frame is adjusted according to the condition that the size of the pedestrian is different due to the change of the distance between the pedestrian and the camera;
step three: tracking the target by adopting a sort algorithm, namely picking out the corresponding targets in all target frames detected in the pedestrian detection work, extracting the characteristics, and then calculating the similarity; predicting the current position through a Kalman filter, and associating a detection box with a target position by using a Hungarian algorithm;
step four: and (4) judging whether the pedestrian falls down, namely performing two-classification operation to judge whether the pedestrian falls down in the target area, wherein when the pedestrian stands, the aspect ratio of the identified pedestrian group route is less than or equal to 0.4, and when the pedestrian falls down, the aspect ratio is increased to about 0.7-1.2, the deflection angle is lower than a set threshold value, and the instantaneous angular acceleration is increased.
Preferably, in the second step of the present invention, during the training process, a part of the graph with a certain light and shadow effect in the COCO data set is selected to perform an individual training test on the YOLOv3 network, the data set is augmented by rotating the appropriate pedestrian image, and the ratio of the number of the graph to the number of the graph in the COCO data set is 7: the ratio of 3 is randomly divided into training sets and test sets and labeled as pedestrian or no pedestrian.
Preferably, in the third step of the present invention, first, the feature values of all target frames of the previous frame, including the center position coordinates, the aspect ratio, the height and the speed, are obtained through the kalman filter, then, the error covariance matrix is used to calculate and predict the position of the current target, and then, the correction is performed according to the following formula:
in the formula KkAs Kalman gain, ZkFor actual measurement, ZkFrom the signal estimate in the previous state and KkThe correction is carried out in such a way that,
and
the posterior state estimated values respectively representing the k moment and the k-1 moment are one of filtering results, namely the updated optimal estimation, and the finally obtained optimal estimated value is the real position of the current frame;
and secondly, realizing the association of the front frame and the rear frame through the Hungarian algorithm, calculating and solving a similarity matrix of the front frame and the rear frame by using the intersection ratio IoU of the target prediction box and the detection box as the weight of the Hungarian algorithm, and thus realizing the target matching and tracking of the front frame and the rear frame.
Preferably, a neural network-based feature extraction method is added into the sort algorithm of the invention, and the surface features and the moving target information are combined;
the degree of association of the moving object is represented by the mahalanobis distance between the prediction frame and the detection frame, namely:
in the formula, T represents transposition, djIs the jth detection box, yiExpressed as the ith predicted target position, SjFor the covariance matrix between the jth detected position and the mean tracking position,
representing its inverse matrix. The mahalanobis distance retains the result of spatial domain distribution, and the degree of association of the surface features is expressed by the minimum cosine distance between the ith track and the jth track, namely:
wherein for each detection frame djCalculating a surface feature descriptor rjAnd rj1, constructing an R for the ith tracking targetiA descriptor for storing the most recent 100 frames;
finally, the two dimensions are fused, namely:
Ci,j＝λd1(i,j)+(1-λ)d2(i,j)
where λ is a hyperparameter for adjusting the weights of the different terms, Ci,jFor the final calculation result, Ci,jThe smaller the correlation degree between the detected target and the tracked target is, the greater the correlation degree between the detected target and the tracked target is, the distance measurement has good effect on short-term prediction and matching, and the apparent information is effective on the track lost for a long time, so that the robustness of the algorithm on target loss and obstacles is improved.
Preferably, the fourth step of the invention adopts a sliding window-based feature extraction method, and stores feature data in a sliding window with a fixed size, as time increases, new data is added at the end of the window, and the leftmost data is removed;
constructing a support vector machine classifier according to the characteristic data in the sliding window to carry out falling detection training and judging whether the pedestrian falls down; in the training process, a large amount of feature data of falling samples and feature data of non-falling samples are put into the SVM, and the falling classifier is obtained by training the samples; projecting the features into a high-dimensional space by using a Gaussian kernel function as a kernel function, wherein the formula is as follows:
wherein, x and z are respectively a training sample and a testing sample, gamma belongs to a super parameter, is required to be more than 0, needs to be adjusted and defined, and represents Norm operation by | … | |, namely, takes the 'measurement' of the vector.
The invention utilizes the latest deep learning computer vision technology to solve the problems in the pedestrian falling detection and judgment task, such as tracking/detection failure caused by complex environmental factors such as scale change, shelter, illumination transformation and the like, and constructs a pedestrian falling detection algorithm solution with simple and clear structure, various application scenes, high precision and high robustness. Compared with the traditional technical scheme, the method not only improves the judgment accuracy of certain falling behaviors, but also has better robustness on the influence factors such as illumination transformation, shelters, scale change and the like.
Drawings
FIG. 1 is a flow chart of the overall scheme of the present invention.
Fig. 2 is a schematic diagram of a network structure of YOLOv 3.
Detailed Description
As shown in fig. 1, in a real-time detection method for pedestrian fall in a complex environment, preprocessing is performed first: and converting each frame of the video stream into a picture, and normalizing the picture to ensure that the picture with the resolution of 416 x 416 is output.
Pedestrian detection
In the task of image target detection, the algorithm based on the deep convolutional neural network is widely applied due to the advantages of the algorithm in feature extraction, and is obviously superior to the traditional detection method. Such algorithms can be divided into three categories: 1) a target identification algorithm based on the region suggestions; 2) a detection algorithm based on learning search; 3) and (3) a target detection algorithm based on a regression mode. Because the first and second algorithms are slow in detection speed and poor in detection accuracy, the target detection algorithm based on the regression mode is adopted for pedestrian detection, and the real-time requirement is met while the detection precision is ensured.
As shown in fig. 2, the structure diagram of YOLOv3 network, the network is divided into two parts: feature extraction and three-time sampling output. The invention adopts Darknet-53 as a main network for feature extraction, and is different from the traditional CNN network structure in that the Darknet-53 abandons a common pooling layer and a full connection layer, a Leaky-ReLU activation function is carried after a convolutional layer, and no deviation is used in the input of the activation function, so that the purposes of simplifying a model and reducing the dimension and parameters of a convolutional kernel channel are achieved, the feature extraction capability of the model is enhanced, and certain timeliness and sensitivity during pedestrian detection are improved.
In order to detect small targets, the invention carries out multi-scale fusion prediction on the characteristics output by Darknet-53. By utilizing the idea of Feature Pyramid (Feature Pyramid Networks), an FPN (field programmable gate array) up-sampling mode is carried out on an input picture to obtain three scale Feature values, fusion is carried out, detection is carried out on a plurality of Feature maps (Feature maps), and the accuracy rate of detecting small targets is obviously improved. The method is improved based on the characteristic of multiple scales, so that the convolution layers of different receptive fields of the network are used as independent outputs to perform classification calculation, the network can adjust the priorbox of the receptive field convolution layer according to the actual size of an object group, and the intersection ratio between bounding boxes is calculated to be IoU:
in the formula, area (A) is the area of the original mark frame region, and area (B) is the area of the candidate frame region. If the overlapping degree of the two images is higher, the result tends to 1, so that the optimal matching image Box is obtained, and finally the size of the detection frame can be adjusted according to the condition that the sizes of pedestrians are different due to the change of the distance between the pedestrians and the camera.
Further, in order to solve the influence of light and shadow transformation, in the training process, the invention selects a part of graphics with certain light and shadow effect in a COCO (common Objects in context) data set to perform individual training test on a YOLOv3 network, amplifies the data set in a mode of rotating an appropriate pedestrian image, and performs the following steps according to the ratio of 7: the ratio of 3 is randomly divided into training sets and test sets and labeled as pedestrian or no pedestrian. The anti-interference capability of the trained network on the light and shadow interference is obviously improved, and the effect of the trained network is obviously higher than that of an SSD network formed by only 600 data sets.
Target tracking
In order to more efficiently analyze the extracted pedestrian target and reduce the operation time of continuous detection, the invention adopts the sort algorithm to track the target. That is, corresponding objects in all object frames detected in the pedestrian detection work are extracted, feature extraction (including apparent features or motion features) is performed, and then similarity calculation is performed. The current position is predicted through a Kalman filter, and the detection box and the target position are associated by using a Hungarian algorithm.
Firstly, acquiring characteristic values of all target frames of a previous frame through a Kalman filter, wherein the characteristic values comprise a central position coordinate, an aspect ratio, a height and a speed, then calculating and predicting the position of a current target by utilizing an error covariance matrix, and then correcting according to the following formula:
in the formula KkAs Kalman gain, ZkFor actual measurement, ZkFrom the signal estimate in the previous state and KkThe correction is carried out in such a way that,
and
the posterior state estimated values respectively representing the k moment and the k-1 moment are one of filtering results, namely the updated optimal estimation, and the finally obtained optimal estimated value is the real position of the current frame;
and secondly, realizing the association of the front frame and the rear frame through the Hungarian algorithm, calculating and solving a similarity matrix of the front frame and the rear frame by using the intersection ratio IoU of the target prediction box and the detection box as the weight of the Hungarian algorithm, and thus realizing the target matching and tracking of the front frame and the rear frame.
Compared with other algorithms, the Sort algorithm has the advantages that the detection and tracking speed is faster, the accuracy is high, and the accuracy is reduced when an occlusion appears in a view screen. Therefore, the sort algorithm is optimized to a certain extent, a characteristic extraction method based on the neural network is added, more reliable measurement is adopted to replace correlation measurement, and the surface characteristic and the motion target information are combined.
The degree of association of the moving object is represented by the mahalanobis distance between the prediction frame and the detection frame, namely:
in the formula, T represents transposition, djIs the jth detection box, yiExpressed as the ith predicted target position, SjFor the covariance matrix between the jth detected position and the mean tracking position,
representing its inverse matrix. The mahalanobis distance retains the result of spatial domain distribution, and the degree of association of the surface features is expressed by the minimum cosine distance between the ith track and the jth track, namely:
in which for eachA detection frame djCalculating a surface feature descriptor rjAnd rj1, constructing an R for the ith tracking targetiA descriptor for storing the most recent 100 frames;
finally, the two dimensions are fused, namely:
Ci,j＝λd1(i,j)+(1-λ)d2(i,j)
where λ is a hyperparameter for adjusting the weights of the different terms, Ci,jFor the final calculation result, Ci,jThe smaller the correlation degree between the detected target and the tracked target is, the greater the correlation degree between the detected target and the tracked target is, the distance measurement has good effect on short-term prediction and matching, and the apparent information is effective on the track lost for a long time, so that the robustness of the algorithm on target loss and obstacles is improved.
1. Fall determination
Through the foregoing detection and extraction steps, this process only requires a binary classification work of whether or not the pedestrian falls in the target area. When a pedestrian stands, the aspect ratio of the identified pedestrian group is less than or equal to 0.4, and when the pedestrian falls, the aspect ratio is increased to be about 0.7 to 1.2, the deflection angle is lower than a certain threshold value (the invention is set to be 37 degrees), and the instantaneous angular acceleration is increased. The invention comprehensively considers the three factors to carry out the falling judgment, the three factors not only have stronger independence and accord with the comprehensive judgment condition, but also can avoid higher space dimensionality of the feature vectors, complex design of the classifier and poorer system real-time property caused by excessive selected feature vectors.
Considering that the falling behavior is a continuous action, compared with the traditional falling judgment algorithm, the invention adopts a characteristic extraction method based on a sliding window. The characteristic data is stored in a sliding window with a fixed size, new data is added at the end of the window as time increases, and the leftmost data is removed.
Constructing a support vector machine classifier according to the characteristic data in the sliding window to carry out falling detection training and judging whether the pedestrian falls down; in the training process, a large amount of feature data of falling samples and feature data of non-falling samples are put into the SVM, and the falling classifier is obtained by training the samples; projecting the features into a high-dimensional space by using a Gaussian kernel function as a kernel function, wherein the formula is as follows:
wherein, x and z are respectively a training sample and a testing sample, gamma belongs to a super parameter, is required to be more than 0, needs to be adjusted and defined, and represents Norm operation by | … | |, namely, takes the 'measurement' of the vector. Compared with a polynomial kernel function, a character string kernel function and the like, the kernel function has the advantages of less required parameters, high effectiveness and reduction of the operation complexity.
The pedestrian detection mark frame is basically matched with the real mark frame, only has deviation on the details of hands or feet, and does not influence the following target tracking and falling judgment process.
The results of the tests performed on this data set are shown in the following table:
TABLE 1 comparison of the results
As can be seen from the table, the false alarm rate, the omission factor and the accuracy rate of the invention are greatly improved compared with the traditional invention.
The pedestrian detection system can accurately detect pedestrians and track the pedestrians in real time in a strong light environment, and has good anti-interference performance on light and shadow transformation; when a person is far away from the camera, the dimension of the detection frame is changed adaptively; the problem that the target is lost due to the fact that the shielding object appears during tracking can be effectively solved.
Claims (5)
1. The real-time detection method for pedestrian falling under the complex environment is characterized by comprising the following steps of:
the method comprises the following steps: preprocessing the acquired video, converting each frame of the video stream into a picture, normalizing the picture, and outputting the picture with the resolution of 416 x 416;
step two: pedestrian detection, namely taking Darknet-53 as a main network for feature extraction, carrying out FPN up-sampling on an input picture by utilizing the idea of a feature pyramid network to obtain three scale feature values and carrying out fusion, further enabling the network to adjust the priorbox of the receptive field convolution layer according to the actual size of an object groudtuth, and calculating the intersection ratio between bounding boxes to IoU:
in the formula, area (a) is the area of the original mark frame area, area (b) is the area of the candidate frame area, if the overlapping degree of the area and the candidate frame area is higher, the result tends to 1, so as to obtain the best matching image Box, and finally, the size of the detection frame is adjusted according to the condition that the size of the pedestrian is different due to the change of the distance between the pedestrian and the camera;
step three: tracking the target by adopting a sort algorithm, namely picking out the corresponding targets in all target frames detected in the pedestrian detection work, extracting the characteristics, and then calculating the similarity; predicting the current position through a Kalman filter, and associating a detection box with a target position by using a Hungarian algorithm;
step four: and (4) judging whether the pedestrian falls down, namely performing two-classification operation to judge whether the pedestrian falls down in the target area, wherein when the pedestrian stands, the aspect ratio of the identified pedestrian group route is less than or equal to 0.4, and when the pedestrian falls down, the aspect ratio is increased to 0.7-1.2, the deflection angle is lower than a set threshold value, and the instantaneous angular acceleration is increased.
2. The method for detecting the fall of a pedestrian under the complex environment according to claim 1, wherein in the second step, a part of the graph with a certain light and shadow effect in the COCO data set is selected during the training process to perform an individual training test on the YOLOv3 network, the data set is augmented by rotating the proper pedestrian class image, and the step is performed according to a rule of 7: the ratio of 3 is randomly divided into training sets and test sets and labeled as pedestrian or no pedestrian.
3. The method according to claim 1, wherein in the third step, the feature values of all target frames in the previous frame, including the coordinates of the center position, the aspect ratio, the height and the speed, are obtained through a kalman filter, and then the position of the current target is predicted by calculating the error covariance matrix, and then the correction is performed according to the following formula:
in the formula KkAs Kalman gain, ZkFor actual measurement, ZkFrom the signal estimate in the previous state and KkThe correction is carried out in such a way that,
and
the posterior state estimated values respectively representing the k moment and the k-1 moment are one of filtering results, namely the updated optimal estimation, and the finally obtained optimal estimated value is the real position of the current frame;
and secondly, realizing the association of the front frame and the rear frame through the Hungarian algorithm, calculating and solving a similarity matrix of the front frame and the rear frame by using the intersection ratio IoU of the target prediction box and the detection box as the weight of the Hungarian algorithm, and thus realizing the target matching and tracking of the front frame and the rear frame.
4. The method for detecting the falling of the pedestrian in the complex environment in real time according to claim 1 or 3, wherein a neural network-based feature extraction method is added to the sort algorithm, and the surface features and the moving target information are combined;
the degree of association of the moving object is represented by the mahalanobis distance between the prediction frame and the detection frame, namely:
in the formula, T represents transposition, djIs the jth detection box, yiExpressed as the ith predicted target position, SjFor the covariance matrix between the jth detected position and the mean tracking position,
representing its inverse matrix. The mahalanobis distance retains the result of spatial domain distribution, and the degree of association of the surface features is expressed by the minimum cosine distance between the ith track and the jth track, namely:
wherein for each detection frame djCalculating a surface feature descriptor rjAnd rj1, constructing an R for the ith tracking targetiA descriptor for storing the most recent 100 frames;
finally, the two dimensions are fused, namely:
Ci,j＝λd1(i,j)+(1-λ)d2(i,j)
where λ is a hyperparameter for adjusting the weights of the different terms, Ci,jFor the final calculation result, Ci,jThe smaller the correlation degree between the detected target and the tracked target is, the greater the correlation degree between the detected target and the tracked target is, the distance measurement has good effect on short-term prediction and matching, and the apparent information is effective on the track lost for a long time, so that the robustness of the algorithm on target loss and obstacles is improved.
5. The method for detecting the fall of the pedestrian in the complex environment in real time as claimed in claim 1, wherein the step four adopts a sliding window-based feature extraction method, feature data are stored in a sliding window with a fixed size, new data are added at the end of the window as time increases, and leftmost data are removed;
constructing a support vector machine classifier according to the characteristic data in the sliding window to carry out falling detection training and judging whether the pedestrian falls down; in the training process, a large amount of feature data of falling samples and feature data of non-falling samples are put into the SVM, and the falling classifier is obtained by training the samples; projecting the features into a high-dimensional space by using a Gaussian kernel function as a kernel function, wherein the formula is as follows:
wherein, x and z are respectively a training sample and a testing sample, gamma belongs to a super parameter, is required to be more than 0, needs parameter adjustment definition, and is | |.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011427824.9A CN112541424A (en) | 2020-12-07 | 2020-12-07 | Real-time detection method for pedestrian falling under complex environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011427824.9A CN112541424A (en) | 2020-12-07 | 2020-12-07 | Real-time detection method for pedestrian falling under complex environment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112541424A true CN112541424A (en) | 2021-03-23 |
Family
ID=75019655
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011427824.9A Pending CN112541424A (en) | 2020-12-07 | 2020-12-07 | Real-time detection method for pedestrian falling under complex environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112541424A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113076899A (en) * | 2021-04-12 | 2021-07-06 | 华南理工大学 | High-voltage transmission line foreign matter detection method based on target tracking algorithm |
| CN113269033A (en) * | 2021-04-12 | 2021-08-17 | 南京瀚元科技有限公司 | Fall detection algorithm based on video sequence |
| CN113327244A (en) * | 2021-06-25 | 2021-08-31 | 南京爱奇艺智能科技有限公司 | Handle controller LED lamp positioning method and system based on computer vision |
| CN114220119A (en) * | 2021-11-10 | 2022-03-22 | 深圳前海鹏影数字软件运营有限公司 | Human body posture detection method, terminal device and computer readable storage medium |
| CN114463788A (en) * | 2022-04-12 | 2022-05-10 | 深圳市爱深盈通信息技术有限公司 | Fall detection method, system, computer equipment and storage medium |
| CN116824549A (en) * | 2023-08-29 | 2023-09-29 | 所托(山东)大数据服务有限责任公司 | Target detection method and device based on multi-detection network fusion and vehicle |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109145696A (en) * | 2017-06-28 | 2019-01-04 | 安徽清新互联信息科技有限公司 | A kind of Falls Among Old People detection method and system based on deep learning |
| CN109670396A (en) * | 2018-11-06 | 2019-04-23 | 华南理工大学 | A kind of interior Falls Among Old People detection method |
| CN110532850A (en) * | 2019-07-02 | 2019-12-03 | 杭州电子科技大学 | A kind of fall detection method based on video artis and hybrid classifer |
| CN110895671A (en) * | 2018-09-13 | 2020-03-20 | 纬创资通股份有限公司 | Fall detection method and electronic system using same |
| CN111488795A (en) * | 2020-03-09 | 2020-08-04 | 天津大学 | Real-time pedestrian tracking method applied to unmanned vehicle |
| CN111724566A (en) * | 2020-05-20 | 2020-09-29 | 同济大学 | Pedestrian falling detection method and device based on intelligent lamp pole video monitoring system |
| CN111931582A (en) * | 2020-07-13 | 2020-11-13 | 中国矿业大学 | Image processing-based highway traffic incident detection method |
-
2020
- 2020-12-07 CN CN202011427824.9A patent/CN112541424A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109145696A (en) * | 2017-06-28 | 2019-01-04 | 安徽清新互联信息科技有限公司 | A kind of Falls Among Old People detection method and system based on deep learning |
| CN110895671A (en) * | 2018-09-13 | 2020-03-20 | 纬创资通股份有限公司 | Fall detection method and electronic system using same |
| CN109670396A (en) * | 2018-11-06 | 2019-04-23 | 华南理工大学 | A kind of interior Falls Among Old People detection method |
| CN110532850A (en) * | 2019-07-02 | 2019-12-03 | 杭州电子科技大学 | A kind of fall detection method based on video artis and hybrid classifer |
| CN111488795A (en) * | 2020-03-09 | 2020-08-04 | 天津大学 | Real-time pedestrian tracking method applied to unmanned vehicle |
| CN111724566A (en) * | 2020-05-20 | 2020-09-29 | 同济大学 | Pedestrian falling detection method and device based on intelligent lamp pole video monitoring system |
| CN111931582A (en) * | 2020-07-13 | 2020-11-13 | 中国矿业大学 | Image processing-based highway traffic incident detection method |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113076899A (en) * | 2021-04-12 | 2021-07-06 | 华南理工大学 | High-voltage transmission line foreign matter detection method based on target tracking algorithm |
| CN113269033A (en) * | 2021-04-12 | 2021-08-17 | 南京瀚元科技有限公司 | Fall detection algorithm based on video sequence |
| CN113076899B (en) * | 2021-04-12 | 2023-04-07 | 华南理工大学 | High-voltage transmission line foreign matter detection method based on target tracking algorithm |
| CN113327244A (en) * | 2021-06-25 | 2021-08-31 | 南京爱奇艺智能科技有限公司 | Handle controller LED lamp positioning method and system based on computer vision |
| CN114220119A (en) * | 2021-11-10 | 2022-03-22 | 深圳前海鹏影数字软件运营有限公司 | Human body posture detection method, terminal device and computer readable storage medium |
| CN114220119B (en) * | 2021-11-10 | 2022-08-12 | 深圳前海鹏影数字软件运营有限公司 | Human body posture detection method, terminal device and computer readable storage medium |
| CN114463788A (en) * | 2022-04-12 | 2022-05-10 | 深圳市爱深盈通信息技术有限公司 | Fall detection method, system, computer equipment and storage medium |
| CN116824549A (en) * | 2023-08-29 | 2023-09-29 | 所托(山东)大数据服务有限责任公司 | Target detection method and device based on multi-detection network fusion and vehicle |
| CN116824549B (en) * | 2023-08-29 | 2023-12-08 | 所托(山东)大数据服务有限责任公司 | Target detection method and device based on multi-detection network fusion and vehicle |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112541424A (en) | Real-time detection method for pedestrian falling under complex environment | |
| Nadeem et al. | Automatic human posture estimation for sport activity recognition with robust body parts detection and entropy markov model | |
| WO2019101220A1 (en) | Deep learning network and average drift-based automatic vessel tracking method and system | |
| CN107016357B (en) | Video pedestrian detection method based on time domain convolutional neural network | |
| CN109766796B (en) | Deep pedestrian detection method for dense crowd | |
| CN106570490B (en) | A kind of pedestrian's method for real time tracking based on quick clustering | |
| CN110782483A (en) | Multi-view multi-target tracking method and system based on distributed camera network | |
| CN107194950B (en) | Multi-person tracking method based on slow feature analysis | |
| CN103886325A (en) | Cyclic matrix video tracking method with partition | |
| CN106575363A (en) | Method for tracking keypoints in scene | |
| CN112541403B (en) | Indoor personnel falling detection method by utilizing infrared camera | |
| CN116403139A (en) | Visual tracking and positioning method based on target detection | |
| Yadav | Efficient method for moving object detection in cluttered background using Gaussian Mixture Model | |
| Li et al. | The application of Yolov4 and a new pedestrian clustering algorithm to implement social distance monitoring during the COVID-19 pandemic | |
| CN117036404A (en) | Monocular thermal imaging simultaneous positioning and mapping method and system | |
| CN115527269A (en) | Intelligent human body posture image identification method and system | |
| CN113971688B (en) | Anchor-free multi-target tracking method for enhancing ID re-identification | |
| Xu-Wei et al. | Real-time hand tracking based on YOLOv4 model and Kalman filter | |
| CN116862832A (en) | Three-dimensional live-action model-based operator positioning method | |
| Wang et al. | Pmds-slam: Probability mesh enhanced semantic slam in dynamic environments | |
| CN117037204A (en) | Tumble detection method, tumble detection device, electronic equipment and computer program product | |
| CN113781521A (en) | Improved YOLO-Deepsort-based bionic robot fish detection and tracking method | |
| Li et al. | Review of multi-object tracking based on deep learning | |
| CN117372844B (en) | Skeleton detection and fall detection method based on improved space-time adaptive graph convolution | |
| KR20070082078A (en) | Vehicle tracking system using compare background vector weight |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |