CN117011946B - Unmanned rescue method based on human behavior recognition - Google Patents
Unmanned rescue method based on human behavior recognition Download PDFInfo
- Publication number
- CN117011946B CN117011946B CN202311292989.3A CN202311292989A CN117011946B CN 117011946 B CN117011946 B CN 117011946B CN 202311292989 A CN202311292989 A CN 202311292989A CN 117011946 B CN117011946 B CN 117011946B
- Authority
- CN
- China
- Prior art keywords
- coefficient
- value
- abnormal
- monitoring target
- skeleton key
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000012544 monitoring process Methods 0.000 claims abstract description 125
- 230000002159 abnormal effect Effects 0.000 claims abstract description 90
- 238000012549 training Methods 0.000 claims abstract description 14
- 230000004044 response Effects 0.000 claims abstract description 6
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 230000033001 locomotion Effects 0.000 abstract description 13
- 230000008859 change Effects 0.000 abstract description 12
- 238000004458 analytical method Methods 0.000 abstract description 6
- 230000000694 effects Effects 0.000 abstract description 3
- 238000012545 processing Methods 0.000 abstract description 2
- 230000006399 behavior Effects 0.000 description 32
- 206010000117 Abnormal behaviour Diseases 0.000 description 10
- 230000005856 abnormality Effects 0.000 description 7
- 206010013647 Drowning Diseases 0.000 description 6
- 230000009182 swimming Effects 0.000 description 5
- 239000002699 waste material Substances 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 206010001488 Aggression Diseases 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B63—SHIPS OR OTHER WATERBORNE VESSELS; RELATED EQUIPMENT
- B63C—LAUNCHING, HAULING-OUT, OR DRY-DOCKING OF VESSELS; LIFE-SAVING IN WATER; EQUIPMENT FOR DWELLING OR WORKING UNDER WATER; MEANS FOR SALVAGING OR SEARCHING FOR UNDERWATER OBJECTS
- B63C9/00—Life-saving in water
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
- G06Q10/047—Optimisation of routes or paths, e.g. travelling salesman problem
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
- G06Q50/265—Personal security, identity or safety
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/62—Extraction of image or video features relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/04—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons
- G08B21/0407—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons based on behaviour analysis
Abstract
The application relates to the field of data processing, in particular to an unmanned rescue method based on human behavior recognition, which comprises the following steps: acquiring skeleton key points of a monitoring target in an acquired video image according to a pre-training model of the preset human body gesture recognition; calculating abnormal coefficients of key points of a single skeleton, and generating an abnormal coefficient sequence; calculating an abnormal value at each moment to generate an abnormal value sequence at each moment; calculating a first coefficient; responding to the fact that the first coefficient is larger than a preset first coefficient threshold value, obtaining a first abnormal value sequence and a second abnormal value sequence, calculating similarity, and generating a similarity distance value; calculating a second coefficient according to the similarity distance value; an alarm signal is generated in response to the second coefficient being greater than a second coefficient threshold. The method has the effect of carrying out rescue judgment on the monitoring target according to the change of the abnormal coefficient of the skeleton key point, and improves the accuracy and stability of motion analysis and rescue judgment.
Description
Technical Field
The application relates to the field of data processing, in particular to an unmanned rescue method based on human behavior recognition.
Background
Human behavior recognition refers to a technology for recognizing and understanding human behaviors by analyzing and explaining physiological characteristics such as actions, gestures and the like of a person, and is generally dependent on methods and tools in the fields of sensors, computer vision, machine learning and the like.
In the prior art, through carrying out drowning monitoring and human behavior recognition to the sea, can automatic monitoring and discernment latent dangerous behavior such as drowning, drowning analysis is carried out to the limb behavior change of drowning accessible monitoring target, is favorable to providing timely early warning and rescue, improves marine rescue efficiency.
However, in the prior art, abnormal behavior recognition is only performed by the intensity of limb behavior change, and due to the diversity of swimming postures, erroneous judgment is easily generated under the condition of performing motion analysis only according to limb change, so that effective intelligent rescue is difficult to realize, and rescue resources are wasted.
Disclosure of Invention
In order to improve accuracy and stability of motion analysis and rescue judgment and reduce waste of rescue resources, the application provides an unmanned rescue method based on human behavior recognition, which adopts the following technical scheme:
the unmanned rescue method based on human behavior recognition comprises the following steps: acquiring skeleton key points of a monitoring target in an acquired video image according to a pre-training model of the preset human body gesture recognition; calculating an abnormal coefficient of a single skeleton key point to generate an abnormal coefficient sequence, wherein the abnormal coefficient is the speed variation of the single skeleton key point; calculating an abnormal value at each moment to generate an abnormal value sequence at each moment; calculating a first coefficient, wherein the first coefficient is an absolute value of a difference value between an abnormal value at the latest moment and a mean value of abnormal values at a plurality of moments before the latest moment; responding to the fact that the first coefficient is larger than a preset first coefficient threshold value, obtaining a first abnormal value sequence and a second abnormal value sequence, calculating similarity, and generating a similarity distance value, wherein the first abnormal value is an abnormal value of a monitoring target before a target moment, the second abnormal value is an abnormal value of the monitoring target after the target moment, and the target moment is the moment when the first coefficient is larger than the first coefficient threshold value; calculating a second coefficient according to the similarity distance value; an alarm signal is generated in response to the second coefficient being greater than a second coefficient threshold.
Through the technical scheme, the skeleton key points of the monitoring target in the video image acquired by the camera equipment are acquired through the pre-training model for human body gesture recognition, the abnormal coefficients of the skeleton key points are calculated, whether different parts of the current monitoring target have violent abnormal behaviors or not is judged according to the overall abnormal coefficient change of the skeleton key points, then the overall violent behavior analysis is carried out on the monitoring target, the monitoring target is used for rescue judgment, the accuracy and the stability of the rescue judgment are improved, and the waste of rescue resources is reduced.
Optionally, in acquiring skeleton key points of a monitoring target in the acquired video image, the method includes the steps of: acquiring video images acquired by each camera device; and detecting skeleton key points of the video image based on a pre-training model of human body gesture recognition according to the preset skeleton key point number value, and generating a coordinate sequence of the monitoring target in the video image.
Optionally, calculating an anomaly coefficient of a single skeleton key point to generate an anomaly coefficient sequence, including the steps of: calculating the Euclidean distance and time interval of coordinates of the same skeleton key point in the coordinate sequence between two continuous frames of the video image; according to the Euclidean distance and the time interval, calculating an anomaly coefficient, wherein the calculation formula is as follows:wherein->Anomaly coefficient of jth skeleton key point of ith monitored object between the a-th frame and the a+1-th frame of video image, +.>For the first superparameter,/->For monitoring the depth value of the jth skeleton key point of the target in the a-th frame of the video image,/>Depth value of jth skeleton key point in a (a+1) th frame of video image for ith monitoring target,/->Distance value between the a-th frame and the a+1-th frame of the video image relative to the image capturing apparatus for the i-th monitoring target +.>For the value of Euclidean distance between the first coordinate of the jth skeleton key point and the second coordinate of the (a+1) th frame,/for the jth skeleton key point>Is a time interval.
Optionally, in calculating the outlier at each time, the calculation formula is:wherein->Abnormal value at the latest time of the ith monitoring target,/->Generating a variance value in an abnormal coefficient sequence for the jth skeleton key point of the current ith monitoring target at the latest moment; />And n is the total number of skeleton key points, wherein the value is the abnormal coefficient value of the jth skeleton key point at the current moment.
Optionally, the calculation formula of the second coefficient is:wherein->Second coefficient of the ith monitoring target for the target moment,/>For the sequence of outliers of the ith monitoring target before the target moment,/th monitoring target>The average value of the abnormal value sequence from the target time to the latest time of the ith monitoring target is obtained.
Optionally, the method further comprises the steps of: and responding to the alarm signal, and generating coordinate information of a monitoring target and a rescue route.
Optionally, in response to the alarm signal, sending coordinate information of a monitoring target and a rescue route, including the steps of: responding to the alarm signal, acquiring all position coordinates of skeleton key points of the monitoring target in the video image, and calculating the average value of all position coordinates to obtain the average value coordinates of the skeleton key points; acquiring actual geographic coordinates of a monitoring target; and generating a rescue route through a path planning algorithm according to the geographical coordinates of the rescue boat and the geographical coordinates of the monitoring target.
The application has the following technical effects:
1. the method comprises the steps of acquiring skeleton key points of a monitoring target in a video image acquired by camera equipment through a pre-training model for human body gesture recognition, calculating abnormal coefficients of a plurality of skeleton key points, judging whether different parts of the current monitoring target have intense abnormal behaviors according to overall abnormal coefficient changes of the skeleton key points, and then analyzing the overall intense behaviors of the monitoring target and using the monitoring target for rescue judgment, so that accuracy and stability of rescue judgment are improved, and waste of rescue resources is reduced.
2. The abnormal behavior recognition during the target swimming is monitored through the change of the skeleton key points on the time sequence, after the abnormal behavior is judged, the transient continuous judgment is carried out, and whether the person to be rescued gets out of the way or not is judged, so that an effective and accurate abnormal behavior recognition result is obtained.
3. When the monitoring target needs to be rescued, according to all position coordinates of the skeleton key points corresponding to the monitoring target in the shooting equipment in the monitoring image, the average value among all coordinates of all skeleton key points is calculated, the skeleton key point average value coordinates are obtained, according to the calibration relation between the camera coordinate system and the real coordinate system, the real coordinate positions corresponding to the skeleton key point average value coordinates corresponding to the monitoring target in the real world are obtained, and then the coordinate positions are sent to the unmanned rescue boat, so that the intelligent rescue effect of unmanned monitoring is realized.
Drawings
The above, as well as additional purposes, features, and advantages of exemplary embodiments of the present application will become readily apparent from the following detailed description when read in conjunction with the accompanying drawings. Several embodiments of the present application are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar or corresponding parts and in which:
fig. 1 is a flowchart of a method of steps S1-S8 in an unmanned rescue method based on human behavior recognition according to an embodiment of the present application.
Fig. 2 is a flowchart of a method of steps S10-S11 in an unmanned rescue method based on human behavior recognition according to an embodiment of the present application.
Fig. 3 is a flowchart of a method of steps S20-S21 in an unmanned rescue method based on human behavior recognition according to an embodiment of the present application.
Fig. 4 is a flowchart of a method of steps S80-S82 in an unmanned rescue method based on human behavior recognition according to an embodiment of the present application.
Fig. 5 is a logic block diagram of an unmanned rescue system based on human behavior recognition according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all, of the embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
It should be understood that when the terms "first," "second," and the like are used in the claims, specification, and drawings of this application, they are used merely for distinguishing between different objects and not for describing a particular sequential order. The terms "comprises" and "comprising," when used in the specification and claims of this application, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
The embodiment of the application discloses an unmanned rescue method based on human behavior recognition, which is mainly used for recognizing and alarming the drowning situation of a swimmer in outdoor swimming scenes such as seaside, lake side and the like, and referring to fig. 1, the unmanned rescue method comprises the following steps S1-S8, wherein the method comprises the following steps:
s1: and acquiring skeleton key points of the monitoring target in the acquired video image according to a pre-training model of the preset human body gesture recognition. Referring to fig. 2, step S1 includes steps S10 to S11, specifically as follows:
s10: video images acquired by each image pickup apparatus are acquired.
A plurality of camera shooting devices are arranged on a sea level field at sea, wherein the camera shooting devices transmit collected sea level video images at sea to a preset data processing center in a wired or wireless data transmission mode.
S11: and detecting skeleton key points of the video image based on a pre-training model of human body gesture recognition according to the preset skeleton key point number value, and generating a coordinate sequence of the monitoring target in the video image.
The method comprises the steps that a data processing center obtains a sea level video image at the sea side, and then obtains skeleton key points of an ith monitoring target in the video image collected by a single camera device, wherein the monitoring target is a swimming person, and details are omitted later.
The method comprises the steps that a pre-training model of OpenPose (human body gesture recognition) is utilized to detect skeleton key points of videos acquired by single camera equipment, and a coordinate sequence of a j-th skeleton key point of an i-th monitoring target in a video image can be acquired:。
the number of the skeleton key points selected by the method is n=18, and the skeleton key points can be adjusted by a user according to specific implementation scenes.
The pre-training model of openPose is disclosed in the prior art, and the pre-training model of openPose is called in the prior art, which is not described in detail in the present application. The working principle of openPose is that pose estimation is carried out on each video frame, so that skeleton key point information of all human bodies in a current frame is extracted. And then matching and tracking skeleton key points among different frames through a matching and association algorithm, so that continuous tracking of skeleton key points of a human body is realized, and skeleton key point data from appearance to disappearance of a single monitoring target in a video can be obtained.
In other embodiments, the user may replace the pre-trained model of openPose with the pre-trained model of skeletal keypoint detection with tracking functionality, depending on the implementation scenario.
S2: and calculating an abnormal coefficient of the single skeleton key point to generate an abnormal coefficient sequence, wherein the abnormal coefficient is the speed variation of the single skeleton key point.
Specifically, the coordinate sequence of the jth skeleton key point of the ith monitored target in the video image in the video acquired by the single camera equipmentThen, two continuous frames of video images and the time interval between the two frames of video images are acquired, wherein the time interval is a fixed parameterThe frame rate value set by the image pickup apparatus is acquired.
The speed change amount of the jth skeleton key point of the ith monitored target=the euclidean distance/time interval of the jth skeleton key point coordinate of the ith monitored target between two consecutive frames of video images.
Since the speed variation amplitude is relatively stable between two continuous frames of video images when no severe motion is generated, the smaller the value of the speed variation is, the smaller the abnormal coefficient of the skeleton key point is.
When the video image is acquired, the area of the acquired monitoring target in the picture is small when the video image is far from the monitoring target, and the area of the acquired monitoring target in the picture is large when the video image is near to the monitoring target, so that inaccurate conditions can occur when the speed variation of the key points of the single skeleton is used as an abnormal coefficient, in order to reduce the influence, the distance of the monitoring target needs to be judged, and the interference when the speed variation of the shooting distance to the key points of the single skeleton is used as a corresponding abnormal coefficient is eliminated.
In order to acquire a distance value of an ith monitoring target in video data relative to image capturing equipment, the scheme selects a fake Depth camera to acquire Depth images and Depth information of each pixel point in the Depth images, wherein RGB (Red, green, blue, red, green and Blue) images of the image capturing equipment and the Depth images need to be aligned, an alignment process is not repeated in the scheme in the prior art, or an aligned RGB-D (Red, green, blue-Depth, red, green and Blue-Depth) camera is selected, and then R, G, B, D (Red, green, blue, depth, red, green and Blue) data information of a single pixel point is further acquired.
Referring to fig. 3, step S2 includes steps S20 to S21, specifically as follows:
s20: and calculating the Euclidean distance and time interval of coordinates of the same skeleton key point in the coordinate sequence between two continuous frames of the video image.
S21: and calculating an anomaly coefficient according to the Euclidean distance and the time interval.
Calculating anomaly coefficient of jth skeleton key point of ith monitoring target between the (a) th frame and the (a+1) th frameThe calculation formula is as follows:
wherein,for the first super parameter, which is used for adjusting the relative relation between the depth value and the speed change between the monitoring target and the image capturing device, the user can adjust according to the specific implementation scene, and the application takes c=0.2.
Depth values of jth skeleton key points in an a-th frame of the video image are monitored for the target.
And (3) the depth value of the j-th skeleton key point in the a+1-th frame of the video image for the i-th monitoring target.
Distance value of the ith monitoring target with respect to the image capturing apparatus in the a-th frame and the a+1-th frame of the video image, specifically, +.>The smaller the value of the current skeleton key point is, the data of the current skeleton key point is still normal data when the current skeleton key point has larger speed change, the larger limb change of the current monitoring target cannot be considered, the behavior is normal, and rescue is not needed.
First coordinate of jth skeleton key point in a th frame +.>And the second coordinates of the a+1th frameThe Euclidean distance between them, calculated as a known matter is not described in detail in this application,/->Is a time interval. />The larger the value of (c) is, the greater the likelihood that at a time interval, the greater the limb movement may occur at the jth skeletal key of the ith monitored object, and the greater the safety risk. Wherein the first coordinate->And a second coordinateIs the position coordinates of skeleton key points in the video image.
The greater the value of the anomaly coefficient is, the greater limb movement is likely to occur between the (a) frame and the (a+1) frame of the video image for the (j) th skeleton key point of the (i) th monitoring target, and the greater the possibility that the safety risk exists.
Obtaining an abnormal coefficient sequence corresponding to the j-th skeleton key point of the i-th monitoring target according to the plurality of abnormal coefficients, and marking the abnormal coefficient sequence as。
S3: and calculating the abnormal value at each moment to generate an abnormal value sequence at each moment.
Because a single monitoring target often needs a plurality of skeleton key points to perform joint analysis when performing behavior recognition by using the skeleton key points, the single monitoring target can be obtained through an OpenPose pre-training modelObtaining n=18 skeleton key points. Therefore, the method can obtain n=18 abnormal coefficient sequences corresponding to n=18 skeleton key points of the ith monitoring targetWhere n=18 anomaly coefficient sequences are identical in data length.
The monitoring target is a person with limb cooperative capability, so that the person should have unique abnormal coefficient at the same moment, and different skeleton key points correspond to different human body parts, so that the abnormal coefficient values of the different skeleton key points are different, for example, when the person swims normally, skeleton key points corresponding to parts such as head, shoulders and the like are relatively stable, arms and legs keep moving, heads are relatively stable, the abnormal coefficient values corresponding to the skeleton key points of the heads are smaller, arms continuously move, the abnormal coefficient values corresponding to the skeleton key points of the arms are larger, and when severe movement occurs, the abnormal coefficient values of the skeleton key points of the arms possibly indicate that the risk is large, and the monitoring target is swimming at the moment. However, when the monitoring target is struggled in drowning, not only the arm will struggle vigorously, the whole person produces great violent movement, so the abnormal value of the abnormal coefficient needs to be calculated, and the calculation formula of the abnormal value is as follows:
wherein the method comprises the steps ofn is the total number of skeleton key points, and j represents traversal of n.
For the abnormal value of the ith monitoring target at the latest moment, the greater the abnormal value corresponding to the current ith monitoring target at the current latest moment, the greater the possibility of injury risk.
The jth skeleton switch of the current ith monitoring targetThe key point has generated a variance value in the sequence of anomaly coefficients at the latest instant,/for the key point>The larger the value of (c) is, the larger the fluctuation of the current abnormality coefficient data is, the risk thereof may not be high even if there is a large risk. For->Negative correlation mapping is performed using exp (-x) functions.
And the abnormal coefficient value of the j-th skeleton key point at the current moment is obtained. />The larger the value of (c) is, the faster the change speed corresponding to the jth skeleton key point of the current ith monitoring target is, and the greater the risk is possibly.
In order to prevent excessive fluctuation of data caused by small amount of data at first, when 30 frames are accumulated, namely, the length of the anomaly coefficient sequence is greater than 30, anomaly value calculation corresponding to each time point of the ith monitoring target is performed. Wherein 30 is a second super parameter that can be adjusted by the user according to the specific implementation scenario. An abnormal value can be obtained for the ith monitoring target at each corresponding time point after 30 frames, i.e. the length of the abnormal coefficient sequence is greater than 30.
When judging whether a person needs to be rescued, whether the person needs to get rid of the trouble by himself after having a larger abnormal value when abnormal behaviors occur to the person is judged whether the current monitoring target needs to be rescued. The method and the device are used for judging through the first coefficient, wherein the first coefficient represents whether the current monitoring target has abnormal behaviors, namely whether the current monitoring target has severe movement of limbs, and is used for judging whether the current monitoring target has normal behaviors. Step S4 is a method for calculating a first coefficient, specifically:
s4: and calculating a first coefficient, wherein the first coefficient is an absolute value of a difference value between an abnormal value at the latest moment and a mean value of abnormal values at a plurality of moments before the latest moment.
The first coefficient can represent whether the current monitoring target has abnormal behaviors, namely whether the current monitoring target has severe movement of limbs, and is used for judging whether the current monitoring target has normal behaviors.
S5: and responding to the first coefficient being larger than a preset first coefficient threshold value, obtaining a first abnormal value sequence and a second abnormal value sequence, calculating similarity, and generating a similarity distance value.
The first abnormal sequence value is an abnormal sequence value of the monitoring target before the target time, the second abnormal sequence value is an abnormal sequence value of the monitoring target after the target time, and the target time is a time when the first coefficient is larger than the first coefficient threshold value.
When the first coefficient is larger than the first coefficient threshold, the current ith monitoring target is considered to have abnormal behaviors at the moment, namely the severe movement of limbs occurs, and the current monitoring target is abnormal in behaviors.
And taking the moment when the first coefficient is larger than the first coefficient threshold value as a limit, and comparing the difference degrees between the front moment and the rear moment to finish whether rescue is needed or not. Wherein the coefficient threshold may be adjusted by the user according to the specific implementation scenario, first coefficient threshold 26 of the present application.
The similarity calculation is carried out on the first abnormal sequence and the second abnormal sequence by using a DTW (Dynamic Time Warping ) algorithm to obtain a similarity distance value, which is recorded as，/>The larger the current ith monitoring target, the larger the behavior of the ith monitoring target is changed, the larger the data difference is, the higher the possibility that the monitoring target generates the overdriving behavior is, and the higher the possibility that rescue is needed. DTW is utilized because the lengths of the first abnormal sequence and the second abnormal sequence are inconsistent.
When the monitored target limb movement speed is continuously high, the corresponding risk is higher, and the corresponding abnormal coefficient mean value is lower after the target moment. And acquiring the overall abnormal change peak value of the current monitoring target, and after the peak value, if the monitoring target continues to struggle vigorously, needing alarm rescue.
When calculating the abnormal value sequence after the target time, a certain error is caused if the data is too small, so the method and the device set the sequence length of the second abnormal sequence to be more than 10, then calculate the similarity, wherein 10 is a third super parameter, and the user can adjust according to the specific implementation scene.
S6: and calculating a second coefficient according to the similarity distance value.
Taking the rescue coefficient of the ith monitoring target at the latest moment after the moment when the first coefficient is larger than the first coefficient threshold value as a second coefficient, wherein the calculation formula of the second coefficient is as follows:
wherein,second coefficient of the ith monitoring target for the target moment,/>The larger the value of (c) is, the more rescue is needed for the ith monitoring target at the latest moment.
For the sequence of outliers of the ith monitoring target before the target moment,/th monitoring target>The larger the value of (c) is, the larger the behavior of the ith monitoring target is changed before and after the moment that the first coefficient is larger than the preset first coefficient threshold value, and the larger the possibility of needing rescue is.
For the mean value of the sequence of outliers of the ith monitored target from the target moment to the latest moment, if +.>The larger the data fluctuation representing the abnormality coefficient is, the larger the behavior abnormality is, and the monitoring target is required to keep the more drastic limb movement behavior continuously because the monitoring target may need rescue at this time.
S7: an alarm signal is generated in response to the second coefficient being greater than a second coefficient threshold.
Setting a second coefficient threshold, and when the second coefficient is larger than the second coefficient threshold, indicating that the current monitoring target needs rescue, and sending an alarm signal. The second coefficient threshold is set to 10, and the second coefficient threshold can be set in a self-defined mode according to user requirements.
S8: and responding to the alarm signal, and generating coordinate information of the monitoring target and a rescue route. Referring to fig. 4, step S8 includes steps S80-S81, specifically as follows:
s80: and responding to the alarm signal, acquiring all position coordinates of the skeleton key points of the monitoring target in the video image, and calculating the average value of all position coordinates to obtain the average value coordinates of the skeleton key points.
When the current monitoring target needs rescue, according to all position coordinates of skeleton key points corresponding to the monitoring target in the shooting equipment in the monitoring image, the average value among all coordinates of all skeleton key points is calculated, and the skeleton key point average value coordinates are obtained.
S81: and acquiring actual geographic coordinates of the monitoring target.
And according to the calibration relation between the camera coordinate system and the real coordinate system, acquiring the real coordinate azimuth corresponding to the mean value coordinates of the skeleton key points corresponding to the monitoring target in the real world, and further transmitting the coordinate azimuth to the unmanned rescue boat.
S82: and generating a rescue route through a path planning algorithm according to the geographical coordinates of the rescue boat and the geographical coordinates of the monitoring target.
And carrying out rescue path planning of the unmanned rescue boat by using the existing path planning method, for example, an A-shaped path planning algorithm, so as to finish unmanned rescue.
The implementation principle of the unmanned rescue method based on human behavior recognition is as follows: the method comprises the steps of collecting a current beach water surface monitoring image, obtaining a monitoring target skeleton key point according to the current monitoring image, obtaining a current single skeleton key point abnormality coefficient according to the monitoring target skeleton key point, completing monitoring target behavior abnormality identification according to the single skeleton key point abnormality coefficient, and realizing unmanned rescue according to a monitoring target behavior abnormality identification result.
The embodiment of the application also discloses an unmanned rescue system based on human behavior recognition, referring to fig. 5, comprising a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the unmanned rescue method based on human behavior recognition is realized.
The above system further comprises other components well known to those skilled in the art, such as a communication bus and a communication interface, the arrangement and function of which are known in the art and therefore are not described in detail herein.
In the context of this application, the foregoing memory may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, the computer readable storage medium may be any suitable magnetic or magneto-optical storage medium, such as, for example, resistive random access memory RRAM (ResistiveRandomAccessMemory), dynamic random access memory DRAM (DynamicRandomAccessMemory), static random access memory SRAM (static random access memory), enhanced dynamic random access memory EDRAM (EnhancedDynamicRandomAccessMemory), high-bandwidth memory HBM (High-bandwidth memory), hybrid storage cube HMC (HybridMemoryCube), etc., or any other medium that may be used to store the desired information and that may be accessed by an application, a module, or both. Any such computer storage media may be part of, or accessible by, or connectable to, the device. Any of the applications or modules described herein may be implemented using computer-readable/executable instructions that may be stored or otherwise maintained by such computer-readable media.
While various embodiments of the present application have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. Many modifications, changes, and substitutions will now occur to those skilled in the art without departing from the spirit and spirit of the application. It should be understood that various alternatives to the embodiments of the present application described herein may be employed in practicing the application.
The foregoing are all preferred embodiments of the present application, and are not intended to limit the scope of the present application in any way, therefore: all equivalent changes in structure, shape and principle of this application should be covered in the protection scope of this application.
Claims (3)
1. The unmanned rescue method based on human behavior recognition is characterized by comprising the following steps of:
acquiring skeleton key points of a monitoring target in an acquired video image according to a pre-training model of the preset human body gesture recognition;
calculating an abnormal coefficient of a single skeleton key point to generate an abnormal coefficient sequence, wherein the abnormal coefficient is the speed variation of the single skeleton key point;
calculating an abnormal value at each moment to generate an abnormal value sequence at each moment;
calculating a first coefficient, wherein the first coefficient is an absolute value of a difference value between an abnormal value at the latest moment and a mean value of abnormal values at a plurality of moments before the latest moment;
responding to the fact that the first coefficient is larger than a preset first coefficient threshold value, obtaining a first abnormal value sequence and a second abnormal value sequence, calculating similarity, and generating a similarity distance value, wherein the first abnormal value is an abnormal value of a monitoring target before a target moment, the second abnormal value is an abnormal value of the monitoring target after the target moment, and the target moment is the moment when the first coefficient is larger than the first coefficient threshold value;
calculating a second coefficient according to the similarity distance value;
generating an alarm signal in response to the second coefficient being greater than a second coefficient threshold;
according to a pre-training model for recognizing the preset human body gesture, acquiring skeleton key points of a monitoring target in an acquired video image, wherein the method comprises the following steps:
acquiring video images acquired by each camera device;
according to the preset skeleton key point number value, based on a pre-training model of human body gesture recognition, detecting skeleton key points of the video image, and generating a coordinate sequence of a monitoring target in the video image;
calculating the abnormal coefficient of the single skeleton key point to generate an abnormal coefficient sequence, comprising the following steps:
calculating the Euclidean distance and time interval of coordinates of the same skeleton key point in the coordinate sequence between two continuous frames of the video image;
according to the Euclidean distance and the time interval, calculating an anomaly coefficient, wherein the calculation formula is as follows:
wherein,anomaly coefficient of jth skeleton key point of ith monitored object between the a-th frame and the a+1-th frame of video image, +.>For the first superparameter,/->For monitoring the depth value of the jth skeleton key point of the target in the a-th frame of the video image,/>The depth value of the j-th skeleton key point in the a+1-th frame of the video image is used as the i-th monitoring target,for the ith monitored object, between the a-th frame and the a+1-th frame of the video image, relative to the cameraThe distance value of the device is set to,for the value of Euclidean distance between the first coordinate of the jth skeleton key point and the second coordinate of the (a+1) th frame,/for the jth skeleton key point>Is a time interval;
in calculating the outlier at each moment, the calculation formula is:
wherein,abnormal value at the latest time of the ith monitoring target,/->Generating a variance value in an abnormal coefficient sequence for the jth skeleton key point of the current ith monitoring target at the latest moment; />The abnormal coefficient value of the jth skeleton key point at the current moment is obtained, and n is the total number of the skeleton key points;
the calculation formula of the second coefficient is:
wherein,second coefficient of the ith monitoring target for the target moment,/>For the sequence of outliers of the ith monitoring target before the target moment,/th monitoring target>The average value of the abnormal value sequence from the target time to the latest time of the ith monitoring target is obtained.
2. The unmanned rescue method based on human behavior recognition according to claim 1, further comprising the steps of:
and responding to the alarm signal, and generating coordinate information of a monitoring target and a rescue route.
3. The unmanned rescue method based on human behavior recognition according to claim 2, wherein in response to the alarm signal, transmitting the coordinate information of the monitoring target and the rescue route, comprises the steps of:
responding to the alarm signal, acquiring all position coordinates of skeleton key points of the monitoring target in the video image, and calculating the average value of all position coordinates to obtain the average value coordinates of the skeleton key points;
acquiring actual geographic coordinates of a monitoring target;
and generating a rescue route through a path planning algorithm according to the geographical coordinates of the rescue boat and the geographical coordinates of the monitoring target.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311292989.3A CN117011946B (en) | 2023-10-08 | 2023-10-08 | Unmanned rescue method based on human behavior recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311292989.3A CN117011946B (en) | 2023-10-08 | 2023-10-08 | Unmanned rescue method based on human behavior recognition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117011946A CN117011946A (en) | 2023-11-07 |
| CN117011946B true CN117011946B (en) | 2023-12-19 |
Family
ID=88573103
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311292989.3A Active CN117011946B (en) | 2023-10-08 | 2023-10-08 | Unmanned rescue method based on human behavior recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117011946B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107145861A (en) * | 2017-05-05 | 2017-09-08 | 中国科学院上海高等研究院 | A kind of abandon automatic testing method |
| CN109460702A (en) * | 2018-09-14 | 2019-03-12 | 华南理工大学 | Passenger's abnormal behaviour recognition methods based on human skeleton sequence |
| CN109522793A (en) * | 2018-10-10 | 2019-03-26 | 华南理工大学 | More people's unusual checkings and recognition methods based on machine vision |
| CN110147743A (en) * | 2019-05-08 | 2019-08-20 | 中国石油大学(华东) | Real-time online pedestrian analysis and number system and method under a kind of complex scene |
| CN111011974A (en) * | 2019-12-27 | 2020-04-17 | 北京京成兴华医疗科技有限公司 | Anti-falling shoes, falling early warning method and device and computer readable storage medium |
| WO2021179773A1 (en) * | 2020-03-07 | 2021-09-16 | 华为技术有限公司 | Image processing method and device |
| CN113537005A (en) * | 2021-07-02 | 2021-10-22 | 福州大学 | On-line examination student behavior analysis method based on attitude estimation |
| CN114120438A (en) * | 2022-01-11 | 2022-03-01 | 北京国电电力新能源技术有限公司 | Human motion posture detection method and device |
| WO2022067606A1 (en) * | 2020-09-30 | 2022-04-07 | 中国科学院深圳先进技术研究院 | Method and system for detecting abnormal behavior of pedestrian, and terminal and storage medium |
| CN114359974A (en) * | 2022-03-08 | 2022-04-15 | 广东履安实业有限公司 | Human body posture detection method and device and storage medium |
| CN114663972A (en) * | 2021-11-05 | 2022-06-24 | 范书琪 | Target marking method and device based on motion difference |
| CN116311497A (en) * | 2023-02-02 | 2023-06-23 | 山东大学 | Tunnel worker abnormal behavior detection method and system based on machine vision |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103458772B (en) * | 2011-04-07 | 2017-10-31 | 香港中文大学 | Retinal images analysis method and device |
| US20220051437A1 (en) * | 2020-08-17 | 2022-02-17 | Northeastern University | 3D Human Pose Estimation System |
-
2023
- 2023-10-08 CN CN202311292989.3A patent/CN117011946B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107145861A (en) * | 2017-05-05 | 2017-09-08 | 中国科学院上海高等研究院 | A kind of abandon automatic testing method |
| CN109460702A (en) * | 2018-09-14 | 2019-03-12 | 华南理工大学 | Passenger's abnormal behaviour recognition methods based on human skeleton sequence |
| CN109522793A (en) * | 2018-10-10 | 2019-03-26 | 华南理工大学 | More people's unusual checkings and recognition methods based on machine vision |
| CN110147743A (en) * | 2019-05-08 | 2019-08-20 | 中国石油大学(华东) | Real-time online pedestrian analysis and number system and method under a kind of complex scene |
| CN111011974A (en) * | 2019-12-27 | 2020-04-17 | 北京京成兴华医疗科技有限公司 | Anti-falling shoes, falling early warning method and device and computer readable storage medium |
| WO2021179773A1 (en) * | 2020-03-07 | 2021-09-16 | 华为技术有限公司 | Image processing method and device |
| WO2022067606A1 (en) * | 2020-09-30 | 2022-04-07 | 中国科学院深圳先进技术研究院 | Method and system for detecting abnormal behavior of pedestrian, and terminal and storage medium |
| CN113537005A (en) * | 2021-07-02 | 2021-10-22 | 福州大学 | On-line examination student behavior analysis method based on attitude estimation |
| CN114663972A (en) * | 2021-11-05 | 2022-06-24 | 范书琪 | Target marking method and device based on motion difference |
| CN114120438A (en) * | 2022-01-11 | 2022-03-01 | 北京国电电力新能源技术有限公司 | Human motion posture detection method and device |
| CN114359974A (en) * | 2022-03-08 | 2022-04-15 | 广东履安实业有限公司 | Human body posture detection method and device and storage medium |
| CN116311497A (en) * | 2023-02-02 | 2023-06-23 | 山东大学 | Tunnel worker abnormal behavior detection method and system based on machine vision |
Non-Patent Citations (2)
| Title |
|---|
| Frame-level refinement networks for skeleton-based gait recognition;Likai Wang等;《Computer Vision and Image Understanding》;第1-8页 * |
| 基于深度学习的3D人体姿态估计研究;张婧;《中国优秀硕士学位论文全文数据库信息科技辑》(第01期);第I138-1961页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117011946A (en) | 2023-11-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108955718A (en) | A kind of visual odometry and its localization method, robot and storage medium | |
| CN108168539A (en) | A kind of blind man navigation method based on computer vision, apparatus and system | |
| WO2021169924A1 (en) | Behavior prediction method and apparatus, gait recognition method and apparatus, electronic device, and computer readable storage medium | |
| CN110555975A (en) | Drowning prevention monitoring method and system | |
| CN114022910B (en) | Swimming pool drowning prevention supervision method and device, computer equipment and storage medium | |
| CN112541424A (en) | Real-time detection method for pedestrian falling under complex environment | |
| Wang et al. | Real-time 3D human tracking for mobile robots with multisensors | |
| CN116259002A (en) | Human body dangerous behavior analysis method based on video | |
| CN114529583B (en) | Power equipment tracking method and tracking system based on residual regression network | |
| US9990857B2 (en) | Method and system for visual pedometry | |
| CN115100744A (en) | Badminton game human body posture estimation and ball path tracking method | |
| CN117011946B (en) | Unmanned rescue method based on human behavior recognition | |
| CN114170686A (en) | Elbow bending behavior detection method based on human body key points | |
| WO2021248564A1 (en) | Panoramic big data application monitoring and control system | |
| CN116152928A (en) | Drowning prevention early warning method and system based on lightweight human body posture estimation model | |
| CN114639168B (en) | Method and system for recognizing running gesture | |
| Uddin et al. | A thermal camera-based activity recognition using discriminant skeleton features and rnn | |
| CN113095321B (en) | Roller bearing temperature measurement and fault early warning method and device for belt conveyor | |
| CN114120444A (en) | 3D convolution neural network unsafe behavior detection system based on human skeleton characteristics | |
| CN114202587A (en) | Visual feature extraction method based on shipborne monocular camera | |
| CN113536950A (en) | Personnel electric shock detection method and system based on artificial intelligence | |
| JP2022019339A (en) | Information processing apparatus, information processing method, and program | |
| CN111827740A (en) | Swimming pool monitoring system and method based on binocular vision | |
| Nakashima et al. | Self-organization of a sound source localization robot by perceptual cycle | |
| CN112257568B (en) | Intelligent real-time supervision and error correction system and method for individual soldier queue actions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |