CN111241874A - Behavior monitoring method and device and computer readable storage medium - Google Patents
Behavior monitoring method and device and computer readable storage medium Download PDFInfo
- Publication number
- CN111241874A CN111241874A CN201811436518.4A CN201811436518A CN111241874A CN 111241874 A CN111241874 A CN 111241874A CN 201811436518 A CN201811436518 A CN 201811436518A CN 111241874 A CN111241874 A CN 111241874A
- Authority
- CN
- China
- Prior art keywords
- determining
- target object
- behavior
- target
- key point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 238000012544 monitoring process Methods 0.000 title claims abstract description 60
- 238000012806 monitoring device Methods 0.000 claims abstract description 15
- 230000006399 behavior Effects 0.000 claims description 174
- 230000015654 memory Effects 0.000 claims description 29
- 238000005192 partition Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 11
- 230000003542 behavioural effect Effects 0.000 claims description 5
- 238000000638 solvent extraction Methods 0.000 claims description 5
- 206010000117 Abnormal behaviour Diseases 0.000 description 28
- 238000005516 engineering process Methods 0.000 description 20
- 210000001508 eye Anatomy 0.000 description 10
- 230000009471 action Effects 0.000 description 9
- 210000000707 wrist Anatomy 0.000 description 9
- 210000003414 extremity Anatomy 0.000 description 8
- 230000001360 synchronised effect Effects 0.000 description 8
- 238000001514 detection method Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 210000003127 knee Anatomy 0.000 description 6
- 238000012417 linear regression Methods 0.000 description 6
- 230000002159 abnormal effect Effects 0.000 description 5
- 210000003423 ankle Anatomy 0.000 description 5
- 230000001815 facial effect Effects 0.000 description 4
- 210000002414 leg Anatomy 0.000 description 4
- 210000001331 nose Anatomy 0.000 description 4
- 230000000474 nursing effect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000014509 gene expression Effects 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 230000036544 posture Effects 0.000 description 3
- 208000020016 psychiatric disease Diseases 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 210000004247 hand Anatomy 0.000 description 2
- 210000003128 head Anatomy 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 230000000977 initiatory effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 230000002860 competitive effect Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 210000004709 eyebrow Anatomy 0.000 description 1
- 230000008921 facial expression Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 210000000214 mouth Anatomy 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 230000003238 somatosensory effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Alarm Systems (AREA)
Abstract
The embodiment of the application discloses a behavior monitoring method, which comprises the following steps: determining target key points of a target object in the acquired real-time video stream; determining behavior information of the target object according to the target key point; determining the behavior state of the target object according to the behavior information; the embodiment of the application also discloses a behavior monitoring device and a computer readable storage medium.
Description
Technical Field
The embodiment of the application relates to the technical field of computer vision, and relates to but is not limited to a behavior monitoring method, a behavior monitoring device and a computer-readable storage medium.
Background
Techniques for monitoring user behavior in the related art include: the Kinect sensor technology comprises carrier monitoring, contour detection, Gaussian background modeling and somatosensory.
When monitoring user behaviors, the carrier monitoring technology requires a user to wear a carrier, is inconvenient to use and may bring danger to special people; when the contour detection technology monitors user behaviors, the situation that fitting coordinates are not hit completely and tracking is lost exists; when the Gaussian background modeling technology monitors the user behavior, behavior image acquisition needs to be carried out on the user in advance; when monitoring user behaviors, the Kinect sensor technology needs to record behavior videos in advance on one hand and needs to purchase a Kinect sensor on the other hand.
Disclosure of Invention
The embodiment of the application provides a behavior monitoring method and device and a computer readable storage medium.
The technical scheme of the embodiment of the application is realized as follows:
the application provides a method of behavior monitoring, the method comprising: determining target key points of a target object in the acquired real-time video stream; determining behavior information of the target object according to the target key point; and determining the behavior state of the target object according to the behavior information.
In the above solution, the determining a target key point of a target object in a captured real-time video stream includes: identifying a target object and a key point of the target object in the video stream; and determining the target key points in the key points according to an application scene through a deep network model.
In the above solution, the determining a target key point of a target object in a captured real-time video stream includes: identifying a target object and a key point of the target object in the video stream; partitioning the key points of the target object to obtain at least one partition, wherein the partition comprises at least one key point; and selecting the key point of one of the partitions as the target key point.
In the foregoing scheme, the determining the behavior information of the target object according to the target key point includes: determining coordinate information of the target key point; and determining the behavior information of the target object by utilizing a deep network model according to the coordinate information.
In the above aspect, the method further includes: acquiring sample data of different behavior states of an object in different application scenes; determining the behavior information of the object according to the behavior state of the sample data; determining a target key point of the object according to the behavior information; determining the corresponding relation among an application scene, a behavior state, behavior information and a target key point; and establishing the deep network model according to the corresponding relation.
In the above aspect, the method further includes: setting an application scene; correspondingly, the determining the behavior state of the target object according to the behavior information includes: and determining the behavior state of the target object according to the behavior information and the application scene.
In the above aspect, the method further includes: acquiring a user identifier corresponding to the behavior state; and outputting the behavior state of the target object by taking the user identification as a target address.
The present application further provides a device for behavioral monitoring, the device comprising: the device comprises a first determining module, a second determining module and a third determining module; the first determining module is used for determining target key points of a target object in the acquired real-time video stream; the second determining module is used for determining the behavior information of the target object according to the target key point; and the third determining module is used for determining the behavior state of the target object according to the behavior information.
The present application further provides an apparatus for behavioral monitoring, comprising a processor and a memory for storing a computer program capable of running on the processor; wherein the processor is configured to execute the steps of the behavior monitoring method in the above-mentioned scheme applied to the terminal device when running the computer program.
The present application also provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method for behavior monitoring as described in the above-mentioned solution, applied to a terminal device.
The behavior monitoring method, the behavior monitoring device and the computer-readable storage medium provided by the embodiment of the application determine the target key points of the target object in the acquired real-time video stream; determining behavior information of the target object according to the target key point; determining the behavior state of the target object according to the behavior information; therefore, the behavior of the user can be identified by determining and monitoring the key points of the human body by utilizing the collected real-time video stream, and the user experience is improved.
Drawings
Fig. 1 is a first schematic flow chart of an implementation of a behavior monitoring method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of key points of a human body according to an embodiment of the present application;
fig. 3 is a schematic flow chart illustrating an implementation process of the behavior monitoring method according to the embodiment of the present application;
FIG. 4 is a schematic diagram of single person key point monitoring according to an embodiment of the present application;
FIG. 5 is a schematic diagram of multi-user key point monitoring according to an embodiment of the present application;
fig. 6 is a first schematic structural diagram of a behavior monitoring device according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a behavior monitoring device according to an embodiment of the present application.
Detailed Description
The present application will be described in further detail with reference to the following drawings and specific embodiments.
Fig. 1 is a schematic flow chart of an implementation of a behavior monitoring method in an embodiment of the present application, and as shown in fig. 1, the method includes the following steps:
step 101: determining target key points of a target object in the acquired real-time video stream;
wherein the target key points characterize different parts of the target object, such as: neck, left wrist, etc. And determining a target object and a target key point of the target object in the acquired real-time video stream.
In practical application, a camera can be used for collecting video streams, the collected video streams are processed in real time, a target object in the video streams is determined, the target object is positioned, and a target key point of the target object is determined.
When determining the target object, determining the contour information of the objects in the video stream, and comparing the contour information of each object with preset contour information; and determining the object with the contour information conforming to the preset contour information as a target object.
In an embodiment, the determining target keypoints of a target object in the captured real-time video stream comprises: identifying a target object and a key point of the target object in the video stream; and determining the target key points in the key points according to an application scene through a deep network model.
Acquiring an image by using a camera, processing the acquired real-time video stream, and acquiring all key points of a target object after determining the target object; and selecting target key points corresponding to the application scene from all key points of the target object through the depth network model. Wherein, the application scene can include: the method comprises the following steps that according to application scenes such as an old people monitoring scene and a patient monitoring scene, a corresponding target key point is selected from key points by a depth network model according to different application scenes.
Such as: when the target object in the video stream is an old person, the key points are points corresponding to 18 parts, such as: keypoint 1, keypoint 2 … keypoint 18, wherein different keypoints characterize different sites. When the application scene is the monitoring of the old people, the target key points in the monitoring scene of the old people are determined to be 6 key points according to the corresponding relation between the monitoring scene of the old people and the target key points in the depth network model: keypoint 1, keypoint 3, keypoint 7, keypoint 11, keypoint 14, keypoint 16.
In an embodiment, the determining target keypoints of a target object in the captured real-time video stream comprises: identifying a target object and a key point of the target object in the video stream; partitioning the key points of the target object to obtain at least one partition, wherein the partition comprises at least one key point; and selecting the key point of one of the partitions as the target key point.
Acquiring an image by using a camera, processing the acquired real-time video stream, and acquiring all key points of a target object after determining the target object; and partitioning all key points of the target object to obtain a plurality of partitions, wherein each partition comprises a plurality of key points, and selecting the plurality of key points in one partition of the partitions as target key points.
Such as: the key points of the target object determined by the camera include points corresponding to 18 parts, such as: nose, neck, right shoulder, right elbow, right wrist, left shoulder, left elbow, left wrist, right hip, right knee, right ankle, left hip, left knee, left eye, right eye, left ear, and right ear. All the key points are partitioned to form two partitions: a face keypoint zone and a leg keypoint zone. The keypoints of the facial keypoint zone may include: the corresponding points of the nose, the left eye, the right eye, the left ear and the right ear; the keypoints of the leg keypoint zone may include: the corresponding points of the parts of the right hip, the right knee, the right ankle, the left hip, the left knee and the left eye. When the facial behavior of the target object needs to be determined, key points of the facial key point partition can be selected as target key points; when the leg behavior of the target object needs to be determined, the key points of the leg key point partition can be selected as the target key points.
Step 102: determining behavior information of the target object according to the target key point;
the behavior information may include: slope, angle, etc. characterize the behavior state of the target object.
After selecting the target key points from the key points of the target object, determining the behavior information of the target object by using a deep network model according to the information of the target key points.
In an embodiment, the determining behavior information of the target object according to the target key point includes: determining coordinate information of the target key point; and determining the behavior information of the target object by utilizing a deep network model according to the coordinate information.
Determining coordinate information of the target key point by using the acquired real-time video stream according to the target key point; and inputting the coordinate information of the target key points into a depth network model, substituting the coordinate information of the target key points into a fitting equation of the linear regression model by the depth network model, and solving to determine the behavior information of the target object.
The fitting equation is shown in equation (1):
wherein, yi、xiAs coordinate information of the target key point, eiFor the error of the determined coordinate information of the target keypoint from the actual coordinate information,
are regression coefficients.
Here, the coordinates of the target key points of the target object are substituted into x of the fitting equation described abovei、yiAccording to the least square method
Carry out the solution of
As behavior information of the target object.
Step 103: and determining the behavior state of the target object according to the behavior information.
Determining the behavior state of the target object according to the acquired behavior information, wherein the behavior state may include: user actions such as tilting, falling, standing, etc.
Such as: the acquired behavior information of the target object is a slope, and if an angle corresponding to the slope is equal to 45 degrees, the behavior state of the target object can be determined to be an inclination.
In an embodiment, the method further comprises: acquiring sample data of different behavior states of an object in different application scenes; determining the behavior information of the object according to the behavior state of the sample data; determining a target key point of the object according to the behavior information; determining the corresponding relation among an application scene, a behavior state, behavior information and a target key point; and establishing the deep network model according to the corresponding relation.
Before determining the target key point of the target object in the acquired video stream, the method further comprises the following steps: modeling is based on the behavior of the object. The behavior information may include: information representing the behavior state of the target object, such as slope, angle and the like; the behavioral states may include: user actions such as tilting, falling, standing, etc.
In practical application, sample data of different behavior states of an object in different application scenes is obtained, the behavior states of the sample data are analyzed, behavior information of the object is determined, a linear regression model is used according to the behavior information, target key points of the object are determined, accordingly, the corresponding relation among the application scenes, the behavior states, the behavior information and the target key points can be obtained, and a depth network model is determined. Wherein, the application scene can include: monitoring scenes such as an old people monitoring scene and a patient monitoring scene.
Such as: the method comprises the steps of obtaining sample data of the body inclination state of an old person in an old person monitoring scene, analyzing the body inclination state of the old person, determining the slope of the body of the old person in the body inclination state, namely behavior information, building a fitting equation by using a linear regression model, substituting the slope into the fitting equation to solve, obtaining coordinates of target key points, building corresponding relations among an application scene, the behavior state, the behavior information and the target key points, and building a depth network model according to the corresponding relations.
In an embodiment, the method further comprises: setting an application scene; correspondingly, the determining the behavior state of the target object according to the behavior information includes: and determining the behavior state of the target object according to the behavior information and the application scene.
And setting an application scene, and determining the behavior state of the target object according to the behavior information and the application scene when determining the behavior state of the target object according to the behavior information.
Such as: the set application scene is a patient monitoring scene, the acquired behavior information of the patient is a slope, and an angle corresponding to the slope is smaller than 45 degrees, and under the patient monitoring scene, when the angle corresponding to the slope is smaller than 45 degrees, the behavior state of the patient is represented as falling.
In an embodiment, the method further comprises: acquiring a user identifier corresponding to the behavior state; and outputting the behavior state of the target object by taking the user identification as a target address.
Wherein, the user identification can include: cell phone numbers, micro signal codes, etc. Presetting a user identifier, and after determining the behavior state of the target object, acquiring the user identifier corresponding to the behavior state of the target object, such as: when the behavior state of the target object is inclined, the corresponding user identification is the user identification of family members; and when the behavior state of the target object is falling down, the corresponding user identification is a rescue number.
And sending the behavior state of the target object to a user identifier related to the target object by taking the user identifier as a target address, or initiating a call between the user identifier and the target object according to the obtained user identifier.
Such as: and when the behavior state of the target object is determined to be a tumble, initiating a call between the user identifier and the target object according to the user identifier, or sending the behavior state of the target object to the user identifier related to the target object by taking the user identifier as a target address.
The behavior state of the target object can be sent to the user identification, and meanwhile, a suggestion for processing the behavior state can be given.
In the embodiment of the application, the target key points of the target object in the collected real-time video stream are determined; determining behavior information of the target object according to the target key point; determining the behavior state of the target object according to the behavior information; therefore, the behavior of the user can be identified by determining and monitoring the key points of the human body by utilizing the collected real-time video stream, and the behavior of a certain part of the target object can be carefully monitored; when the abnormal behavior of the user is monitored, early warning can be performed according to the abnormal behavior, suggestions are given, and user experience is further improved.
In this embodiment, a behavior monitoring method provided in the embodiment of the present application is described through a scenario by taking a user as an example. The main key points in this embodiment correspond to the target key points in the above embodiments.
In the aspect of monitoring abnormal behaviors of patients, the related technologies mainly comprise vehicle monitoring, contour detection, Gaussian background modeling, Kinect sensor technology and the like.
The carrier monitoring technology needs to wear related products such as an upper body carrier, a finger carrier and a hand single-node carrier in limb recognition to monitor abnormal behaviors, real-time recognition cannot be achieved directly through a camera, use experience is reduced by wearing the carrier, and meanwhile danger can be brought to certain special crowds.
When abnormal behavior monitoring is carried out through a contour detection technology, contour detection needs to be carried out on an acquired monitoring image, and the change of the number of pixels in a contour area of an old man is calculated by using a frame difference method to judge the state of the old man. Moreover, in the monitoring process, all possible fitting coordinates are missed, and the tracking is lost.
When abnormal behavior monitoring is performed through a Gaussian background modeling technology, a section of video needs to be extracted based on a motion sequence technology and a Radio Frequency Identification (RFID) tag technology, Gaussian background modeling is performed by using the extracted video, then contour and matrix characteristic information of a target in a foreground is extracted, the characteristic information of the foreground is compared with a human abnormal action characteristic library, and whether abnormal behavior of a human exists in the foreground is judged. The RFID tag technology is also used for wearing RFID tags for all the old people needing to be monitored, and a method based on template matching needs to collect behavior images of each monitored person in advance and extract features to form a feature template.
Skeleton information is obtained through the Kinect sensor technology, the angle rotation movement of the skeleton pair forms matrix information, and the matrix information is compared with recorded dangerous action information to judge the difference. On one hand, the technology needs to record dangerous action information in advance, is not very flexible in mechanism, and needs to purchase a Kinect sensor to increase hardware cost; on the other hand, only 20 key points in the limb can be tracked by using the Kinect, the head is only represented by one point, and the use is limited in a complex background. In the practical application process, the method has a plurality of inconveniences, and the use experience is reduced.
The related art has a lot of problems in the abnormal behavior monitoring process of the patient applied to the health medical field, can not simply depend on the intelligent camera to monitor the effective limbs and behaviors of the patient, and can early warn and inform the family and medical care personnel at the first time when danger occurs. The technical disadvantages of the related art are as follows:
1) some prior art rely on wearing relevant products such as upper part of the body carrier, finger carrier, hand single node carrier, could carry out the key point and detect, and then discern the action, and the hardware cost is great, and the scheme limitation is very big in the actual scene. Meanwhile, it is considered that in the remote care process, the patient is not suitable to wear the relevant carrier due to the patient himself, and the relevant carrier may cause some damage to the patient himself.
2) Some methods need to record abnormal behavior information in advance as reference, and cannot achieve real-time identification. The abnormal behaviors of a plurality of patients cannot be detected in real time, so that serious consequences are likely to be caused, and the rescue is not timely.
According to the embodiment of the application, the algorithm capable of identifying the body behaviors is applied to the camera monitoring device without the need of wearing a carrier by a patient and recording dangerous actions in advance as reference, the abnormal behaviors of the patient can be automatically monitored and identified by monitoring key points of the human body of the patient, and an alarm is timely sent out and the patient is in contact with rescue when danger is generated due to the abnormal behaviors. In addition, the embodiment of the application adopts a novel human body detection algorithm, and 130 human body key points including basic key points of 2 multiplied by 21 hands can be detected in real time; the system comprises 70 basic face key points, is presented in a 3D color matchmaker mode, integrates multiple functions of multiple computer vision projects such as real-time multi-person 2D posture estimation, dynamic 3D reconstruction and hand key point detection, can well correspond to a complex background, and is more competitive compared with the traditional monitoring application. As shown in fig. 2, the human key points are characterized in the form of 3D color matchers.
The embodiment of the application adopts a novel body language recognition technology to monitor the abnormal behavior of the patient, and can make up the defect that the traditional image processing technology only recognizes partial postures of the patient such as lying, sitting and standing. In the unusual behavior monitoring of traditional patient, hardly catch patient's facial expression, in novel monitoring is used, key points such as eyes, mouth all are described, and limbs language and expression homoenergetic are discerned simultaneously. The application can be applied to nursing and nursing of the old, auxiliary rehabilitation, auxiliary treatment of mental diseases and auxiliary treatment and nursing of other mental diseases and the like.
The video source transmitted in the camera is read in real time, abnormal behaviors of the patient can be judged after identification of key points of limbs and judgment of abnormal behavior parameters in a specific scene, intelligent voice communication is carried out on the abnormal behaviors of the patient in time, when the abnormal behaviors of the patient do not have signs of elimination in a short time, a warning can be sent out immediately, and families and medical staff of the patient are informed in time in a telephone mode, a short message mode and the like; and through limb and facial key point identification, user psychology can be guessed in combination with the situation at that time, communication of people in reality is more approached, and a more personalized scheme is provided.
A schematic flow diagram of an embodiment of the present application is shown in fig. 3, and includes:
step 301: acquiring a real-time video stream;
and acquiring a real-time video stream by adopting the existing high-definition network camera.
Step 302: determining main key points;
and carrying out human body positioning on the obtained video stream by using a depth network model, identifying key points, and selecting main key points from the key points. Such as: the identified key points are 18 points in total, and are respectively: 0 nose, 1 neck, 2 right shoulder, 3 right elbow, 4 right wrist, 5 left shoulder, 6 left elbow, 7 left wrist, 8 right hip, 9 right knee, 10 right ankle, 11 left hip, 12 left knee,13 left eye, 14 left eye, 15 right eye, 16 left ear and 17 right ear, and each key point coordinate is ai(xi，yi) And selecting main key points from the key points. If the selected main key points are as follows: 1 neck, 4 right wrist, 7 left wrist, 8 right hip, 10 right ankle and 11 left hip, which can be represented as ai(xi，yi) (i is 1, 4, 7, 8, 10, 11), where neither the x-coordinate nor the y-coordinate is 0.
Step 303: detecting main key points;
detecting the main key points to obtain coordinate information of the main key points;
step 304: fitting the main key points to obtain a slope;
performing linear regression on the main key points according to a linear regression model, and obtaining the slope of a regression line by using a least square method;
the linear regression model is shown as formula (1), and the formula (1) is solved by using a least square method to obtain a regression coefficient
Is calculated as in equation (2),
is calculated as in formula (3), wherein
To calculate the obtained slope.
Wherein, yi、xiAs coordinate information of the target key point, eiFor the error of the determined coordinate information of the target keypoint from the actual coordinate information,
are regression coefficients.
Step 305: judging whether the behavior is abnormal according to the determined slope;
judging whether the behavior of the detected person is abnormal or not according to the slope; if the behavior is abnormal, go to step 306; if the behavior is normal, returning to step 303;
such as: when slope of the line
If the inclination angle is less than 1, the inclination angle of the detected person is less than 45 degrees, the detected person is considered to have dangerous behaviors and possibly fall over or fall, and the behavior of the detected person is judged to be abnormal.
Step 306: carrying out voice inquiry according to the abnormal behavior;
if the voice message replied by the detected person is received within the set time, returning to step 303;
if the forward feedback voice information of the detected person is received and recognized, timing is started and monitoring is continued, if the abnormal behavior in the continuous time has the eliminated sign and the behavior of the detected person is normal, danger early warning is not carried out at this time, and the system continues to monitor the behavior of the patient.
Further, if the detected person receives and recognizes the forward feedback voice information and the abnormal behavior is not eliminated within the set time, the family members and the medical staff of the patient are immediately informed, and the first time is for rescue after confirmation.
If the voice message replied by the detected person is not received within the set time, go to step 307.
Step 307: and acquiring a preset user identifier, and outputting the abnormal behavior of the target object to a terminal corresponding to the user identifier.
If the forward feedback information of the monitored person is not received through the voice recognition technology, the family members, medical personnel and other related personnel of the monitored person are immediately notified through short messages and telephone modes, and after the family members and the medical personnel check the instant video to confirm danger, rescue measures are immediately taken.
As shown in fig. 4, is a schematic diagram of single-person key point monitoring in the embodiment of the present application; the key points of the human body in a falling state are as follows: 1 neck, 4 right wrist, 7 left wrist, 8 right hip, 10 right ankle and 11 left hip; as shown in fig. 5, a schematic view of multi-person key point monitoring according to an embodiment of the present application is shown, and different key points are determined according to different monitoring requirements of different persons.
The application embodiment has the following use characteristics:
(1) human body key points can be applied to different scenes;
in the unusual behavior monitoring of novel patient was used, behind the camera capture 2D image, the position of key point detector can discern and mark out the body characteristics helps the performance of every posture under the different angles of body tracking algorithm understanding to appear with the form of the colored match people of 3D. The human body key points are applied to different dangerous scene scenes through related rules, and the method is more flexible in practical application.
(2) Key points are tracked more carefully;
the tracking function of the novel patient abnormal behavior monitoring application is much more detailed than the traditional method which can only track 20 key points. With the same action, the traditional method senses that a person is raising hands, and a novel patient abnormal behavior monitoring application based on a new generation of recognition system can observe that the person actually points to something with a finger.
(3) Newly adding face tracking and identifying expressions;
in the aspect of face tracking, the whole head is only one point in the traditional method, but in the application of novel patient abnormal behavior monitoring, eyebrows, eyes, noses, mouths and the like can be depicted by dozens of key points, and body languages and expressions can be identified.
(4) Performing library building and modeling based on abnormal behaviors of the patient;
a database is established based on observed patient behaviors, a model is established aiming at specific behaviors of a certain disease by utilizing a big data modeling technology, and the deep representative significance of the behaviors is identified. The behavior can be directly identified by using the model during early warning, the danger level is evaluated, and the suggested measures are given.
This application embodiment does not need the patient in use to wear carrier, label, gather in advance and be monitored person's unusual action etc. and need not purchase equipment such as Kinect sensor with the help of the hardware performance of camera, acquires and the key point fitting can monitor through the key point. The universal applicability is realized, and the use experience of patients, family medical personnel and the like is improved; the method can be used for real-time monitoring, delay can not be caused when emergency situations occur to abnormal behaviors of patients, and the method is suitable for multi-scene application under a complex background.
On the basis of identifying the coordinates of each key point of the limb of the patient, the embodiment of the application further judges the abnormal behaviors of the patient by pertinently defining the abnormal behaviors in different scenes, such as the old, the mental disease patient and the like; the embodiment of the application applies the novel method for recognizing the human body limb actions in real time of the human body key points to the fields of nursing of old people, auxiliary rehabilitation and the like in the field of health care, and has positive and novel industrial application significance by combining related technologies such as intelligent voice recognition and the like.
The present embodiment provides a behavior monitoring device, as shown in fig. 6, a behavior monitoring device 60 includes: a first determination module 601, a second determination module 602, and a third determination module 603; wherein,
a first determining module 601, configured to determine a target key point of a target object in a collected real-time video stream;
a second determining module 602, configured to determine behavior information of the target object according to the target key point;
a third determining module 603, configured to determine a behavior state of the target object according to the behavior information.
In an embodiment, the first determining module 601 is configured to: identifying a target object and a key point of the target object in the video stream; and determining the target key points in the key points according to an application scene through a deep network model.
In an embodiment, the first determining module 601 is configured to: identifying a target object and a key point of the target object in the video stream; partitioning the key points of the target object to obtain at least one partition, wherein the partition comprises at least one key point; and selecting the key point of one of the partitions as the target key point.
In an embodiment, the second determining module 602 is configured to: determining coordinate information of the target key point; and determining the behavior information of the target object by utilizing a deep network model according to the coordinate information.
In one embodiment, the behavior monitoring device 60 further includes: the modeling module 604 is configured to obtain sample data of different behavior states of the object in different application scenarios; determining the behavior information of the object according to the behavior state of the sample data; determining a target key point of the object according to the behavior information; determining the corresponding relation among an application scene, a behavior state, behavior information and a target key point; and establishing the deep network model according to the corresponding relation.
In one embodiment, the behavior monitoring device 60 further includes: a setting module 605, configured to set an application scenario; accordingly, a third determining module 603 is configured to: and determining the behavior state of the target object according to the behavior information and the application scene.
In one embodiment, the behavior monitoring device 60 further includes: a notification module 606, configured to obtain a user identifier corresponding to the behavior state; and outputting the behavior state of the target object by taking the user identification as a target address.
It should be noted that: in the behavior monitoring device provided in the above embodiment, when performing behavior monitoring, only the division of the program modules is illustrated, and in practical applications, the processing may be distributed to different program modules according to needs, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the behavior monitoring device and the behavior monitoring method provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments in detail and are not described herein again.
Based on the foregoing embodiments, the present application provides a behavior monitoring apparatus, as shown in fig. 7, including a processor 702 and a memory 701 for storing a computer program capable of running on the processor 702; wherein the processor 702 is configured to implement, when running the computer program, the following:
determining target key points of a target object in the acquired real-time video stream;
determining behavior information of the target object according to the target key point;
and determining the behavior state of the target object according to the behavior information.
The method disclosed in the embodiments of the present application may be applied to the processor 702, or implemented by the processor 702. The processor 702 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be implemented by integrated logic circuits of hardware or instructions in the form of software in the processor 702. The processor 702 described above may be a general purpose processor, a DSP, or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. The processor 702 may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed in the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium located in the memory 701, and the processor 702 reads the information in the memory 701 to complete the steps of the foregoing method in combination with the hardware thereof.
It will be appreciated that the memory (memory 701) of embodiments of the present application may be either volatile memory or nonvolatile memory, and may include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read Only Memory (EPROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a magnetic random access Memory (FRAM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical Disc, or a Compact Disc Read Only Memory (CD-ROM); the magnetic surface storage may be disk storage or tape storage. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of illustration and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM, Double Data Synchronous Random Access Memory), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM, Enhanced Synchronous Dynamic Random Access Memory), Synchronous link Dynamic Random Access Memory (SLDRAM, Synchronous Dynamic Random Access Memory), Direct Memory bus (DRmb Access Memory, Random Access Memory). The memories described in the embodiments of the present application are intended to comprise, without being limited to, these and any other suitable types of memory.
Here, it should be noted that: the description of the terminal embodiment is similar to the description of the method, and has the same beneficial effects as the method embodiment, and therefore, the description is omitted. For technical details that are not disclosed in the terminal embodiment of the present application, those skilled in the art should refer to the description of the method embodiment of the present application for understanding, and for the sake of brevity, will not be described again here.
In an exemplary embodiment, the present application further provides a computer-readable storage medium, for example, a memory 701 storing a computer program, which can be processed by the processor 702 to implement the steps of the foregoing method. The computer readable storage medium may be Memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface Memory, optical disk, or CD-ROM.
Embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when processed by a processor, implements:
determining target key points of a target object in the acquired real-time video stream;
determining behavior information of the target object according to the target key point;
and determining the behavior state of the target object according to the behavior information.
Here, it should be noted that: the above description of the computer medium embodiment is similar to the above description of the method, and has the same beneficial effects as the method embodiment, and therefore, the description thereof is omitted. For technical details that are not disclosed in the terminal embodiment of the present application, those skilled in the art should refer to the description of the method embodiment of the present application for understanding, and for the sake of brevity, will not be described again here.
The above description is only a preferred embodiment of the present application, and is not intended to limit the scope of the present application.
Claims (10)
1. A method of behavioral monitoring, the method comprising:
determining target key points of a target object in the acquired real-time video stream;
determining behavior information of the target object according to the target key point;
and determining the behavior state of the target object according to the behavior information.
2. The method of claim 1, wherein determining target keypoints for a target object in the captured real-time video stream comprises:
identifying a target object and a key point of the target object in the video stream;
and determining the target key points in the key points according to an application scene through a deep network model.
3. The method of claim 1, wherein determining target keypoints for a target object in the captured real-time video stream comprises:
identifying a target object and a key point of the target object in the video stream;
partitioning the key points of the target object to obtain at least one partition, wherein the partition comprises at least one key point;
and selecting the key point of one of the partitions as the target key point.
4. The method of claim 1, wherein the determining behavior information of the target object according to the target keypoints comprises:
determining coordinate information of the target key point;
and determining the behavior information of the target object by utilizing a deep network model according to the coordinate information.
5. The method according to claim 2 or 4, characterized in that the method further comprises:
acquiring sample data of different behavior states of an object in different application scenes;
determining the behavior information of the object according to the behavior state of the sample data;
determining a target key point of the object according to the behavior information;
determining the corresponding relation among an application scene, a behavior state, behavior information and a target key point;
and establishing the deep network model according to the corresponding relation.
6. The method of claim 1, further comprising:
setting an application scene;
correspondingly, the determining the behavior state of the target object according to the behavior information includes:
and determining the behavior state of the target object according to the behavior information and the application scene.
7. The method of claim 1, further comprising:
acquiring a user identifier corresponding to the behavior state;
and outputting the behavior state of the target object by taking the user identification as a target address.
8. A performance monitoring device, the device comprising: the device comprises a first determining module, a second determining module and a third determining module; wherein,
the first determining module is used for determining target key points of a target object in the acquired real-time video stream;
the second determining module is used for determining the behavior information of the target object according to the target key point;
and the third determining module is used for determining the behavior state of the target object according to the behavior information.
9. A behavioural monitoring device comprising a processor and a memory for storing a computer program operable on the processor; wherein the processor is adapted to perform the steps of the method of any one of claims 1 to 7 when running the computer program.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811436518.4A CN111241874A (en) | 2018-11-28 | 2018-11-28 | Behavior monitoring method and device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811436518.4A CN111241874A (en) | 2018-11-28 | 2018-11-28 | Behavior monitoring method and device and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111241874A true CN111241874A (en) | 2020-06-05 |
Family
ID=70863614
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811436518.4A Pending CN111241874A (en) | 2018-11-28 | 2018-11-28 | Behavior monitoring method and device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111241874A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107320090A (en) * | 2017-06-28 | 2017-11-07 | 广东数相智能科技有限公司 | A kind of burst disease monitor system and method |
| US20170345181A1 (en) * | 2016-05-27 | 2017-11-30 | Beijing Kuangshi Technology Co., Ltd. | Video monitoring method and video monitoring system |
| CN108830240A (en) * | 2018-06-22 | 2018-11-16 | 广州通达汽车电气股份有限公司 | Fatigue driving state detection method, device, computer equipment and storage medium |
-
2018
- 2018-11-28 CN CN201811436518.4A patent/CN111241874A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170345181A1 (en) * | 2016-05-27 | 2017-11-30 | Beijing Kuangshi Technology Co., Ltd. | Video monitoring method and video monitoring system |
| CN107320090A (en) * | 2017-06-28 | 2017-11-07 | 广东数相智能科技有限公司 | A kind of burst disease monitor system and method |
| CN108830240A (en) * | 2018-06-22 | 2018-11-16 | 广州通达汽车电气股份有限公司 | Fatigue driving state detection method, device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210183516A1 (en) | Systems and methods to identify persons and/or identify and quantify pain, fatigue, mood, and intent with protection of privacy | |
| CN110738192B (en) | Auxiliary evaluation method, device, equipment, system and medium for human body movement function | |
| US10262196B2 (en) | System and method for predicting neurological disorders | |
| US20150320343A1 (en) | Motion information processing apparatus and method | |
| EP4104037A1 (en) | System and method for monitoring clinical activities | |
| CN112784662A (en) | Video-based fall risk evaluation system | |
| US20170243354A1 (en) | Automatic frontal-view gait segmentation for abnormal gait quantification | |
| WO2022022551A1 (en) | Method and device for analyzing video for evaluating movement disorder having privacy protection function | |
| JP7185805B2 (en) | Fall risk assessment system | |
| Withanage et al. | Fall recovery subactivity recognition with RGB-D cameras | |
| CN107077214A (en) | For the method and system of the communication used within the hospital | |
| Beddiar et al. | Fall detection using body geometry and human pose estimation in video sequences | |
| CN113823376B (en) | Intelligent medicine taking reminding method, device, equipment and storage medium | |
| CN112185514A (en) | Rehabilitation training effect evaluation system based on action recognition | |
| Fieraru et al. | Learning complex 3D human self-contact | |
| Zhang et al. | Visual surveillance for human fall detection in healthcare IoT | |
| Lee et al. | Video analytic based health monitoring for driver in moving vehicle by extracting effective heart rate inducing features | |
| Nouisser et al. | Deep learning and kinect skeleton-based approach for fall prediction of elderly physically disabled | |
| CN111563397B (en) | Detection method, detection device, intelligent equipment and computer storage medium | |
| CN116013548B (en) | Intelligent ward monitoring method and device based on computer vision | |
| CN116823591A (en) | Human shape detection and privacy removal method and device based on convolutional neurons | |
| CN111241874A (en) | Behavior monitoring method and device and computer readable storage medium | |
| Xie et al. | Skeleton-based fall events classification with data fusion | |
| An et al. | Support vector machine algorithm for human fall recognition kinect-based skeletal data | |
| Barelle et al. | The BIOTELEKINESY home care service: A holistic concept to prevent fall among the elderly based on ICT and computer vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200605 |
|
| RJ01 | Rejection of invention patent application after publication |