CN114863310A - Hand hygiene automatic monitoring method and system based on artificial intelligence - Google Patents
Hand hygiene automatic monitoring method and system based on artificial intelligence Download PDFInfo
- Publication number
- CN114863310A CN114863310A CN202210274653.3A CN202210274653A CN114863310A CN 114863310 A CN114863310 A CN 114863310A CN 202210274653 A CN202210274653 A CN 202210274653A CN 114863310 A CN114863310 A CN 114863310A
- Authority
- CN
- China
- Prior art keywords
- hand
- hand washing
- washing
- posture
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/34—Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/107—Static hand or arm
- G06V40/11—Hand-related biometrics; Hand pose recognition
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an artificial intelligence based automatic hand hygiene monitoring method and system, wherein the method comprises the following steps: acquiring video data of the posture of the hand washer in the left direction, the upper direction and the right direction, wherein the shooting frame rate is M frames/second; recognizing a plurality of bone nodes of each hand, extracting three-dimensional coordinates of the bone nodes of every N frames of videos as input of a sub-discrimination space model, and realizing judgment of hand washing postures and hand washing steps for M/N times per second, wherein M is more than or equal to N; intercepting a video segment of each hand washing step from the video data by adopting a smoothing algorithm; using a repeated action calculation module to calculate the hand washing and rubbing times in the video segment; and (4) calculating the score of a certain hand washing step in a combined manner according to the hand washing and rubbing times and the average probability value of the hand washing step discrimination result of the corresponding video segment to monitor the hand hygiene of the hand washer. The invention also discloses an automatic hand hygiene monitoring system based on artificial intelligence. The invention realizes hand hygiene monitoring and greatly reduces calculation force.
Description
Technical Field
The invention relates to the technical field of medical health, in particular to an artificial intelligence-based automatic hand hygiene scoring method and system.
Background
Hand hygiene is the most effective measure for preventing and controlling hospital infection, particularly ICU hospital infection is higher than that in a common ward, the infection link is complex, and the probability that the hands of medical care personnel including nursing workers contact critical patients in the process of diagnosis and treatment is the highest. However, the hand hygiene compliance rate of medical staff is not ideal at present, and is only 33.17%. Hospital infection, 1/3, can be effectively controlled by strict hand hygiene, with hospital staff touching the patient or not washing their hands after contamination, with a 100% out-of-limit bacterial count. How to judge whether the doctor performs strict hand hygiene is a test faced in the hospital prevention and control hospital for the occurrence of infection events.
The existing technologies are mainly divided into two types of technologies, one is based on wearable devices, and the other is based on videos.
The method based on the wearable device generally requires a hand cleaner to wear the wearable device for hand washing, data acquisition is performed through a gyroscope and an accelerator in an armband, then the data is analyzed by using a machine learning method, and finally hand washing gesture recognition and classification are achieved. Firstly, the method causes great inconvenience to the hand-washing of the hand-washing person, and firstly, the hand-washing person needs to roll up the sleeves to the arms and wear the arm straps to wash hands, so that the hand-washing burden of the hand-washing person is greatly increased, and the hand-washing person cannot use the hand-washing method in winter and is difficult to apply on a large scale.
The current hand washing mode based on video directly inputs the video and uses a deep learning method to identify the hand washing gesture. The method is difficult to realize real-time monitoring of the hand washing gesture, and the recognition accuracy rate is low. The method has the biggest problems that real application cannot be realized, the hand washing recognition effect of non-training set participants is poor, and the accuracy rate is lower than 60%. The equipment required by the method has larger computational power, and the video is directly used as a judgment basis, so that the data volume generated by the video stream is too large, and the system operation needs to depend on a large computational power machine.
Furthermore, in a real medical environment, a hand-washing person often cannot perform hand-washing according to a seven-step hand-washing method in a standard manner in a timely and quantitative manner due to the fact that the hand-washing posture of the hand-washing person is not standard, the time is short, and the like. Hospital admissions and hospital clinics can only check whether a doctor strictly washes hands in this way in a spot check mode, and automatically scoring the doctor's hand washing actions with AI becomes particularly critical in preventing and controlling nosocomial infections. The existing accurate and real-time hand washing gesture recognition in a real medical scene has high dependence on the environment, the hand washing environment is replaced, the model is often difficult to work, and effective hand washing scores cannot be formed.
Disclosure of Invention
The invention provides an artificial intelligence-based automatic hand hygiene monitoring method and system, which are used for realizing hand hygiene monitoring, avoiding the input of the whole video serving as a classifier and greatly reducing the input parameter quantity.
The invention adopts the following technical scheme:
in one aspect, the invention relates to an artificial intelligence-based automatic hand hygiene monitoring method, which comprises the following steps:
acquiring video data of the posture of a hand washer in the left direction, the upper direction and the right direction of a hand washing position, wherein the video frame rate is M frames per second;
identifying a plurality of key bone nodes of each hand from the video data, and obtaining three-dimensional coordinates of each key bone node;
extracting three-dimensional coordinates of the key bone nodes of every N frames of videos, and using the three-dimensional coordinates as input of a sub-discrimination space model to judge the hand washing posture of a hand washer in a hand washing method for M/N times per second and then judge the hand washing step to which the hand washing posture belongs, wherein M is more than or equal to N;
extracting a judgment result sequence of the continuous and same hand washing steps in each hand washing step from the video data by adopting a smoothing algorithm, wherein the judgment result of each hand washing posture represents the videos of N frames, so that a video segment corresponding to each hand washing step is captured;
using a repeated action calculation module to calculate the hand washing and rubbing times in the video segment of each hand washing step;
and jointly calculating the score of the hand-washing step of the hand-washer according to the two index scores of the hand-washing rubbing times and the average probability value of the discrimination result of the corresponding video-band hand-washing step, thereby monitoring the hand hygiene execution of the hand-washing step of the hand-washer.
In another aspect, the present invention is an artificial intelligence based automatic hand hygiene monitoring system comprising:
the video acquisition module is used for acquiring video data of the posture of the hand washer in the left direction, the upper direction and the right direction of the hand washing position, and the video frame rate is M frames per second;
the bone node extraction module is used for identifying a plurality of key bone nodes of each hand from the video data to obtain the three-dimensional coordinates of each key bone node;
the hand washing posture judging module is used for extracting three-dimensional coordinates of the key bone nodes of every N frames of videos and inputting the three-dimensional coordinates as a sub-judgment space model, so that the judgment of the hand washing posture of a hand washer in a hand washing method for M/N times per second is realized, and then the hand washing step to which the hand washing posture belongs is judged, wherein M is more than or equal to N;
the hand washing posture confidence degree judging module is used for extracting a judging result sequence of the continuously same hand washing steps in each hand washing step from the video data by adopting a smoothing algorithm, and the judging result of each hand washing posture represents N frames of videos, so that a video segment corresponding to each hand washing step is intercepted, and the average probability value of the judging result of the hand washing step corresponding to the video segment is calculated;
the kneading frequency calculating module is used for calculating the hand washing kneading frequency in the video segment of each hand washing step by using the repeated action calculating module;
and the scoring module is used for combining the two index scores of the rubbing times and the average probability value of the hand washing step discrimination result corresponding to the video segment to calculate the score of a hand washer in a certain hand washing step, so that the hand hygiene execution of the hand washer is monitored.
Advantageous effects
The method and the system respectively extract the hand skeleton of the hand washer from three directions by using a deep learning skeleton extraction technology, and extract a plurality of key skeleton point coordinate information from each hand, thereby avoiding the whole video as the input of a classifier and greatly reducing the input parameter quantity. Meanwhile, the strong analysis capability of the invention can accurately judge the hand washing postures of doctors and give scores to each posture, thereby having great significance for promoting hospital sense prevention and control.
The method and the system have strong popularization and can greatly improve the hand washing quality of hospitals and communities. Particularly for hospitals, the system can count whether each doctor washes hands according to a seven-step hand washing method, the hand washing quality of each step and can feed back the monitoring condition of each hand washing in real time. Through the Internet, a doctor end can be reminded and fed back in time; the hospital end can accurately monitor the hand washing condition of each doctor and overall management.
Drawings
FIG. 1 is a schematic diagram of the extraction of key bone points by the bone extraction technique provided by the embodiments of the present invention;
FIG. 2 is a schematic flow chart of an artificial intelligence-based automatic hand hygiene monitoring method provided in embodiment 1 of the present invention;
FIG. 3 is a schematic flow chart of an automatic monitoring method for hand hygiene based on artificial intelligence according to embodiment 2 of the present invention;
fig. 4 is a schematic structural diagram of an automatic hand hygiene monitoring system based on artificial intelligence provided in embodiment 3 of the present invention.
Detailed Description
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.
The world health organization recommends a six-step hand washing method, and hospital departments generally add a hand washing wrist link as a hand hygiene hand washing standard, and the hand washing method is called a seven-step hand washing method. The seven-step washing method is a hand washing method before operation by medical staff, and the method requires the medical staff to wash hands by using 7 postures. The invention relates to an automatic hand hygiene monitoring method based on artificial intelligence, which is mainly used for judging the posture of a hand washer according to 7 postures of a seven-step washing method to judge whether the hand washing posture is correct or not.
Fig. 2 is a schematic flow chart of an automatic hand hygiene monitoring method based on artificial intelligence according to embodiment 1 of the present invention. The method comprises the following steps:
step S101: and acquiring video data of the posture of the hand washer in the left, upper and right directions of the hand washing position, wherein the video frame rate is M frames per second.
In the step, the cameras are arranged in the left direction, the upper direction and the right direction of the hand basin, and video data of hand washing postures of the hand washers are provided.
Step S102: and identifying a plurality of key bone nodes of each hand from the video data, and obtaining the three-dimensional coordinates of each key bone node.
In the step, the skeleton points of the hands in each frame of the video are extracted by using an artificial intelligent hand skeleton extraction technology, wherein the left-hand skeleton point is extracted by the left-hand camera, the right-hand skeleton point is extracted by the right-hand camera, and the left-hand skeleton point and the right-hand skeleton point are extracted by the upper camera, so that the modeling of each hand is realized.
Step S103: creating a filtering algorithm which filters out actions of posture conversion and/or flushing non-seven-step washing manipulation through the space between the key bone nodes, wherein the filtering algorithm is
Wherein S represents the square of the distance between two skeletal points, n represents the number of key skeletal nodes of each finger, X k X-axis coordinate, Y, representing the Kth bone node k Y-axis coordinate, Z, representing the Kth bone node k Representing the Z-axis coordinate of the Kth bone node, the equation calculates the square of the distance from the first of the hand bone points to the nth point.
Preferably, the filtering threshold S of the non-hand washing person is set to be 0.391-0.693.
In this step, hands that are brought into the mirror due to other reasons such as passing by may affect the hand recognition of the real hand washer in the video data, and thus non-hand washing personnel need to be filtered. The distance between the non-hand-washing person and the camera is farther than that between the hand-washing person and the camera, so that the hands of the non-hand-washing person are smaller than those of the camera. The algorithm filters hands with close distances by calculating the distance between the identified hand skeleton points, i.e., the hands of non-hand washing people can be filtered.
Step S104: and extracting three-dimensional coordinates of the key bone nodes of every N frames of videos to be used as input of a sub-discrimination space model, so that judgment of hand washing postures of the hand washer in the hand washing method is realized for M/N times per second, and hand washing steps to which the hand washing postures belong are judged, wherein M is larger than or equal to N.
In the step, the key bone nodes of the hands in the video are extracted, so that the influence on recognition caused by different scenes for shooting the video is avoided, and the function of noise reduction is achieved.
The hand washing posture is judged whether the currently recognized hand washing posture is the hand washing posture in the hand washing method in the hand washing posture training set. The hand washing step is a hand washing step for determining which of the hand washing techniques is the hand washing posture identified as described above.
Step S105: and extracting a judgment result sequence of the continuous same hand washing steps in each hand washing step from the video data by adopting a smoothing algorithm, wherein each hand washing posture judgment result represents the videos of N frames, so that a video segment corresponding to each hand washing step is captured.
In this step, specifically, in the step of extracting, by using a smoothing algorithm, a sequence of consecutive identical determination results corresponding to each hand washing gesture from the video data with respect to the determination result of the hand washing gesture, when a difference occurs once in the consecutive sequence, the difference is ignored.
Step S106: and calculating the frequency of hand washing and rubbing in the video segment of each hand washing step by using a repeated action calculation module.
Step S107: and jointly calculating the score of the hand washer in a certain hand washing step according to the two index scores of the hand washing rubbing times and the average probability value of the hand washing step discrimination result of the corresponding video segment, thereby monitoring the hand hygiene execution of the hand washer.
In this step, the sum of the number of hand washing and rubbing times and the average probability value of the discrimination result of the hand washing step corresponding to the video segment is used to obtain the score of the final hand washer in a certain hand washing step.
In summary, the embodiment of the invention uses the deep learning skeleton extraction technology to extract the skeleton of the hand washer from three directions respectively, and extracts a plurality of key skeleton point coordinate information from each hand, thereby avoiding the whole video as the input of a classifier, greatly reducing the input parameter amount, reducing the video flow calculation power, ensuring that the hand washing gesture recognition model is independent of the environment, and ensuring that the recognition rate is as high as 99%. The method can count whether each doctor completes hand washing according to a seven-step hand washing method or not and the hand washing quality of each step, and can feed back the monitoring condition of each hand washing in real time. Through the Internet, a doctor end can be reminded and fed back in time; the hospital end can accurately monitor the hand washing condition of each doctor and overall management.
To identify precisely, 7 postures in the seven-step washing manipulation can be finely divided into 19 postures, or more.
Seven steps of hand washing method: washing palms, washing finger slits on the back side of the hands, washing finger slits on the side of the palms, washing finger backs, washing thumbs, washing finger tips and washing wrists. In the seven-step washing method, the two hands are required to be alternately used for the 5 steps of the second step, the fourth step, the fifth step, the sixth step and the seventh step, so that in the seven-step washing method, 12 different steps are actually required to be distinguished, and the steps are respectively: washing the palm, washing the back finger slit of the left hand, washing the back finger slit of the right hand, washing the finger slit of the palm side, washing the back of the left hand, washing the back of the right hand, washing the thumb of the left hand, washing the thumb of the right hand, washing the fingertip of the left hand, washing the tip of the right finger, washing the arm of the left wrist and washing the arm of the right wrist. However, in any hand washing step, the doctor does not usually wash hands in the same way, so even if the hand-washing person uses different accepted hand-washing postures or different hand-placing angles, the model should be able to identify which hand-washing posture the way belongs to. Because the hand cleaners have a large number of hand washing postures and cannot exhaust all hand washing postures, 12 hand washing steps have 1 to 2 different hand washing postures according to different hand washing modes or different hand placing angles, so that the hand washing device has 19 different hand washing postures in total, which requires that a discrimination model can have accurate classification capability on the 19 postures, and the details of the 12 hand washing steps including the 19 hand washing postures are shown in the following table 1:
fig. 3 is a schematic flow chart of a specific method for automatically monitoring hand hygiene based on artificial intelligence according to embodiment 2 of the present invention. The method comprises the following steps:
step S201: cameras are arranged in the left direction, the upper direction and the right direction of a hand washing position to acquire video data of postures of the hand washer, and the video frame rate is 30 frames per second.
Step S202: the position of the hand is identified from the video data using a skeletal extraction technique to extract each frame of image. The skeleton of only one hand is extracted when the skeleton extraction device is deployed on the left camera and the right camera, and the skeleton extraction device is deployed on the upper camera and the bones of two hands, so that 4 hands can be extracted from each frame of video.
Step S203: after the position of the hand is located, 21 key bone nodes of each hand are identified by using a bone extraction technology, and three-dimensional coordinates (x, y, z) of each key bone node are obtained to realize modeling of the hand.
Step S204: creating a filtering algorithm, wherein the filtering algorithm filters out actions of posture conversion and/or flushing non-seven-step hand washing methods through the intervals among the key bone nodes; wherein the filtering algorithm is
Where S represents the square of the distance between two skeletal points, and n represents the number of key skeletal nodes 20 in each finger, i.e., n is 20, X k X-axis coordinate, Y, representing the Kth bone node k Y-axis coordinate, Z, representing the Kth bone node k Representing the Z-axis coordinate of the Kth bone node, the equation calculates the square of the distance from the first of the hand bone points to the 20 th point.
When a hand washer washes hands, the hands of the non-hand washer are easy to be caught by the cameras arranged at the left side and the right side of the hand washing pool when the non-hand washer passes by the hand washing pool, so that the hand washing posture of the hand washer is influenced. To avoid the effect of this situation, a filtering algorithm was created to filter the hands of the non-handwash users, setting the filtering threshold S of the non-handwash users to 0.5.
Step S205: and extracting three-dimensional coordinates of the key bone nodes of every 10 frames of videos, using the three-dimensional coordinates as input of a sub-discrimination space model, judging the hand washing posture of a hand washer in a hand washing method for 3 times per second, and judging the hand washing step to which the hand washing posture belongs.
In this step, each 10 frames of video are used as input, and each input has 10 frames of video, 4, 21 key bone nodes, 3 times, 2520 variables.
It can also be understood that: the hand washing video of the hand washer needing to be scored is input into the trained hand washing posture distinguishing model by taking every 10 frames as an input, and the model judges that the hand washing steps are performed 3 times per second because the video is 30 frames per second.
In the washing steps of the seven-step washing method, for the purpose of precise classification, the washing method can be further subdivided into 19 postures or more according to different washing modes or different hand placing angles. In the embodiment, the preferred 19 hand washing postures are shown in table 1, which is a training set of hand washing posture discrimination models. 2520 variables generated every 10 frames of video are used as input to determine which of the 19 hand washing postures the hand washer has washed hand in the 10 frames of video. After the model is trained, the accuracy of the model is verified in a 10-fold cross validation mode, and the accuracy of the model for distinguishing 19 hand-washing postures can reach 97.5%. And then, converting the 19 types of judgment into 12 hand washing steps, namely, because the first hand washing mode and the second hand washing mode belong to the first step, when the model is judged to be the first hand washing mode or the second hand washing mode, the accuracy of the model can reach 99 percent if the model is judged to be the same.
Step S206: and extracting a judgment result sequence of the continuous same hand washing steps in each hand washing step from the video data by adopting a smoothing algorithm, wherein the judgment result of each hand washing posture represents a 10-frame video, so that a video segment corresponding to each hand washing step is captured. For example, model output: 1, 3, 4, 5, 1, 2, 4, 1, 1, 1, 1, 1, 3, 1, 1, 4, 1, 1, 1, 1, 5, 6, 7, 4, wherein the model outputs a result of 1, representing the first of the 12 hand washing steps; extracting continuous identical judgment results through a smoothing algorithm, when the sequences are different once, neglecting, and extracting the hand washing video segment in the first step is as follows: 1,1,1,1,1,1,3,1,1,4,1,1,1,1.
Step S207: and calculating the frequency of hand washing and rubbing in the video segment of each hand washing step by using a repeated action calculation module.
Step S208: and jointly calculating the score of the hand washer in a certain hand washing step according to the sum of the hand washing and rubbing times and the average probability value of the hand washing step discrimination result of the corresponding video segment, thereby monitoring the hand hygiene execution of the hand washer.
Specifically, the index is given a full score of 50 when the number of hand washing and rubbing times exceeds 6 (hospital feeling requires that each posture cannot be less than 3 seconds, and the number of hand washing and rubbing times cannot be less than 6 on average two times per second), and a score of 50 x (times/6) when the number of hand washing and rubbing times does not exceed 6.
The average of the average probability values output by the model, for example, for the hand wash video segment of the first step intercepted is: 1, 1, 1, 1, 1, 1, 3, 1, 1, 4, 1, 1, 1, the probability value when the model output is 1 in this section is accumulated and then averaged. The index is scored as 50 x the average probability value.
The hand washing step is scored as the sum of the scores of the two indexes.
In summary, in the embodiment, the deep learning skeleton extraction technology is used to extract the hand skeleton of the hand washer from three directions respectively, and a plurality of pieces of key skeleton point coordinate information are extracted from each hand, so that the whole video is prevented from being used as the input of a classifier, the input parameter amount is greatly reduced, the video flow calculation force is reduced, and the hand hygiene of the hand washer is monitored through two index evaluation scores, namely the rubbing times and the average value of the average probability value output by each judgment model in the whole video corresponding to each hand washing posture. The method can count whether each doctor completes hand washing according to a seven-step hand washing method or not and the hand washing quality of each step, and can feed back the monitoring condition of each hand washing in real time. Through the Internet, a doctor end can be reminded and fed back in time; the hospital end can accurately monitor the hand washing condition of each doctor and overall management.
Fig. 4 is a schematic structural diagram of an automatic hand hygiene monitoring system based on artificial intelligence according to embodiment 3 of the present invention. The system comprises a video acquisition module 301, a bone node extraction module 302, a hand washing posture judgment module 303, a hand washing posture confidence judgment module 304, a kneading frequency calculation module 305 and a grading module 306.
And the video acquisition module is used for acquiring video data of the posture of the hand washer, and the video shooting rate is M frames per second.
And the bone node extraction module is used for identifying a plurality of key bone nodes of each hand from the video data to obtain the three-dimensional coordinates of each key bone node.
And the hand washing posture judgment module is used for extracting the three-dimensional coordinates of the key bone nodes of every N frames of videos and taking the three-dimensional coordinates as the input of the sub-judgment space model to realize the judgment of the hand washing posture of the hand washer in the hand washing method for M/N times per second, wherein M is more than or equal to N.
And the hand washing posture confidence degree judging module is used for extracting a continuous same judging result sequence corresponding to each hand washing posture from the video data by adopting a smoothing algorithm according to the judging result of the hand washing posture, wherein each judging result represents N frames of videos, so that the whole section of video corresponding to each hand washing posture is intercepted, and the average value of the average probability value output by each judging model in the whole section of video is calculated.
And the rubbing frequency calculating module is used for calculating the frequency of hand washing and rubbing in the whole video of each hand washing posture by using the repeated action calculating module.
And the scoring module is used for combining the rubbing times and the average value of the average probability value output by each judgment model in the whole video corresponding to each hand washing gesture to calculate the final score of the hand washer for monitoring the hand hygiene of the hand washer according to the score of a certain hand washing step.
In one possible embodiment, the system further comprises a filtering module 307 for creating a filtering algorithm that filters out actions in gesture conversion and/or flushing non-seven-step hand washing via the spacing between the key bone nodes; wherein the filtering algorithm is
Wherein S represents the square of the distance between two skeletal points, n represents the number of key skeletal nodes of each finger, and X k X-axis coordinate, Y, representing the Kth bone node k Y-axis coordinate, Z, representing the Kth bone node k Representing the Z-axis coordinate of the Kth bone node, the formula calculates the sum of the squares of the distances from the first to the nth of the hand bone points. In order to avoid the problem that when a hand washer washes hands, the hands of a non-hand washer are easy to be caught by the cameras arranged at the left side and the right side of the hand washing pool when the non-hand washer passes by the hand washing pool, so that the hand washing posture of the hand washer is influenced, a filtering module is created for filtering the hands of the non-hand washer.
In one possible embodiment, the system further comprises a threshold setting module 308 for filtering non-handwashers, wherein the set threshold S is 0.391-0.693.
In a possible embodiment, the system further comprises cameras disposed in the left, upper and right directions of the hand basin and used for connecting the video acquisition module. The accurate monitoring of the posture of the hand washer in three-dimensional and multi-dimensional is realized.
It should be noted that: in the above embodiment, when performing the monitoring and evaluation, the automatic manual hygiene monitoring system based on artificial intelligence is only illustrated by the division of the above functional modules, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the server serving as the automatic manual hygiene monitoring system is divided into different functional modules to complete all or part of the above described functions. In addition, the automatic hand hygiene monitoring system provided by the above embodiment and the automatic hand hygiene monitoring method embodiment belong to the same concept, and the specific implementation process thereof is described in detail in the method embodiment and is not described herein again.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention.
Claims (9)
1. An artificial intelligence based automatic hand hygiene monitoring method, comprising:
acquiring video data of the posture of a hand washer in the left, upper and right directions of a hand washing position, wherein the video frame rate is M frames per second;
identifying a plurality of key bone nodes of each hand from the video data, and obtaining three-dimensional coordinates of each key bone node;
extracting three-dimensional coordinates of the key bone nodes of every N frames of videos, and using the three-dimensional coordinates as input of a sub-discrimination space model to judge the hand washing posture of a hand washer in a hand washing method for M/N times per second and then judge the hand washing step to which the hand washing posture belongs, wherein M is more than or equal to N;
extracting a judgment result sequence of the continuous and same hand washing steps in each hand washing step from the video data by adopting a smoothing algorithm, wherein the judgment result of each hand washing posture represents the videos of N frames, so that a video segment corresponding to each hand washing step is captured;
using a repeated action calculation module to calculate the hand washing and rubbing times in the video segment of each hand washing step;
and jointly calculating the score of the hand-washing step of the hand-washer according to the two index scores of the hand-washing rubbing times and the average probability value of the discrimination result of the corresponding video-band hand-washing step, thereby monitoring the hand hygiene execution of the hand-washing step of the hand-washer.
2. The method as claimed in claim 1, wherein before the step of extracting three-dimensional coordinates of the key bone nodes of every N frames of video as input of a sub-discriminant space model, the method further comprises the steps of:
creating a filtering algorithm, wherein the filtering algorithm filters out actions of posture conversion and/or flushing non-seven-step hand washing methods through the intervals among the key bone nodes; wherein the filtering algorithm is
Wherein S represents the square of the distance between two skeletal points, n represents the number of key skeletal nodes of each finger, and X k X-axis coordinate, Y, representing the Kth bone node k Y-axis coordinate, Z, representing the Kth bone node k Representing the Z-axis coordinate of the Kth bone node, the equation calculates the square of the distance from the first of the hand bone points to the nth point.
3. The method of claim 2, wherein the non-handwash user' S filtering threshold S is set to 0.391 to 0.693.
4. The method of claim 1, wherein in the step of extracting a sequence of consecutive identical determinations for each hand washing gesture from the video data using a smoothing algorithm for the determination of the hand washing gesture, the sequence of consecutive determinations is ignored when a difference occurs once in the sequence.
5. The method of claim 1, wherein M is 30, N is 10, and the number of key bone nodes is 21, located on the five fingers, palm and wrist, respectively.
6. Hand health automatic monitoring system based on artificial intelligence, its characterized in that includes:
the video acquisition module is used for acquiring video data of the posture of the hand washer in the left direction, the upper direction and the right direction of the hand washing position, and the video frame rate is M frames per second;
the bone node extraction module is used for identifying a plurality of key bone nodes of each hand from the video data to obtain the three-dimensional coordinates of each key bone node;
the hand washing posture judging module is used for extracting three-dimensional coordinates of the key bone nodes of every N frames of videos and inputting the three-dimensional coordinates as a sub-judgment space model, so that the judgment of the hand washing posture of a hand washer in a hand washing method for M/N times per second is realized, and then the hand washing step to which the hand washing posture belongs is judged, wherein M is more than or equal to N;
the hand washing posture confidence degree judging module is used for extracting a judging result sequence of the continuously same hand washing steps in each hand washing step from the video data by adopting a smoothing algorithm, and the judging result of each hand washing posture represents N frames of videos, so that a video segment corresponding to each hand washing step is intercepted, and the average probability value of the judging result of the hand washing step corresponding to the video segment is calculated;
the kneading frequency calculating module is used for calculating the hand washing kneading frequency in the video segment of each hand washing step by using the repeated action calculating module;
and the scoring module is used for calculating the score of a hand washer in a certain hand washing step by combining the two index scores of the rubbing times and the average probability value of the hand washing step discrimination result of the corresponding video segment, so that the hand hygiene execution of the hand washer is monitored.
7. The system of claim 6, further comprising a filtering module to create a filtering algorithm that filters out actions that are in gesture conversion and/or flushing non-seven-step hand washing via spacing between the key bone nodes; wherein the filtering algorithm is
Wherein S represents the square of the distance between two skeletal points, n represents the number of key skeletal nodes of each finger, and X k X-axis coordinate, Y, representing the Kth bone node k Y-axis coordinate, Z, representing the Kth bone node k Representing the Z-axis coordinate of the Kth bone node, the equation calculates the square of the distance from the first of the hand bone points to the nth point.
8. The system of claim 7, further comprising a threshold setting module for filtering non-handwashers, wherein the threshold S is set to 0.391-0.693.
9. The system of claim 6, further comprising cameras disposed in left, upper and right directions of the wash basin for connecting the video acquisition module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210274653.3A CN114863310A (en) | 2022-03-18 | 2022-03-18 | Hand hygiene automatic monitoring method and system based on artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210274653.3A CN114863310A (en) | 2022-03-18 | 2022-03-18 | Hand hygiene automatic monitoring method and system based on artificial intelligence |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114863310A true CN114863310A (en) | 2022-08-05 |
Family
ID=82627786
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210274653.3A Pending CN114863310A (en) | 2022-03-18 | 2022-03-18 | Hand hygiene automatic monitoring method and system based on artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114863310A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116664819A (en) * | 2023-05-17 | 2023-08-29 | 武汉大学中南医院 | Medical staff hand recognition positioning method, device, equipment and storage medium |
-
2022
- 2022-03-18 CN CN202210274653.3A patent/CN114863310A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116664819A (en) * | 2023-05-17 | 2023-08-29 | 武汉大学中南医院 | Medical staff hand recognition positioning method, device, equipment and storage medium |
| CN116664819B (en) * | 2023-05-17 | 2024-01-09 | 武汉大学中南医院 | Medical staff hand recognition positioning method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020177498A1 (en) | Non-intrusive human body thermal comfort detection method and system based on posture estimation | |
| WO2021143353A1 (en) | Gesture information processing method and apparatus, electronic device, and storage medium | |
| Liu et al. | Towards unified surgical skill assessment | |
| Sittig et al. | Physiologic trend detection and artifact rejection: a parallel implementation of a multi-state Kalman filtering algorithm | |
| Chaaraoui et al. | Abnormal gait detection with RGB-D devices using joint motion history features | |
| CN106726030A (en) | Brain machine interface system and its application based on Clinical EEG Signals control machinery hands movement | |
| Xie et al. | Feature enrichment based convolutional neural network for heartbeat classification from electrocardiogram | |
| CN111582158A (en) | Tumbling detection method based on human body posture estimation | |
| CN109887595A (en) | Heartbeat anomalous identification algorithm based on depth learning technology | |
| Fieraru et al. | Learning complex 3D human self-contact | |
| CN107132915A (en) | A kind of brain-machine interface method based on dynamic brain function network connection | |
| Das et al. | Eldo-care: Eeg with kinect sensor based telehealthcare for the disabled and the elderly | |
| CN116269355B (en) | Safety monitoring system based on figure gesture recognition | |
| CN114863310A (en) | Hand hygiene automatic monitoring method and system based on artificial intelligence | |
| CN114358194A (en) | Gesture tracking based detection method for abnormal limb behaviors of autism spectrum disorder | |
| CN111401435B (en) | Human body motion mode identification method based on motion bracelet | |
| CN109126045A (en) | intelligent motion analysis and training system | |
| Likitlersuang et al. | Interaction detection in egocentric video: Toward a novel outcome measure for upper extremity function | |
| CN113974612B (en) | Automatic evaluation method and system for upper limb movement function of stroke patient | |
| Yang et al. | Automatic detection pipeline for accessing the motor severity of Parkinson’s disease in finger tapping and postural stability | |
| Li et al. | Non-invasive thermal sensation recognition based on human behavior postures in office environment | |
| CN116189311B (en) | Protective clothing wears standardized flow monitoring system | |
| Li et al. | Study on action recognition based on kinect and its application in rehabilitation training | |
| CN111312363A (en) | Double-hand coordination enhancing system based on virtual reality | |
| CN115067934A (en) | Hand motion function analysis system based on machine intelligence |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |