CN115691762A - Autism child safety monitoring system and method based on image recognition - Google Patents
Autism child safety monitoring system and method based on image recognition Download PDFInfo
- Publication number
- CN115691762A CN115691762A CN202211383640.6A CN202211383640A CN115691762A CN 115691762 A CN115691762 A CN 115691762A CN 202211383640 A CN202211383640 A CN 202211383640A CN 115691762 A CN115691762 A CN 115691762A
- Authority
- CN
- China
- Prior art keywords
- behavior
- dangerous
- module
- human body
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention discloses a system and a method for monitoring the safety of autism children based on image recognition, and belongs to the technical field of safety monitoring. The invention provides a short-time simple response and processing, which helps doctors to effectively and timely perform high-level duplication and analysis after an event, promotes the understanding of children with autism, and helps the children with autism to learn and integrate into the society in a comfortable environment.
Description
Technical Field
The invention relates to the technical field of safety monitoring, in particular to a system and a method for monitoring the safety of autism children based on image recognition.
Background
Autism is a developmental disorder, a family becomes the most direct and important support system for autism children in the growth process of the autism children, the treatment, the rehabilitation and the returning to the society of autism are difficult and complicated processes, the care and the intervention training of the autism children need to spend huge energy, at least one side of parents is required to take off production to take care of the autism children, the autism children often have serious social interaction disorder, language disorder, template repetition and other abnormal behaviors, meanwhile, the autism children are possibly damaged behaviors and self-disabled behaviors, for the autism children with the serious damaged behaviors, a guardian is required to carry out close monitoring and tracking before the damaged behaviors of the autism children disappear, the autism children often lack the judgment power of affairs, do not know what consequences can be caused by the behaviors of the self, and do not understand what damages to the self behaviors; self-injuring behaviors such as head collision, hand clasping and self flapping often occur to the children with the autism, the self-injuring behaviors can be frustrated and confused by patients, the self-injuring behaviors can be caused by no affairs, the self-injuring behaviors can be caused by neglecting of guardians for a while, so that guardians not only need to spend great energy to help the children with the autism to carry out social training, but also need to keep vigilance all the time to deal with the abnormal behaviors and stress reactions of the children with the autism, and according to investigation, the personnel monitoring the children with the autism often have psychological diseases caused by anxiety and psychological stress in different degrees.
Disclosure of Invention
The invention aims to provide an autism child safety monitoring system and method based on image recognition, so as to solve the problems in the background technology.
In order to solve the technical problems, the invention provides the following technical scheme: an autism child safety monitoring system based on image recognition comprises a front-end video acquisition module, a video transmission module, a management control module, a behavior detection module, a behavior correction module, a dangerous behavior marking module and a dangerous behavior classifier;
the video monitoring system comprises a front-end video acquisition module, a video transmission module, a management control module, a time generator, a behavior detection module, a behavior correction module, a dangerous behavior classifier and a dangerous behavior analysis module, wherein the front-end video acquisition module acquires an original video signal shot by front-end monitoring equipment, the video transmission module encodes the original video signal and transmits the encoded original video signal through a network service protocol, the management control module controls the front-end monitoring equipment to store and cloud backup video and sound signals, the management control module is additionally provided with the time generator, time is superposed to enter images, the behavior detection module receives the video signal, calls the dangerous behavior classifier to perform behavior detection on characters in the video, labels dangerous behaviors of the characters and outputs labeled information to a terminal and the behavior correction module, the terminal displays a monitoring picture and a behavior labeled result in real time, the behavior correction module gives different correction measures according to the dangerous behaviors of the characters, the dangerous behavior labeling module provides windows for the guardians and medical staff to label dangerous behaviors in the stored video file, the dangerous behavior analysis module inputs the labeled file into the dangerous behavior classification module to help the dangerous behavior classifier to further train, and the dangerous behavior classifier extracts the characteristics of the dangerous behavior and classifies and outputs the dangerous behavior.
The video monitoring system helps a guardian to obtain effective data, image or sound information in real time, timely monitor and memorize the process of sudden abnormal events, provide simple response and processing in a short time, help doctors to effectively and timely review and analyze the events, promote understanding of autistic children and help the autistic children to learn and integrate into the society in a comfortable environment.
The behavior correction module receives that a user is making dangerous behaviors and makes corresponding correction measures according to the grades of the dangerous behaviors;
when the dangerous behavior grade is one, the behavior correction module records the dangerous behavior and the duration of the dangerous behavior of the user, stores the corresponding labeled video, generates a record file and sends the record file to the guardian;
when the dangerous behavior grade is two, the behavior correction module records the dangerous behavior and the time length of the dangerous behavior of the user, stores the corresponding labeled video, generates a recording file, sends the recording file to a guardian, and sends an instruction during the storage period of the dangerous behavior of the user to prompt the user to enter a new action and behavior;
when the dangerous behavior grade is three, the behavior correction module records the dangerous behavior and the time length of the dangerous behavior of the user, stores the corresponding labeled video, generates a recording file, sends the recording file to a guardian, simultaneously sends a warning to the guardian, and sends a stimulating behavior to the user;
when the dangerous behavior grade is four, the behavior correction module records the dangerous behaviors and the time length of the dangerous behaviors of the user, stores the corresponding labeled video, generates a recording file, sends the recording file to the guardian, simultaneously sends out a warning to the guardian, and sends out preset appeasing terms to the user.
The reasons for strange actions or self-injuring behaviors of the autistic children mainly include two aspects, namely that the autistic children can be frustrated or confused, and the autistic children are in a state of boring nothing and are in a state of being unnoticed in a group, so that the autistic children can relieve the mind and body, avoid the conflict on the environment, stop the current anxiety behavior, help the autistic children to control the body of the autistic children, release the relievements from the current anxiety mood, eliminate the fear of the autistic children to unknown disorder, promote more and more active adaptation of the autistic children to be blended into the society, and realize the balanced harmony of the autistic children and the social environment.
For the behaviors with the danger level of one, namely harmless behaviors, proper indifference is paid to the harmless strange behaviors of the autistic children, and the autistic children are given the freedom of controlling the body of the autistic children in the harmless degree;
when the dangerous behavior grade is two, namely, the behavior which is harmless but needs to be corrected, some instructions are sent out to enable the autistic child to get away from the current behavior scene, to learn new behaviors, to enable the autistic child to do something, and to enable the autistic child to not further generate self-injuring behaviors;
when the dangerous behavior grade is three, namely the behavior is possibly harmful and is a precursor behavior of the onset of an over-excited reaction of the autistic child, a new interest point is generated through the stimulating behavior, the autistic child is attracted to be separated from the current behavior and emotion, the fear emotion in the abreaction of the autistic child is helped, the possibility that the current environment is communicated with other people is found, and the autistic child is helped to master the body again;
when the dangerous behavior level is four, the behavior can generate danger or be an autism child overstimulation reaction self-injuring behavior, the autism child is pacified through the pacifying words of the guardian input in advance, the guardian is timely informed, and further pacifying measures are taken for the autism child.
The stimulation behaviors comprise simulating natural sounds, simulating animal cry and playing youthful music, the behavior correction module records the influence of different appeasing phrases and stimulation behaviors on the user, compares the time of dangerous behaviors in the same level, sorts the adopted appeasing phrases or the stimulation behaviors according to the time of the dangerous behaviors, generates a report and sends the report to the guardian of the user.
The stimulating behaviors do not include stimulating prompts such as loud prompt sound, bright light flashing and the like, the autistic children are often attracted by new interest points and stop the current stereotypy,
the autistic children can not accept strong light and noise, can not accept touch, can easily repeat purposeless meaningless behaviors, move according to a certain sequence, like three primary colors, round and arc, and can be pacified and behavior corrected for the autistic children on the basis of respecting the behavior logic of the autistic children.
An autism child safety monitoring method based on image recognition comprises the following specific steps:
the method comprises the following steps: the front-end video acquisition module acquires real-time video data;
step two: extracting video frames in unit time, and converting the video into images;
step three: the behavior detection module detects whether people exist in the video frame or not and segments the people from the picture;
step four: acquiring key points of a human body;
step five: performing logic judgment on the acquired human body key points, and inputting dangerous behavior labels into a behavior correction module when the human body key points conform to dangerous behaviors;
step six: the behavior correction module takes corresponding correction measures according to the danger level corresponding to the dangerous behavior;
the safety monitoring system is used for supplementing the care of a guardian for the autistic child, the autistic child is not intellectual disorder but social disorder, the understanding of social rules and safety rules is lacked, the dangerous behaviors of the autistic child are identified and corrected by simple measures, the alarm and prompt are sent to the guardian of the autistic child, the autistic child is helped to understand the safety rules, the guardian is helped to nurse the autistic child, and the monitoring burden of the guardian is relieved.
Restraining the self-disabled behaviors of the children with the autism, identifying that the self-disabled behaviors exist, sending a warning to a guardian, wherein the self-disabled behaviors comprise self-alarming, head hitting and eye scratching, judging the self-injurious behavior level of the children with the autism at the moment according to the identified behaviors, making different judgments, wherein the judgment comprises sending the warning to the guardian, transferring the interest of the children with the autism by using stimulation, sending some instructions to enable the children to enter new actions and behaviors, recording the whole process, helping the guardian and a treating doctor to analyze and judge the behavior reasons of the children with the autism, reading the reasons behind the behaviors of the children with the autism, providing the children with the autism with a target in a planned way, and continuously adjusting the danger levels of different dangerous behaviors according to actual conditions.
The behavior detection module comprises a human body detection module, a human body key point extraction module and dangerous behavior judgment, the human body detection module detects and extracts characters in the video frame and inputs the characters into the human body key point extraction module, the human body key point extraction module inputs the human body key points into the dangerous behavior judgment module, and when the key points of the human body accord with the dangerous behaviors, the dangerous behaviors are marked and output;
the behavior detection module builds a network based on a YOLO algorithm, a data set marks information of a human body, a head and four limbs, a CSPDarknet53 is used as a backbone network, feature fusion is carried out through PANet, output pictures are identified through YOLOv3, video frame image data are input, the marking information of the human body, the head and the four limbs in a video frame is output, the human body is segmented from the video frame, RGB three-channel data are converted into one-dimensional data through the video frame image data, and the specific formula is as follows:
I＝(60R+118G+22B+50)/200
wherein, I represents new one-dimensional data, and RGB represents original three-channel image data.
The human body key point extraction module receives a human body segmentation image, the human body segmentation image is marked with a head and four limbs, the human body segmentation image is normalized to 368 × 368 and a network based on a CPMs algorithm, the positions of the human body key points are predicted from the marked head and four limbs through a deep convolution network, the size of a receptive field is 160 × 160, the final regression length is an output vector of P +1, the possibility of existence of the human body key points of each receptive field is represented, and the network is continuously repeated to finally output the human body key points;
the specific calculation formula of the loss function is as follows:
wherein t represents the number of network repetitive training times, p represents the key points of the human body, Z represents the set of all the points of the input human body segmentation image,
representing the characteristic values generated by t times of network repeated training,
representing the distance of the feature value from the human body key point.
The original CPMs algorithm needs to continuously predict from the whole image, more accurate human body joint points are continuously obtained under the constraint of the human body joint points through repeated training of a network, more accurate prediction data are obtained by a smaller calculated amount on the basis of a human body segmentation image with heads and four limbs marked by a behavior detection module, and the problem of gradient messages generated in the model training process is greatly avoided.
The dangerous behavior judgment comprises hand biting judgment, face grabbing judgment, chest beating judgment and head beating judgment;
the hand biting judgment method comprises the following steps: when the distance between the hand joint point and the head joint point is smaller than the hand biting threshold value and the time of accumulating the distance to the small hand biting threshold value is larger than the hand biting threshold value, the angle between the elbow joint point and the shoulder is continuously changed in the hand biting range, and the hand biting behavior of the user is judged;
the face grabbing judgment method comprises the following steps: when the hand joint point is in point contact with the head joint point and moves up and down, the movement speed exceeds a face-grabbing threshold value to judge that the user has a face-grabbing behavior;
the method for judging the chest beating comprises the following steps: when the hand joint point is intermittently contacted with the trunk joint point, the contact time is less than a flapping threshold value, and the intermission time is less than an intermission threshold value, judging that the user has the chest flapping behavior;
the beat judging method comprises the following steps: when the hand joint point is intermittently contacted with the head joint point, the contact time is less than the flapping threshold value, and the intermission time is less than the intermission threshold value, the user is judged to have the flapping behavior.
Whether the autistic children do the above behaviors can be accurately obtained through logic judgment, and the accurate judgment can be made at a very low cost.
The method comprises the following specific steps:
step seven: marking the new dangerous actions in the saved video file by a guardian or a treating doctor of the user, wherein the marking content comprises the start, the name and the danger level of the new dangerous actions;
step eight: the dangerous behavior labeling module converts the labeling file into standard output and puts the standard output and the original input into a dangerous behavior classifier;
step nine: extracting key point features by the dangerous behavior classifier, and training to obtain a new action classifier model;
step ten: after the behavior detection module detects behaviors, inputting the video file into a dangerous behavior classifier, identifying new dangerous actions by the dangerous behavior classifier, and inputting new dangerous behaviors into a behavior correction module;
step eleven: and the behavior correction module takes corresponding correction measures according to the danger level corresponding to the dangerous behavior.
According to the behavior of the user in the using process, after a guardian marks the behavior, the behavior can be automatically trained, and the behavior can be found and monitored in the first time when the user appears in the follow-up process.
The label file does not divide a training set and a testing set, and overfitting is effectively utilized to be closer to personal use of a user.
The standard output in the step eight is specifically as follows: the dangerous behavior labeling module is used for converting joint point data, label data and video frame image data into a one-dimensional array and inputting the one-dimensional array into a dangerous behavior classifier;
the dangerous behavior classifier is based on a Fast R-CNN algorithm, a main network is VGG16, the reception fields of different networks are changed simultaneously, and characteristic values obtained by the different reception fields are fused together;
and outputting the model once each training, taking newly marked data as detection data, calculating the average precision mean value of the model, and continuously outputting the model with the maximum average precision mean value.
A model is built based on a Fast R-CNN algorithm, and the capture of context information by the model of the dangerous behavior classifier is increased through the feature fusion of different receptive fields, the behavior is the expression of the combination of multiple joint points, and the capture of context information in an image by the model is increased, so that the recognition of a new dangerous behavior by the dangerous behavior classifier is effectively improved.
Compared with the prior art, the invention has the following beneficial effects: the video monitoring system helps a guardian to obtain effective data, image or sound information in real time, timely monitor and memorize the process of the sudden abnormal event, provide short-time simple response and processing, help doctors to effectively and timely perform high-degree review and analysis after the event, promote the understanding of children with autism, and help the children with autism to learn and integrate into the society in a comfortable environment;
restraining the self-disabled behaviors of the children with the autism, identifying that the self-disabled behaviors exist, sending a warning to a guardian, wherein the self-disabled behaviors comprise self-alarming, head hitting and eye scratching, judging the self-injurious behavior level of the children with the autism at the moment according to the identified behaviors, making different judgments, wherein the judgment comprises sending the warning to the guardian, transferring the interest of the children with the autism by using stimulation, sending some instructions to enable the children to enter new actions and behaviors, recording the whole process, helping the guardian and a treating doctor to analyze and judge the behavior reasons of the children with the autism, reading the reasons behind the behaviors of the children with the autism, providing the children with the autism with a target in a planned way, and continuously adjusting the danger levels of different dangerous behaviors according to actual conditions.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
fig. 1 is a schematic structural diagram of an autistic child safety monitoring system and method based on image recognition according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, the present invention provides the following technical solutions: an autism child safety monitoring system based on image recognition comprises a front-end video acquisition module, a video transmission module, a management control module, a behavior detection module, a behavior correction module, a dangerous behavior labeling module and a dangerous behavior classifier;
the video monitoring system comprises a front-end video acquisition module, a video transmission module, a management control module, a time generator, a superposition time entering image, a behavior detection module, a dangerous behavior classifier, a terminal and a behavior correction module, wherein the front-end video acquisition module acquires an original video signal shot by front-end monitoring equipment, the video transmission module encodes the original video signal and transmits the encoded original video signal through a network service protocol, the management control module controls the front-end monitoring equipment, the video and sound signals are stored and cloud-backed, the management control module is additionally provided with the time generator, the superposition time enters the image, the behavior detection module receives the video signal, the dangerous behavior classifier is used for carrying out behavior detection on characters in the video, dangerous behaviors of the characters are labeled, labeling information is output to the terminal and the behavior correction module, the terminal displays a monitoring picture and a behavior labeling result in real time, the behavior correction module gives different correction measures according to the dangerous behaviors of the characters, the dangerous behavior labeling module provides a window for allowing guardians and medical staff to label dangerous behaviors of users, the dangerous behaviors, the dangerous behavior analysis module inputs a labeling file into the dangerous behavior module, the dangerous behavior classifier is used for further training, and the dangerous behavior classifier extracts characteristics of the dangerous behaviors and classifies and outputs the dangerous behaviors.
The video monitoring system guardian can obtain effective data, image or sound information in real time, timely monitor and memorize the process of the sudden abnormal event, provide simple response and processing in a short time, help doctors to effectively and timely review and analyze the event, promote the understanding of the children with autism, and help the children with autism to learn and integrate into the society in a comfortable environment.
The behavior correction module receives that the user is making dangerous behaviors and makes corresponding correction measures according to the grades of the dangerous behaviors;
when the dangerous behavior grade is one, the behavior correction module records the dangerous behaviors and the time length of the dangerous behaviors of the user, stores corresponding marked videos, generates a record file and sends the record file to the guardian;
when the dangerous behavior grade is two, the behavior correction module records the dangerous behaviors and the time length of the dangerous behaviors of the user, stores the corresponding marked video, generates a recording file and sends the recording file to a guardian, and sends an instruction in the existence period of the dangerous behaviors of the user to prompt the user to enter new actions and behaviors;
when the dangerous behavior grade is three, the behavior correction module records the dangerous behaviors and the time length of the dangerous behaviors of the user, stores corresponding marked videos, generates a recording file and sends the recording file to a monitor, and simultaneously sends out a warning to the monitor and a stimulation behavior to the user;
when the dangerous behavior grade is four, the behavior correction module records the dangerous behaviors and the time length of the dangerous behaviors of the user, stores the corresponding marked videos, generates a recording file, sends the recording file to a guardian, sends a warning to the guardian, and sends preset appeasing terms to the user.
The self-closed children make strange actions or self-injuring behaviors mainly have two aspects, one is that the self-closed children possibly suffer from frustration or confusion, and the other is that the self-closed children are in a boring state and are in an unnoticed state in a group, so that the self-closed children can relieve the mind and body, avoid the conflict on the environment, stop the current anxiety behaviors, help the self-closed children to control the bodies of the self-closed children, release and relieve from the current anxiety mood, eliminate the fear of the self-closed children to unknown disorder, promote more and more active adaptation of the self-closed children to be integrated into the society, and realize the balanced harmony of the self-closed children and the social environment.
For the behaviors with the danger level of one, namely harmless behaviors, proper indifference is paid to the harmless strange behaviors of the autistic children, and the autistic children are given the freedom of controlling the body of the autistic children in the harmless degree;
when the dangerous behavior grade is two, namely the harmless behavior but the behavior needing to be corrected, some instructions are sent to enable the autistic child to get away from the current behavior scene, learn new behaviors, enable the autistic child to do something and cannot further generate self-injurious behaviors;
when the dangerous behavior grade is three, namely the behavior has the possibility of harm and is a precursor behavior of the autistic child starting to react excessively, a new interest point is generated through the stimulating behavior, the autistic child is attracted to be separated from the current behavior and emotion, the fear emotion in the abreaction of the autistic child is helped, the possibility that the current environment is communicated with other people is found, and the autistic child is helped to master the body again;
when the dangerous behavior level is four, the behavior can generate danger or be an autism child overstimulation reaction self-injuring behavior, the autism child is pacified through the pacifying words of the guardian input in advance, the guardian is timely informed, and further pacifying measures are taken for the autism child.
The stimulation behaviors comprise simulating natural sounds, simulating animal cry and playing youthful music, the behavior correction module records the influence of different appeasing phrases and stimulation behaviors on the user, compares the time of dangerous behaviors in the same level, sorts the adopted appeasing phrases or the stimulation behaviors according to the time of the dangerous behaviors, generates a report and sends the report to the guardian of the user.
The stimulating behaviors do not include stimulating prompts such as loud prompt sound, bright light flashing and the like, the autistic children are often attracted by new interest points and stop the current stereotypy,
the autistic children can not accept strong light and noise, can not accept touch, can easily repeat purposeless meaningless behaviors, move according to a certain sequence, like three primary colors, round and arc, and can be pacified and behavior corrected for the autistic children on the basis of respecting the behavior logic of the autistic children.
An autism child safety monitoring method based on image recognition comprises the following specific steps:
the method comprises the following steps: the front-end video acquisition module acquires real-time video data;
step two: extracting video frames in unit time, and converting the video into images;
step three: the behavior detection module detects whether people exist in the video frame or not and segments the people from the picture;
step four: acquiring key points of a human body;
step five: performing logic judgment on the acquired human body key points, and inputting dangerous behavior labels into a behavior correction module when the human body key points conform to dangerous behaviors;
step six: the behavior correction module takes corresponding correction measures according to the danger level corresponding to the dangerous behavior;
the safety monitoring system is used for supplementing the care of a guardian for the autistic child, the autistic child is not intellectual disorder but social disorder, the understanding of social rules and safety rules is lacked, the dangerous behaviors of the autistic child are identified and corrected by simple measures, the alarm and prompt are sent to the guardian of the autistic child, the autistic child is helped to understand the safety rules, the guardian is helped to nurse the autistic child, and the monitoring burden of the guardian is relieved.
Restraining the self-disabled behaviors of the children with the autism, identifying that the self-disabled behaviors exist, sending a warning to a guardian, wherein the self-disabled behaviors comprise self-alarming, head hitting and eye scratching, judging the self-injurious behavior level of the children with the autism at the moment according to the identified behaviors, making different judgments, wherein the judgment comprises sending the warning to the guardian, transferring the interest of the children with the autism by using stimulation, sending some instructions to enable the children to enter new actions and behaviors, recording the whole process, helping the guardian and a treating doctor to analyze and judge the behavior reasons of the children with the autism, reading the reasons behind the behaviors of the children with the autism, providing the children with the autism with a target in a planned way, and continuously adjusting the danger levels of different dangerous behaviors according to actual conditions.
The behavior detection module comprises a human body detection module, a human body key point extraction module and a dangerous behavior judgment module, wherein the human body detection module detects and extracts characters in a video frame and inputs the characters into the human body key point extraction module, the human body key points are extracted and input into the dangerous behavior judgment module, and when the key points of the human body accord with the dangerous behavior, the dangerous behavior is marked and output;
the behavior detection module builds a network based on a YOLO algorithm, a data set marks information of a human body, a head and four limbs, CSPDarknet53 is used as a backbone network, feature fusion is carried out through PANet, output pictures are identified through YOLOv3, video frame image data are input, human body, head and four limb marking information in a video frame are output, the human body is segmented from the video frame, RGB three-channel data are converted into one-dimensional data through the video frame image data, and the specific formula is as follows:
I＝(60R+118G+22B+50)/200
wherein, I represents new one-dimensional data, and RGB represents original three-channel image data.
The human body key point extraction module receives a human body segmentation image, the human body segmentation image is marked with a head and four limbs, the human body segmentation image is normalized to 368 × 368 and a network based on a CPMs algorithm, the positions of the human body key points are predicted from the marked head and four limbs through a deep convolution network, the size of a receptive field is 160 × 160, the final regression length is an output vector of P +1, the possibility of existence of the human body key points of each receptive field is represented, and the network is continuously repeated to finally output the human body key points;
the specific calculation formula of the loss function is as follows:
wherein t represents the number of times of network repetitive training, p represents a human body key point, Z represents a set of all points of the input human body segmentation image,
representing the characteristic values generated by t times of network repeated training,
representing the distance of the feature value from the human body key point.
The original CPMs algorithm needs to continuously predict from the whole image, more accurate human body joint points are continuously obtained under the constraint of the human body joint points through repeated training of a network, more accurate prediction data are obtained by a smaller calculated amount on the basis of a human body segmentation image with heads and four limbs marked by a behavior detection module, and the problem of gradient messages generated in the model training process is greatly avoided.
The dangerous behavior judgment comprises hand biting judgment, face grabbing judgment, chest beating judgment and head beating judgment;
the method for judging hand biting comprises the following steps: when the distance between the hand joint point and the head joint point is smaller than the hand biting threshold value and the time of accumulating the distance to the small hand biting threshold value is larger than the hand biting threshold value, the angle between the elbow joint point and the shoulder is continuously changed in the hand biting range, and the hand biting behavior of the user is judged;
the face grabbing judgment method comprises the following steps: when the hand joint point contacts with the head joint point and moves up and down, the movement speed exceeds a face grabbing threshold value to judge that the user has a face grabbing behavior;
the method for judging the chest beating comprises the following steps: when the hand joint point is intermittently contacted with the trunk joint point, the contact time is less than a beating threshold value, and the intermittent time is less than an intermittent threshold value, judging that the chest beating action of the user exists;
the beat judging method comprises the following steps: and when the hand joint point is intermittently contacted with the head joint point, the contact time is less than the flapping threshold value, and the intermittent time is less than the intermittent threshold value, judging that the user has the head flapping behavior.
Whether the autistic children do the above behaviors can be accurately obtained through logic judgment, and the accurate judgment can be made at a very low cost.
The method comprises the following specific steps:
step seven: a guardian or a treating doctor of a user marks new dangerous actions in a stored video file, wherein the marking content comprises the start, the name and the danger level of the new dangerous actions;
step eight: the dangerous behavior labeling module converts the labeling file into standard output and puts the standard output and the original input into a dangerous behavior classifier;
step nine: extracting key point characteristics by a dangerous behavior classifier, and training to obtain a new action classifier model;
step ten: after the behavior detection module detects the behaviors, the video file is input into a dangerous behavior classifier, the dangerous behavior classifier identifies new dangerous actions, and new dangerous behaviors are marked and input into the behavior correction module;
step eleven: and the behavior correction module takes corresponding correction measures according to the danger level corresponding to the dangerous behavior.
According to the behavior of the user in the using process, after a guardian marks the behavior, the behavior can be automatically trained, and the behavior can be found and monitored in the first time when the user appears in the follow-up process.
The label file does not divide a training set and a test set, and overfitting is effectively utilized to be closer to the personal use of a user.
The standard output in the step eight is specifically as follows: the dangerous behavior labeling module is used for converting joint point data, label data and video frame image data into a one-dimensional array and inputting the one-dimensional array into a dangerous behavior classifier;
the dangerous behavior classifier is based on a Fast R-CNN algorithm, a main network is VGG16, receptive fields of different networks are changed simultaneously, and characteristic values obtained by the different receptive fields are fused together;
and outputting the model once each training, taking newly marked data as detection data, calculating the average precision mean value of the model, and continuously outputting the model with the maximum average precision mean value.
A model is built based on a Fast R-CNN algorithm, and the capture of context information by the model of the dangerous behavior classifier is increased through the feature fusion of different receptive fields, the behavior is the expression of the union of multiple joint points, and the capture of context information in an image by the model is increased, so that the recognition of a new dangerous behavior by the dangerous behavior classifier is effectively improved.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (9)
1. The utility model provides an autism children safety monitored control system based on image recognition which characterized in that: the system comprises a front-end video acquisition module, a video transmission module, a management control module, a behavior detection module, a behavior correction module, a dangerous behavior marking module and a dangerous behavior classifier;
the video monitoring system comprises a front-end video acquisition module, a video transmission module, a management control module, a time generator, a behavior detection module, a dangerous behavior classifier, a terminal and a behavior correction module, wherein the front-end video acquisition module acquires an original video signal shot by front-end monitoring equipment, the video transmission module encodes the original video signal and transmits the encoded original video signal through a network service protocol, the management control module controls the front-end monitoring equipment to store and cloud backup video and sound signals, the management control module is additionally provided with the time generator, overlapping time enters images, the behavior detection module receives the video signal, calls the dangerous behavior classifier to perform behavior detection on characters in the video, labels dangerous behaviors of the characters and outputs labeling information to the terminal and the behavior correction module, the terminal displays a monitoring picture and behavior labeling results in real time, the behavior correction module gives different correction measures according to the dangerous behaviors of the characters, the dangerous behavior labeling module provides a window for the guardians and medical staff to label the dangerous behaviors of the users in the stored video files, the dangerous behavior analysis module inputs the labeled files into the dangerous behavior analysis module to help the dangerous behavior classifier to further train, and the dangerous behavior classifier extracts the characteristics of the dangerous behaviors and performs classified output training.
2. The autism child safety monitoring system according to claim 1, wherein: the behavior correction module receives that a user is making dangerous behaviors and makes corresponding correction measures according to the grades of the dangerous behaviors;
when the behavior is harmless, setting the dangerous behavior grade as one, recording the dangerous behavior and the time length of the dangerous behavior of the user by the behavior correction module, storing a corresponding marked video, generating a record file and sending the record file to a guardian;
when the behavior is harmless but needs to be corrected, setting the dangerous behavior grade to be two, recording the dangerous behavior and the time length of the dangerous behavior of the user by the behavior correction module, storing a corresponding annotation video, generating a recording file and sending the recording file to a guardian, and sending an instruction during the storage period of the dangerous behavior of the user to prompt the user to enter a new action and behavior;
when the behavior is possibly harmful and is a precursor behavior of the over-excited response of the user, setting the dangerous behavior grade to be three, recording the dangerous behavior and the time length of the dangerous behavior of the user by the behavior correction module, storing a corresponding labeled video, generating a recording file, sending the recording file to a guardian, sending a warning to the guardian, and sending an exciting behavior to the user;
when the behavior is harmful or is an over-excited behavior or a self-injuring behavior of the user, the dangerous behavior grade is set to be four, the behavior correction module records the dangerous behavior and the duration of the dangerous behavior of the user, stores the corresponding labeled video, generates a recording file and sends the recording file to the guardian, meanwhile, the recording file sends a warning to the guardian, and sends a preset appeasing phrase to the user.
3. The autism child safety monitoring system according to claim 2, wherein: the stimulating behaviors comprise simulating natural sounds, simulating animal cry and playing lingering music, the behavior correction module records the influence of different appeasing phrases and stimulating behaviors on the user, compares the time of dangerous behaviors in the same level, sorts the adopted appeasing phrases or stimulating behaviors according to the time of the dangerous behaviors, generates a report and sends the report to the guardian of the user.
4. An autism child safety monitoring method based on image recognition is characterized in that: the method comprises the following specific steps:
the method comprises the following steps: the front-end video acquisition module acquires real-time video data;
step two: extracting video frames in unit time, and converting the video into images;
step three: the behavior detection module detects whether people exist in the video frame or not and divides people from the picture;
step four: the behavior detection module acquires key points of a human body;
step five: the behavior detection module carries out logic judgment on the acquired human body key points, and when the human body key points conform to dangerous behaviors, dangerous behavior labels are input into the behavior correction module;
step six: and the behavior correction module takes corresponding correction measures according to the danger level corresponding to the dangerous behavior.
5. The autism child safety monitoring system according to claim 4, wherein: the behavior detection module comprises a human body detection module, a human body key point extraction module and dangerous behavior judgment, the human body detection module detects and extracts characters in the video frame and inputs the characters into the human body key point extraction module, the human body key point extraction module inputs the human body key points into the dangerous behavior judgment module, and when the key points of the human body accord with the dangerous behaviors, the dangerous behaviors are marked and output;
the behavior detection module builds a network based on a YOLO algorithm, a data set marks information of a human body, a head and four limbs, a CSPDarknet53 is used as a backbone network, feature fusion is carried out through PANet, output pictures are identified through YOLOv3, video frame image data are input, the marking information of the human body, the head and the four limbs in a video frame is output, the human body is segmented from the video frame, RGB three-channel data are converted into one-dimensional data through the video frame image data, and the specific formula is as follows:
I＝(60R+118G+22B+50)/200
wherein, I represents new one-dimensional data, and RGB represents original three-channel image data.
6. The autism child safety monitoring system according to claim 5, wherein: the human body key point extraction module receives a human body segmentation image, the human body segmentation image is marked with a head and four limbs, the human body segmentation image is normalized to 368 × 368 and a network based on a CPMs algorithm, the positions of the human body key points are predicted from the marked head and four limbs through a deep convolution network, the size of a receptive field is 160 × 160, the final regression length is an output vector of P +1, the possibility of existence of the human body key points of each receptive field is represented, and the network is continuously repeated to finally output the human body key points;
the specific calculation formula of the loss function is as follows:
wherein t represents the number of repeated training times of the network, p represents key points of the human body, and Z represents all the input human body segmentation imagesThe set of points is then selected from the group,
representing the characteristic values generated by t times of network repeated training,
representing the distance of the feature value from the human body key point.
7. The image recognition-based autism child safety monitoring system of claim 6, wherein: the dangerous behavior judgment comprises hand biting judgment, face grabbing judgment, chest patting judgment and head patting judgment;
the hand biting judgment method comprises the following steps: when the distance between the hand joint point and the head joint point is smaller than a hand biting threshold value and the time of accumulating the distance to be smaller than the hand biting threshold value is longer than the hand biting threshold value, the angle between the elbow joint point and the shoulder is continuously changed in the hand biting range, and the hand biting behavior of the user is judged;
the face grabbing judgment method comprises the following steps: when the hand joint point is in point contact with the head joint point and moves up and down, the movement speed exceeds a face-grabbing threshold value to judge that the user has a face-grabbing behavior;
the method for judging chest patting comprises the following steps: when the hand joint point is intermittently contacted with the trunk joint point, the contact time is less than a beating threshold value, and the intermittent time is less than an intermittent threshold value, judging that the chest beating action of the user exists;
the beat judging method comprises the following steps: and when the hand joint point is intermittently contacted with the head joint point, the contact time is less than the flapping threshold value, and the intermittent time is less than the intermittent threshold value, judging that the user has the head flapping behavior.
8. The system according to claim 7, wherein the system comprises: the method comprises the following specific steps:
step seven: marking the new dangerous actions in the saved video file by a guardian or a treating doctor of the user, wherein the marking content comprises the start, the name and the danger level of the new dangerous actions;
step eight: the dangerous behavior labeling module converts the labeled file into standard output, and the standard output and the original input are put into a dangerous behavior classifier;
step nine: extracting key point characteristics by the dangerous behavior classifier, and training to obtain a new action classifier model;
step ten: after the behavior detection module detects behaviors, the video file is input into a dangerous behavior classifier, the dangerous behavior classifier identifies new dangerous actions, and the new dangerous actions are marked and input into a behavior correction module;
step eleven: and the behavior correction module takes corresponding correction measures according to the danger level corresponding to the dangerous behavior.
9. The image recognition-based autism child safety monitoring system of claim 8, wherein: the standard output in the step eight is specifically as follows: the dangerous behavior labeling module is used for converting joint point data, label data and video frame image data into a one-dimensional array and inputting the one-dimensional array into a dangerous behavior classifier;
the dangerous behavior classifier is based on a Fast R-CNN algorithm, a main network is VGG16, receptive fields of different networks are changed simultaneously, and characteristic values obtained by the different receptive fields are fused together;
and outputting the model once each training, taking newly marked data as detection data, calculating the average precision mean value of the model, and continuously outputting the model with the maximum average precision mean value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211383640.6A CN115691762A (en) | 2022-11-07 | 2022-11-07 | Autism child safety monitoring system and method based on image recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211383640.6A CN115691762A (en) | 2022-11-07 | 2022-11-07 | Autism child safety monitoring system and method based on image recognition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115691762A true CN115691762A (en) | 2023-02-03 |
Family
ID=85049272
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211383640.6A Pending CN115691762A (en) | 2022-11-07 | 2022-11-07 | Autism child safety monitoring system and method based on image recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115691762A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117253185A (en) * | 2023-09-08 | 2023-12-19 | 杭州大松科技有限公司 | Medical security monitoring management system and method |
| CN117831253A (en) * | 2024-03-06 | 2024-04-05 | 长春汽车工业高等专科学校 | Automatic alarm method and system based on pattern recognition |
-
2022
- 2022-11-07 CN CN202211383640.6A patent/CN115691762A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117253185A (en) * | 2023-09-08 | 2023-12-19 | 杭州大松科技有限公司 | Medical security monitoring management system and method |
| CN117253185B (en) * | 2023-09-08 | 2024-05-28 | 杭州大松科技有限公司 | Medical security monitoring management system and method |
| CN117831253A (en) * | 2024-03-06 | 2024-04-05 | 长春汽车工业高等专科学校 | Automatic alarm method and system based on pattern recognition |
| CN117831253B (en) * | 2024-03-06 | 2024-05-07 | 长春汽车工业高等专科学校 | Automatic alarm method and system based on pattern recognition |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115691762A (en) | Autism child safety monitoring system and method based on image recognition | |
| CN105760861B (en) | Epileptic seizure monitoring method and system based on depth data | |
| Min et al. | Automatic characterization and detection of behavioral patterns using linear predictive coding of accelerometer sensor data | |
| CN110162182B (en) | Electroencephalogram control module device and method for controlling controlled equipment by electroencephalogram control module device | |
| CN110363129A (en) | Autism early screening system based on smile normal form and audio-video behavioural analysis | |
| CN111882820B (en) | Nursing system for special people | |
| CN114241375B (en) | Monitoring method for exercise process | |
| CN116313087A (en) | Method and device for identifying psychological state of autism patient | |
| CN115035498A (en) | Riding safety control method and device, electronic equipment and product | |
| CN108958482A (en) | A kind of similitude action recognition device and method based on convolutional neural networks | |
| CN113040773A (en) | Data acquisition and processing method | |
| CN113080855A (en) | Facial pain expression recognition method and system based on depth information | |
| CN113706824B (en) | Old man nurses system at home based on thing networking control | |
| Min et al. | Optimal sensor location for body sensor network to detect self-stimulatory behaviors of children with autism spectrum disorder | |
| Vettivel et al. | System for detecting student attention pertaining and alerting | |
| CN116884577A (en) | Psychological therapy healing method, device, equipment and medium based on machine vision | |
| Bhargavi et al. | AI-based Emotion Therapy Bot for Children with Autism Spectrum Disorder (ASD) | |
| CN110693508A (en) | Multi-channel cooperative psychophysiological active sensing method and service robot | |
| CN114399993A (en) | Standard law enforcement training system | |
| CN110353703B (en) | Autism assessment device and system based on parrot tongue learning language model behavior analysis | |
| CN113221661A (en) | Intelligent human body tumbling detection system and method | |
| Kasodu et al. | CNN-based Drowsiness Detection with Alarm System to Prevent Microsleep | |
| Bharti et al. | An enhanced feature extraction method and classification method of EEG signals using artificial intelligence | |
| CN115147897A (en) | Active sensory control system of rehabilitation wheelchair | |
| Cerna et al. | Enhancing Infants Early Communication through Sign Language Recognition System using Raspberry Pi and Convolutional Neural Network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |