CN116883946A - Method, device, equipment and storage medium for detecting abnormal behaviors of old people in real time - Google Patents
Method, device, equipment and storage medium for detecting abnormal behaviors of old people in real time Download PDFInfo
- Publication number
- CN116883946A CN116883946A CN202310923016.9A CN202310923016A CN116883946A CN 116883946 A CN116883946 A CN 116883946A CN 202310923016 A CN202310923016 A CN 202310923016A CN 116883946 A CN116883946 A CN 116883946A
- Authority
- CN
- China
- Prior art keywords
- old
- flow field
- real
- image
- optical flow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 206010000117 Abnormal behaviour Diseases 0.000 title claims abstract description 99
- 238000000034 method Methods 0.000 title claims abstract description 61
- 230000033001 locomotion Effects 0.000 claims abstract description 84
- 238000001514 detection method Methods 0.000 claims abstract description 60
- 238000004458 analytical method Methods 0.000 claims abstract description 18
- 230000000474 nursing effect Effects 0.000 claims abstract description 15
- 238000011897 real-time detection Methods 0.000 claims abstract description 11
- 230000003287 optical effect Effects 0.000 claims description 110
- 230000006399 behavior Effects 0.000 claims description 59
- 206010044565 Tremor Diseases 0.000 claims description 24
- 230000002159 abnormal effect Effects 0.000 claims description 23
- 238000004422 calculation algorithm Methods 0.000 claims description 19
- 238000000605 extraction Methods 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 8
- 210000000746 body region Anatomy 0.000 claims description 5
- 239000013598 vector Substances 0.000 description 16
- 238000012545 processing Methods 0.000 description 10
- 238000012544 monitoring process Methods 0.000 description 9
- 230000009471 action Effects 0.000 description 8
- 230000008859 change Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 230000036541 health Effects 0.000 description 5
- 208000018737 Parkinson disease Diseases 0.000 description 4
- 208000010877 cognitive disease Diseases 0.000 description 4
- 238000006073 displacement reaction Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 208000024827 Alzheimer disease Diseases 0.000 description 2
- 208000019901 Anxiety disease Diseases 0.000 description 2
- 208000020084 Bone disease Diseases 0.000 description 2
- 208000028698 Cognitive impairment Diseases 0.000 description 2
- 206010028289 Muscle atrophy Diseases 0.000 description 2
- 208000012902 Nervous system disease Diseases 0.000 description 2
- 208000025966 Neurological disease Diseases 0.000 description 2
- 206010039966 Senile dementia Diseases 0.000 description 2
- 208000006011 Stroke Diseases 0.000 description 2
- 230000032683 aging Effects 0.000 description 2
- 230000036506 anxiety Effects 0.000 description 2
- 206010003246 arthritis Diseases 0.000 description 2
- 230000003920 cognitive function Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000011068 loading method Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000036651 mood Effects 0.000 description 2
- 230000020763 muscle atrophy Effects 0.000 description 2
- 201000000585 muscular atrophy Diseases 0.000 description 2
- 208000024891 symptom Diseases 0.000 description 2
- 208000027523 Abnormality of the hand Diseases 0.000 description 1
- 238000012935 Averaging Methods 0.000 description 1
- 208000012661 Dyskinesia Diseases 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000004907 flux Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000005802 health problem Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/04—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons
- G08B21/0407—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons based on behaviour analysis
- G08B21/043—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons based on behaviour analysis detecting an emergency event, e.g. a fall
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/04—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons
- G08B21/0438—Sensor means for detecting
- G08B21/0476—Cameras to detect unsafe condition, e.g. video cameras
Abstract
The invention relates to the technical field of real-time detection of abnormal behaviors of old people, solves the problem that the abnormal behaviors of the old people cannot be accurately detected in real time in the prior art, and is poor in nursing experience of the old people of users, and provides a real-time detection method, device and equipment of the abnormal behaviors of the old people and a storage medium. The method comprises the following steps: acquiring a real-time video stream under a nursing scene of the aged, and decomposing the real-time video stream into multi-frame real-time images; extracting the characteristics of each real-time image, and acquiring a target image containing the old people according to the extracted human body characteristic information; performing motion analysis on each target image, and outputting motion characteristic information of a preset part of the old; detecting the preset abnormal behavior of the old according to the movement characteristic information, and outputting a detection result; and according to the detection result, when the abnormal behavior of the old is identified, a safety prompt is sent to the user. The invention realizes real-time accurate detection of the abnormal behavior of the old and improves the nursing experience of the old of the user.
Description
Technical Field
The invention relates to the technical field of real-time detection of abnormal behaviors of old people, in particular to a real-time detection method, device and equipment of abnormal behaviors of old people and a storage medium.
Background
With the trend of global population aging increasing, how to provide high-quality and effective care services for the aged, especially to detect and treat various abnormal behaviors in time in their daily lives, has become an important issue of social concern. In recent years, the rapid development of intelligent technology, especially the application of artificial intelligence, machine learning, internet of things and other technologies, provides new possibility for the real-time detection of the abnormal behaviors of the old. By utilizing the technologies, the detection accuracy and the real-time performance can be improved, so that the life safety of the old can be better ensured. However, how to design and implement an effective and accurate real-time detection method for abnormal behaviors of the aged is still an important research topic. Traditional detection of the abnormal behaviour of elderly people mainly relies on manual observation or some simple sensor devices, such as pressure sensors, infrared sensors, etc. These methods have significant limitations. Firstly, manual observation requires a large amount of manpower and material resources, and continuous monitoring for 24 hours cannot be achieved. Second, elderly people may be inattentive due to reduced cognitive function, experience abnormal wandering behavior, forget their own destination or get lost, such as cognitive impairment: cognitive disorder diseases such as senile dementia may cause the elderly to become lost and wander because they have difficulty remembering the environment and direction; depression or anxiety: mood problems may lead to the elderly wandering intentionally and unintentionally seeking comfort or relief; living environment changes: elderly people may become lost and wander due to changing residence or reduced personals; elderly people may also be unable to stand due to problems with bone diseases (such as fractures), muscle atrophy or arthritis; neurological diseases such as stroke, parkinson's disease, etc. may affect balance and standing ability of the elderly; among them, one of the main symptoms of parkinson's disease is trembling of the hands, which may cause difficulty in controlling the hand movements of the elderly. The simple sensor device can monitor all weather, but can only provide limited information, and the abnormal behavior state of the old is difficult to accurately judge.
The prior Chinese patent CN116189232A discloses a machine vision-based method and a machine vision-based system for detecting abnormal behaviors of old people in a nursing home, wherein the method comprises the following steps: acquiring video data shot by a camera; detecting abnormal behaviors of video data containing target objects based on different abnormal behavior recognition models respectively; the abnormal behavior recognition model comprises an active hand lifting help recognition model, a falling behavior recognition model and a long standing behavior recognition model; aiming at the abnormal behavior of active help seeking by lifting the hand, an active help seeking identification model is adopted for detection, and the active help seeking identification model is a target detection algorithm based on YOLOv 5; the falling behavior recognition model adopts the YOLOV5-CBAM detection model to detect a target object, and the shapes of the external rectangular frames of the falling behavior recognition model are obviously different from those of the falling behavior recognition model in the vertical state of a human body, so that the aspect ratio of the external rectangular frames of the human body detection can be adopted to judge so as to determine whether the detection target falls; and detecting the abnormal behavior of the permanent stationary by adopting a permanent stationary behavior identification model, tracking the existing target by combining a Deepsort target tracking model based on the detection result of the YOLOV5-CBAM detection model, creating a new target for the new target, and deleting the target which is not matched for a long time. The above method only uses the YOLOv5 target detection algorithm to detect abnormal behaviors, and the performance and accuracy of the target detection algorithm may be affected by factors, such as target occlusion, illumination change or target scale change, which may cause inaccuracy of target detection and further affect the result of abnormal behavior detection.
Therefore, how to accurately detect the abnormal behavior of the old in real time and improve the nursing experience of the old of the user is a problem to be solved urgently.
Disclosure of Invention
In view of the above, the invention provides a method, a device, equipment and a storage medium for detecting the abnormal behavior of the old, which are used for solving the problems that the abnormal behavior of the old cannot be accurately detected in real time in the prior art and the nursing experience of the old of a user is poor.
The technical scheme adopted by the invention is as follows:
in a first aspect, the present invention provides a method for detecting abnormal behaviors of old people in real time, which is characterized in that the method includes:
s1: acquiring a real-time video stream under a nursing scene of the aged, and decomposing the real-time video stream into multi-frame real-time images;
s2: extracting the characteristics of each real-time image, and acquiring a target image containing the old people according to the extracted human body characteristic information;
s3: performing motion analysis on each target image, and outputting motion characteristic information of a preset part of the old;
s4: detecting the preset abnormal behavior of the old according to the movement characteristic information, and outputting a detection result;
s5: and according to the detection result, when the abnormal behavior of the old is identified, a safety prompt is sent to the user.
Preferably, the S2 includes:
s21: detecting each image according to a target detection algorithm, and outputting upper body position information of a human body;
s22: according to the upper body position information, extracting the characteristics of an upper body region of the human body by utilizing a target classification network, and outputting upper body characteristic information of the human body;
s23: inputting the upper body characteristic information into a pre-trained classifier, and outputting a classification result;
s24: and acquiring the target image when the number of image frames classified as old people in each real-time image is greater than the preset number of image frames according to the classification result.
Preferably, the S3 includes:
s31: tracking the old people in the target image by utilizing a target matching algorithm;
s32: inputting the target image into a pre-trained target detection model, and outputting an image of a preset part of the old, wherein the image of the preset part comprises a hand image and a trunk image;
s33: and carrying out optical flow field calculation on the trunk image and the hand image, and outputting an average optical flow field of the trunk image and a center optical flow field of the hand image as the motion characteristic information.
Preferably, the S4 includes:
s41: acquiring preset abnormal behaviors of the old, wherein the abnormal behaviors of the old at least comprise one of the following behaviors: abnormal loitering of the old, incapacity of standing of the old and trembling of the hands of the old;
S42: acquiring a preset time interval, and identifying abnormal loitering behaviors of the old people when the trunk of the old people is judged to have repeated movement according to the variance of the average light flow field in the time interval;
s43: identifying the standing incapacity of the old people according to the first average value of the average light flow field and the second average value of the central light flow field in the time interval;
s44: and identifying the tremble behavior of the hands of the old people when judging that the hands of the old people have weak motions according to the central light flow field in the time interval.
Preferably, the S42 includes:
s421: acquiring the time interval and a preset first optical flow field threshold value;
s422: calculating the average optical flow field in the time interval, and outputting the variance of the average optical flow field;
s423: and identifying the abnormal loitering behavior of the old person when the variance is greater than the first optical flow field threshold.
Preferably, the S43 includes:
s431: acquiring a preset second optical flow field threshold value and a preset third optical flow field threshold value;
s432: calculating the average optical flow field and the central optical flow field in the time interval, and outputting the first average value and the second average value;
S433: when the first average value is less than the second optical-flow field threshold value and the second average value is greater than the third optical-flow field threshold value, the elderly person is identified as not standing.
Preferably, the S44 includes:
s441: acquiring a preset fourth optical flow field threshold value;
s442: performing Fourier transform on the central optical flow field in the time interval, and outputting frequency components corresponding to the central optical flow field;
s443: comparing the frequency components, and identifying the hand tremor behavior when the maximum value of the frequency components is greater than the fourth optical flow field threshold.
In a second aspect, the present invention provides a real-time detection device for abnormal behavior of the elderly, the device comprising:
the real-time image acquisition module is used for acquiring a real-time video stream in an aged care scene and decomposing the real-time video stream into multi-frame real-time images;
the old man identification module is used for extracting the characteristics of each real-time image and acquiring a target image containing the old man according to the extracted human body characteristic information;
the motion characteristic extraction module is used for performing motion analysis on each target image and outputting motion characteristic information of a preset part of the old;
The old man abnormal behavior detection module is used for detecting preset old man abnormal behaviors according to the motion characteristic information and outputting detection results;
and the safety reminding module is used for sending a safety reminding to a user when the detection result is identified as the abnormal behavior of the old people.
In a third aspect, an embodiment of the present invention further provides an electronic device, including: at least one processor, at least one memory and computer program instructions stored in the memory, which when executed by the processor, implement the method as in the first aspect of the embodiments described above.
In a fourth aspect, embodiments of the present invention also provide a storage medium having stored thereon computer program instructions which, when executed by a processor, implement a method as in the first aspect of the embodiments described above.
In summary, the beneficial effects of the invention are as follows:
the invention provides a method, a device, equipment and a storage medium for detecting abnormal behaviors of old people in real time, wherein the method comprises the following steps: acquiring a real-time video stream under a nursing scene of the aged, and decomposing the real-time video stream into multi-frame real-time images; extracting the characteristics of each real-time image, and acquiring a target image containing the old people according to the extracted human body characteristic information; performing motion analysis on each target image, and outputting motion characteristic information of a preset part of the old; detecting the preset abnormal behavior of the old according to the movement characteristic information, and outputting a detection result; and according to the detection result, when the abnormal behavior of the old is identified, a safety prompt is sent to the user. According to the invention, through analyzing the real-time video stream and detecting the abnormal behavior, the motion characteristic information and the abnormal behavior of the old are analyzed, the individualized knowledge of the old is obtained, so that the abnormal behavior of the old is accurately detected, once the abnormal behavior of the old is detected, the system can send a safety prompt to a user, and timely safety prompt and intervention measures are beneficial to reducing the health and safety threat of accidents and emergency to the old, and the nursing experience of the old of the user is improved.
Drawings
In order to more clearly illustrate the technical solution of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly described, and it is within the scope of the present invention to obtain other drawings according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of the overall operation of the real-time detection method for abnormal behavior of the old in embodiment 1 of the present invention;
fig. 2 is a schematic flow chart of acquiring a target image including the elderly in embodiment 1 of the present invention;
FIG. 3 is a schematic flow chart of motion analysis of a target image in embodiment 1 of the present invention;
FIG. 4 is a schematic flow chart of identifying abnormal behaviors of the elderly in embodiment 1 of the present invention;
fig. 5 is a schematic flow chart of identifying abnormal loitering behavior of the old in embodiment 1 of the present invention;
FIG. 6 is a flow chart of the method for identifying the standing disabled state of the elderly in embodiment 1 of the present invention;
fig. 7 is a flow chart of identifying hand trembling behavior of the elderly in embodiment 1 of the present invention;
FIG. 8 is a block diagram showing the structure of the device for detecting the abnormal behavior of the elderly in real time according to embodiment 2 of the present invention;
fig. 9 is a schematic structural diagram of an electronic device in embodiment 3 of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. In the description of the present application, it should be understood that the terms "center," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like indicate orientations or positional relationships based on the orientation or positional relationships shown in the drawings, merely to facilitate describing the present application and simplify the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present application. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element. If not conflicting, the embodiments of the present application and the features of the embodiments may be combined with each other, which are all within the protection scope of the present application.
Example 1
Referring to fig. 1, embodiment 1 of the invention discloses a real-time detection method for abnormal behaviors of old people, which comprises the following steps:
s1: acquiring a real-time video stream under a nursing scene of the aged, and decomposing the real-time video stream into multi-frame real-time images;
specifically, acquiring a real-time video stream of an aged person in a nursing scene, wherein the real-time video stream is a color video stream shot by a camera installed under an oblique viewing angle, and decomposing the real-time video stream into N frames of real-time images; the camera under the oblique visual angle provides the more comprehensive video information of multi-angle, can real-time supervision old man's state and action, simultaneously, through breaking up real-time video stream into the condition that the old man can be attended to in the long-range of guardian to multiframe real-time image. Thus, potential problems can be found in time, help is provided, remote guidance is carried out, and the sense of safety and the care quality of the old are improved.
S2: extracting the characteristics of each real-time image, and acquiring a target image containing the old people according to the extracted human body characteristic information;
specifically, feature extraction is performed on each of the real-time images, and useful human feature information can be extracted from the images through image processing and computer vision techniques. According to the extracted human body characteristic information, a target image containing the old people can be acquired, namely, the part of the image related to the old people is extracted. In the feature extraction stage, each real-time image is analyzed and processed by utilizing image processing and computer vision technology, and distinguishing human body feature information is extracted. The human body characteristic information at least comprises the outline, key points, gestures, motion tracks, expressions and the like of a human body, and the target image containing the old people is identified and extracted by analyzing the human body characteristic information. Through feature extraction, attention is focused on the part of the image relevant to the old, unnecessary background interference is reduced, and the accuracy and precision of target positioning are improved, so that the actions, behaviors and state changes of the old are better captured; the extracted human body characteristic information is used for detecting abnormal behaviors of the old, and by analyzing the characteristics of the old such as gestures, movement tracks and the like, the abnormal behaviors such as falling, wandering, suddenly leaving and the like can be detected, so that emergency situations of the old can be found in time, and appropriate safety measures can be taken. Meanwhile, by extracting the human body characteristic information, the amount of image data to be processed can be reduced, so that the efficiency of image processing and analysis is improved, only the part related to the old is concerned, the consumption of computing resources and time can be reduced, and the system can respond and process the real-time video stream more quickly.
In one embodiment, referring to fig. 2, the step S2 includes:
s21: detecting each image according to a target detection algorithm, and outputting upper body position information of a human body;
specifically, downloading and preparing a YOLOv8s model and a corresponding weight file thereof, wherein the file contains pre-trained weight parameters and is used for a target detection task; loading the YOLOv8s model and its weights using an appropriate deep learning framework (e.g., pyTorch, tensorFlow); for N frames of images obtained by decomposition, sequentially processing each frame: and preprocessing each frame of image, such as image scaling, normalization, channel adjustment and the like, so as to adapt to the input requirement of the YOLOv8s model, and inputting the preprocessed image into the YOLOv8s model to perform target detection operation. The model extracts the corresponding upper body region according to the position information of the boundary frame of the pedestrian target and the position information of the boundary frame of the pedestrian target, wherein the specific region range of the upper body can be defined according to requirements, for example, the upper body image is saved from the head to the waist for subsequent use or display, and the next frame image is repeatedly processed until all N frame images are traversed.
S22: according to the upper body position information, extracting the characteristics of an upper body region of the human body by utilizing a target classification network, and outputting upper body characteristic information of the human body;
specifically, the ResNet model and the corresponding weight files thereof suitable for pedestrian feature extraction are downloaded and prepared, the files comprise pre-trained weight parameters for the feature extraction task, the ResNet model and the weights thereof are loaded by using a proper deep learning framework (such as PyTorch, tensorFlow), and the upper body image of the pedestrian obtained in the previous step is traversed: for each frame of the upper body image, the following processing is sequentially performed. Preprocessing operations, such as image scaling, normalization, channel adjustment, etc., are performed on each upper body image to accommodate the input requirements of the ResNet model. Inputting the preprocessed upper half body image into a ResNet model for feature extraction operation, wherein the ResNet model outputs feature vectors corresponding to the pedestrian upper half body image, stores the extracted pedestrian upper half body feature vectors in a matrix or vector form, can be used for subsequent tasks such as pedestrian analysis and anomaly detection, repeatedly processes the next frame upper half body image until all the pedestrian upper half body images are traversed, and can utilize the ResNet network for feature extraction of the pedestrian upper half body image to obtain the feature vectors with discriminant. The feature vectors can be used for further tasks such as pedestrian recognition, behavior analysis and anomaly detection, so that understanding and judging capability of pedestrian behaviors in the nursing scene of the aged are improved.
S23: inputting the upper body characteristic information into a pre-trained classifier, and outputting a classification result;
specifically, a trained SVM classifier for classifying the aged is obtained, relevant parameters and weights of the SVM classifier are saved, a machine learning library (such as scikit-learn) is used for loading a trained SVM classifier model and weights of the SVM classifier, the extracted pedestrian upper body characteristic vector is used as input data, the characteristic vector for classifying is prepared, necessary preprocessing, such as normalization, standardization or other conversion operation, is carried out on the characteristic vector according to the requirements of the SVM classifier, the preprocessed characteristic vector is input into the SVM classifier for classifying operation, the SVM outputs a classification label corresponding to each characteristic vector to indicate whether the pedestrian upper body belongs to the aged or not, and whether the pedestrian upper body in each image belongs to the aged or not is judged according to the classification result of the SVM. Further decisions may be made based on the confidence or probability of the classification labels, and based on the classification results, the determination of whether the upper body of the pedestrian in each image belongs to an elderly person may be output, for example, in the form of a binary form (elderly/non-elderly) or a probability value. And classifying the extracted upper body features of the pedestrians by using a trained SVM classifier, and judging whether the pedestrians belong to the old people or not. Therefore, the method can realize the identification and classification of the old and provide a basis for the subsequent analysis of the behaviors of the old, the detection of abnormality and the like.
S24: and acquiring the target image when the number of image frames classified as old people in each real-time image is greater than the preset number of image frames according to the classification result.
Specifically, for each frame of image, the classification result of the SVM classifier is recorded, the classification result is marked as old people or non-old people, the number of times of judging the old people in all image frames is calculated, namely, the number of the image frames judged as the old people is calculated, the threshold value of the image frames exceeding more than 2/3 is calculated according to the total frame number N, namely, N is multiplied by 2/3, and the relation between the judging times of the old people and the threshold value is compared. If the number of times of the old people judgment is greater than the threshold, namely, more than 2/3 of the image frames are judged to be old people, the pedestrian target frame is considered to be the old people, the corresponding target image is acquired, and the judgment result of whether the pedestrian target frame is the old people or not is output, and can be expressed in a binary form (old people/non-old people) or other forms. By comprehensively analyzing the classification result of the N frames of images, whether the pedestrian target frame is the old people or not is determined according to the condition that more than 2/3 of the image frames are judged to be the old people, the judgment accuracy and stability can be improved, and the accurate identification of the old people target can be ensured.
S3: performing motion analysis on each target image, and outputting motion characteristic information of a preset part of the old;
Specifically, motion analysis is carried out on each target image, and motion characteristic information of a preset part of the old is output; and extracting relevant motion characteristic information by analyzing the motion of the preset part of the old in the target image. The characteristic information can help understand the behavior mode of the old and monitor abnormal behaviors, so that the behavior monitoring and health condition assessment of the old are realized.
In one embodiment, referring to fig. 3, the step S3 includes:
s31: tracking the old people in the target image by utilizing a target matching algorithm;
specifically, an old man target in the current frame is detected by a target detection algorithm (such as YOLO, SSD, etc.), and the bounding box (bounding box) position information thereof is acquired. Performing IOU calculation on the boundary frames in the current frame and the boundary frames in the previous frame, measuring the overlapping degree of the boundary frames, performing target matching according to the IOU calculation result, adopting a maximum IOU matching strategy, namely matching the boundary frames in the current frame with the boundary frames with the maximum IOU in the previous frame, updating tracking information of the old people targets according to the target matching result, including boundary frame position, speed, motion state and the like, performing tracking state estimation and updating by using Kalman filtering, kalman-IOU and other methods, and repeatedly detecting the old people targets in the subsequent frame, performing IOU calculation and target matching, thereby realizing tracking updating of the old people targets in the target image. The method for performing the object tracking of the old by using the IOU has the advantages of simplicity, good real-time performance, high robustness, continuous tracking, object relevance and the like, and is suitable for the scenes of the old monitoring, behavior analysis, safety monitoring and the like.
S32: inputting the target image into a pre-trained target detection model, and outputting an image of a preset part of the old, wherein the image of the preset part comprises a hand image and a trunk image;
specifically, the images are processed by using a YoloV8s target detection algorithm to detect the hands and trunk of the old, wherein the hands refer to palms, the trunk refers to the body and limbs (except palms), head, hands and trunk areas of the old are extracted from the images according to the target detection result, corresponding image areas are intercepted by using the boundary frame information of the targets, and preset part images of the old are output, wherein the preset part images comprise the hand images and the trunk images; the YoloV8s algorithm has higher positioning and identification accuracy, can accurately detect the head, the hands and the body trunk of the old, provides reliable target position information, has higher processing speed and higher calculation efficiency, is suitable for real-time application scenes, and can rapidly detect different body parts of the old in real-time video streaming. By detecting the head, the hands and the body trunk, the characteristic information of different body parts of the old can be obtained, so that a more comprehensive and diversified analysis basis is provided.
S33: and carrying out optical flow field calculation on the trunk image and the hand image, and outputting an average optical flow field of the trunk image and a center optical flow field of the hand image as the motion characteristic information.
Specifically, an optical flow vector of each pixel point in the trunk image is calculated by using an optical flow algorithm (for example, lucas-Kanade algorithm, farnesback algorithm, etc.), the optical flow vector represents the motion direction and speed of the pixel point between adjacent frames, and an average operation is performed on the optical flow vectors of all the pixel points in the trunk image to obtain an average optical flow value of the trunk image. This can be achieved by accumulating the x and y components of all the optical flow vectors, divided by the number of pixels; and extracting optical flow vectors of the region according to the position of the central point region of the hand image. The optical flow vector in the region can be extracted with a certain region range as a radius by selecting the hand center point as the center. By calculating the average luminous flux value of the trunk image, the motion information of the whole trunk can be obtained. This helps to determine the overall movement state and movement pattern of the elderly, such as whether abnormal movement or wandering behavior is occurring. The old people usually move slowly because of aging and hypofunction of the body, and because the optical flow method is a method based on pixel displacement calculation, the method is very sensitive to slow movement, and even if the old people act slowly, the optical flow method can effectively capture the tiny displacements, so that the extraction of the movement characteristics of the old people is realized; the movement of the old is often slow, and in many cases, real-time monitoring is needed to prevent potential accidents, and the optical flow method has the capability of processing images in real time, can extract movement characteristics in video stream in real time, and feeds back the action state of the old in time so as to take necessary measures as soon as possible. Although the movement of the old is slower, the displacement calculation precision of the optical flow method is higher, and the micro displacement between pixels can be accurately captured, so that the optical flow method is excellent in tracking the movement track and the movement change of the old, is beneficial to providing more accurate behavior monitoring and analysis results, is an image processing-based technology, does not need to carry out additional sensor wearing or intervention on the old, and is more easily accepted and applied in the behavior monitoring of the old without discomfort or trouble; by calculating the optical flow velocity field in the center point area of the hand image, specific features of hand motion can be obtained. Hand limb motion may not coincide with the motion of the hand region, so by analyzing the optical flow velocity field in the hand center point region, hand motion information can be captured more accurately.
S4: detecting the preset abnormal behavior of the old according to the movement characteristic information, and outputting a detection result;
specifically, detecting preset abnormal behaviors of the old according to the movement characteristic information, and outputting a detection result; by detecting the preset abnormal behaviors of the old, possible problems or emergency situations can be found in advance. Timely early warning can prompt relevant personnel to take appropriate measures so as to ensure the safety and health of the old people.
In one embodiment, referring to fig. 4, the step S4 includes:
s41: acquiring preset abnormal behaviors of the old, wherein the abnormal behaviors of the old at least comprise one of the following behaviors: abnormal loitering of the old, incapacity of standing of the old and trembling of the hands of the old;
specifically, the elderly may be inattentive due to reduced cognitive function, experience abnormal wandering behavior, forget their own destination or get lost, such as cognitive impairment: cognitive disorder diseases such as senile dementia may cause the elderly to become lost and wander because they have difficulty remembering the environment and direction; depression or anxiety: mood problems may lead to the elderly wandering intentionally and unintentionally seeking comfort or relief; living environment changes: elderly people may become lost and wander due to changing residence or reduced personals; elderly people may also be unable to stand due to problems with bone diseases (such as fractures), muscle atrophy or arthritis; neurological diseases such as stroke, parkinson's disease, etc. may affect balance and standing ability of the elderly; among them, one of the main symptoms of parkinson's disease is trembling of the hands, which may cause difficulty in controlling the hand movements of the elderly. Firstly, presetting a group of definition and rules of the abnormal behavior of the old according to specific application requirements and the abnormal behavior of the concerned old, for example, setting abnormal loitering of the old, incapability of standing of the old and tremble of hands of the old aiming at the problems; specifically, according to actual demands, preset abnormal behaviors of the old can be flexibly defined and adjusted, customization is performed according to specific scenes and characteristics of the old, and detection accuracy and adaptability are improved.
S42: acquiring a preset time interval, and identifying abnormal loitering behaviors of the old people when the trunk of the old people is judged to have repeated movement according to the variance of the average light flow field in the time interval;
in one embodiment, referring to fig. 5, the step S42 includes:
s421: acquiring the time interval and a preset first optical flow field threshold value;
s422: calculating the average optical flow field in the time interval, and outputting the variance of the average optical flow field;
s423: and identifying the abnormal loitering behavior of the old person when the variance is greater than the first optical flow field threshold.
Specifically, the optical flow field L1 is used to represent an average optical flow field of the trunk image within a preset time interval, the trunk part of the body comprises a majority of cores and balance supporting areas, in the abnormal behavior monitoring of the elderly, the motion characteristics of the trunk of the body are important for preventing falling and evaluating the stability of the posture, the motion of the trunk part of the body is more integrated, the average optical flow field can capture the overall motion trend and direction, and the method helps to judge whether the motion of the elderly is stable in one direction. If the variance of L1 exceeds a set threshold within a time window, we can consider that there may be wandering behavior: defining the optical flow field of the trunk part of the body as L1 (t, t+Δt) in the time t to t+Δt), setting a first optical flow field threshold Th1, and judging that abnormal loitering behaviors of the old exist if Var [ L1 (t, t+Δt) ] > Th 1; wherein Var [ ] represents variance, t represents current time, Δt represents a set time window, determined depending on experiments, th1 is a set first optical flow field threshold, and if the change of body position exceeds the threshold in Δt time, then the abnormal loitering behavior of the elderly is considered to exist; the variance of the optical flow field is calculated based on the real-time video flow, so that the behavior of the old can be monitored in real time, the behavior of the old can be continuously tracked and monitored by continuously calculating the variance of the optical flow field, abnormal wandering behavior can be found timely, meanwhile, the degree of body position change can be judged according to the variance of the optical flow field by setting a proper first optical flow field threshold, when the variance exceeds the threshold, abnormal wandering behavior of the old can be accurately identified, the detection accuracy and sensitivity are improved, the size of the threshold and the time window of the first optical flow field can be specifically set to be adjusted according to actual conditions, and different threshold values and time window settings can be provided for different old, different environments and different demands, so that the adaptability and adjustability of the algorithm can be improved.
S43: identifying the standing incapacity of the old people according to the first average value of the average light flow field and the second average value of the central light flow field in the time interval;
in one embodiment, referring to fig. 6, the step S43 includes:
s431: acquiring a preset second optical flow field threshold value and a preset third optical flow field threshold value;
s432: calculating the average optical flow field and the central optical flow field in the time interval, and outputting the first average value and the second average value;
s433: when the first average value is less than the second optical-flow field threshold value and the second average value is greater than the third optical-flow field threshold value, the elderly person is identified as not standing.
Specifically, the central optical flow field L2 is used for representing the central optical flow field of the hand image, the hand is one of the most flexible and fine parts of the human body, various fine actions such as pinching, holding, writing and the like are carried out in the daily life of the old, and the central optical flow field can be used for more sensitively detecting the fine movements of pixels around the central point of the hand, so that the capturing of the fine action characteristics of the hand is facilitated; in the behavior monitoring of the elderly, there may be integral camera or image movements, such as lens movements or camera shake, and the use of a central optical flow field may partially remove the effects of these integral movements, focusing on the fine movements of the hand image itself, reducing unnecessary interference. The inability of the elderly to stand means that his torso should remain in a relatively small range of motion, i.e., L1 will be small, while he may try to support it by hand, so L2 will become large, judging that the elderly is unable to stand when Mean [ L1 (t, t+Δt) ] < Th2 and Mean [ L2 (t, t+Δt) ] > Th3, where Mean represents averaging, t is the current time, Δt is the set time window, th2 and Th3 are the set second and third optical flow field thresholds. The movement characteristics of the hand image are analyzed by utilizing the optical flow field, so that the behavior of the old which cannot stand can be more accurately identified, the old can be distinguished from other abnormal behaviors, and the detection accuracy and reliability are improved; by comprehensively considering the two directions (L1 and L2) of the optical flow field, the characteristic that the old cannot stand can be better captured, and misjudgment caused by only relying on a single characteristic is avoided. By setting the second optical flow field threshold and the third optical flow field threshold appropriately, the behavior can be flexibly adjusted according to the actual situation, and the adaptability and the adjustability of the algorithm are improved.
S44: and identifying the tremble behavior of the hands of the old people when judging that the hands of the old people have weak motions according to the central light flow field in the time interval.
In one embodiment, referring to fig. 7, the step S44 includes:
s441: acquiring a preset fourth optical flow field threshold value;
s442: performing Fourier transform on the central optical flow field in the time interval, and outputting frequency components corresponding to the central optical flow field;
s443: comparing the frequency components, and identifying the hand tremor behavior when the maximum value of the frequency components is greater than the fourth optical flow field threshold.
Specifically, the hand tremor of the elderly refers to the abnormality of the hands of the elderly, the hands of the elderly tremor involuntarily, and the behavior of the hands may cause the L2 to change frequently within a small range: setting a time interval, and counting an optical flow field of a hand in the time interval; calculating the fast Fourier transform of the hand optical flow; when Max [ Freq (L2 (t, t+Δt)) ] > Th4, a possible hand shake behavior is judged; where Freq represents a frequency component after the fast fourier transform, t is a current time, Δt is a set time window, th4 is a set fourth optical flow field threshold, and Max represents a maximum value. The hand trembling behavior is judged by utilizing the change of the optical flow field and the frequency component, the abnormal condition of the hand of the old can be accurately identified, and the possible health problem can be found in time; frequency information in the hand optical flow field can be extracted by performing fast Fourier transform, so that hand trembling behaviors are distinguished from other motion behaviors, and the accuracy and reliability of detection are improved; by setting the appropriate fourth optical flow field threshold Th4, the tremble behavior of the hand can be judged according to the actual situation, and the situation of misjudgment or missing judgment is avoided.
S5: and according to the detection result, when the abnormal behavior of the old is identified, a safety prompt is sent to the user.
Specifically, when abnormal behaviors of the old are identified, the system can send out a safety reminder to the user in a proper mode. For example, a notification is sent to a user by means of a mobile application program, a short message, a telephone or an alarm system, etc., so that the user is reminded of the condition of the old, the behavior of the old is monitored in real time, and a safety reminder is sent immediately when abnormal behavior is identified, so that the user can take measures in time to treat possible emergency; by sending the safety reminding to the user, the nursing safety of the old can be enhanced, the user is helped to pay attention to the health and safety conditions of the old in time, and potential risks and accidents are avoided.
Example 2
Referring to fig. 8, embodiment 2 of the present invention further provides a device for detecting abnormal behavior of the elderly in real time, where the device includes:
the real-time image acquisition module is used for acquiring a real-time video stream in an aged care scene and decomposing the real-time video stream into multi-frame real-time images;
the old man identification module is used for extracting the characteristics of each real-time image and acquiring a target image containing the old man according to the extracted human body characteristic information;
In one embodiment, the senior citizen identification module comprises:
the upper body position acquisition unit is used for detecting each image according to a target detection algorithm and outputting upper body position information of a human body;
the upper body characteristic extraction unit is used for extracting the characteristics of the upper body region of the human body by utilizing the target classification network according to the upper body position information and outputting the upper body characteristic information of the human body;
the upper body characteristic classifying unit is used for inputting the upper body characteristic information into a pre-trained classifier and outputting a classifying result;
and the target image acquisition unit is used for acquiring the target image when the number of image frames classified as the old people in each real-time image is larger than the preset number of image frames according to the classification result.
The motion characteristic extraction module is used for performing motion analysis on each target image and outputting motion characteristic information of a preset part of the old;
in an embodiment, the motion feature extraction module comprises:
the old man tracking unit is used for tracking the old man in the target image by utilizing a target matching algorithm;
the old man preset part image acquisition unit is used for inputting the target image into a pre-trained target detection model and outputting an old man preset part image, wherein the preset part image comprises a hand part image and a trunk part image;
And the optical flow field calculation unit is used for calculating the optical flow field of the trunk image and the hand image and outputting the average optical flow field of the trunk image and the central optical flow field of the hand image as the motion characteristic information.
The old man abnormal behavior detection module is used for detecting preset old man abnormal behaviors according to the motion characteristic information and outputting detection results;
in one embodiment, the senior citizen abnormal behavior detection module comprises:
the device comprises an aged abnormal behavior acquisition unit, a control unit and a control unit, wherein the aged abnormal behavior acquisition unit is used for acquiring preset aged abnormal behaviors, and the aged abnormal behaviors at least comprise one of the following behaviors: abnormal loitering of the old, incapacity of standing of the old and trembling of the hands of the old;
the old man abnormal loitering identification unit is used for acquiring a preset time interval, and identifying abnormal loitering behavior of the old man when the trunk of the old man is judged to have repeated movement according to the variance of the average light flow field in the time interval;
a first optical flow field threshold value acquisition subunit, configured to acquire the time interval and a preset first optical flow field threshold value;
the variance calculating subunit is used for calculating the average optical flow field in the time interval and outputting the variance of the average optical flow field;
And the old man abnormal loitering detection subunit is used for identifying the old man abnormal loitering behavior when the variance is larger than the first optical flow field threshold value.
The old man unable standing identification unit is used for identifying the old man unable standing behavior according to the first average value of the average light flow field and the second average value of the central light flow field in the time interval;
in an embodiment, the elderly people non-standing identification unit comprises:
a second optical flow field threshold value and a third optical flow field threshold value acquisition subunit configured to acquire a preset second optical flow field threshold value and third optical flow field threshold value;
a first average value and a second average value calculating subunit, configured to calculate the average optical flow field and the central optical flow field in the time interval, and output the first average value and the second average value;
and the old man can not stand, and is used for identifying the old man to stand when the first average value is smaller than the second optical flow field threshold value and the second average value is larger than the third optical flow field threshold value.
And the old man hand tremble identification unit is used for identifying the old man hand tremble behavior when judging that weak movement exists in the old man hand according to the central light flow field in the time interval.
In an embodiment, the old man's hand tremor recognition unit includes:
a fourth optical flow field threshold value acquisition subunit, configured to acquire a preset fourth optical flow field threshold value;
a frequency component calculating subunit, configured to perform fourier transform on the central optical flow field in the time interval, and output a frequency component corresponding to the central optical flow field;
and the hand tremble behavior detection subunit is used for comparing the frequency components, and identifying the hand tremble behavior when the maximum value of the frequency components is greater than the fourth optical flow field threshold value.
And the safety reminding module is used for sending a safety reminding to a user when the detection result is identified as the abnormal behavior of the old people.
Specifically, the device for detecting abnormal behaviors of the aged provided in embodiment 2 of the present invention includes: the real-time image acquisition module is used for acquiring a real-time video stream in an aged care scene and decomposing the real-time video stream into multi-frame real-time images; the old man identification module is used for extracting the characteristics of each real-time image and acquiring a target image containing the old man according to the extracted human body characteristic information; the motion characteristic extraction module is used for performing motion analysis on each target image and outputting motion characteristic information of a preset part of the old; the old man abnormal behavior detection module is used for detecting preset old man abnormal behaviors according to the motion characteristic information and outputting detection results; and the safety reminding module is used for sending a safety reminding to a user when the detection result is identified as the abnormal behavior of the old people. According to the device, through analyzing the real-time video stream and detecting the abnormal behaviors, the motion characteristic information and the abnormal behaviors of the old are analyzed, the individualized knowledge of the old is obtained, so that the abnormal behaviors of the old are accurately detected, once the abnormal behaviors of the old are detected, the system can send safety reminding to a user, timely safety reminding and intervention measures are helpful for reducing the threat of accidents and emergency to the health and safety of the old, and the nursing experience of the old of the user is improved.
Example 3
In addition, the method for detecting the abnormal behavior of the elderly people in real time according to embodiment 1 of the present invention described in connection with fig. 1 may be implemented by an electronic device. Fig. 9 shows a schematic hardware structure of an electronic device according to embodiment 3 of the present invention.
The electronic device may include a processor and memory storing computer program instructions.
In particular, the processor may comprise a Central Processing Unit (CPU), or an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or may be configured as one or more integrated circuits that implement embodiments of the present invention.
The memory may include mass storage for data or instructions. By way of example, and not limitation, the memory may comprise a Hard Disk Drive (HDD), floppy Disk Drive, flash memory, optical Disk, magneto-optical Disk, magnetic tape, or universal serial bus (Universal Serial Bus, USB) Drive, or a combination of two or more of the foregoing. The memory may include removable or non-removable (or fixed) media, where appropriate. The memory may be internal or external to the data processing apparatus, where appropriate. In a particular embodiment, the memory is a non-volatile solid state memory. In a particular embodiment, the memory includes Read Only Memory (ROM). The ROM may be mask programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically Erasable PROM (EEPROM), electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these, where appropriate.
The processor reads and executes the computer program instructions stored in the memory to implement any one of the methods for detecting abnormal behaviors of the elderly in real time in the above embodiments.
In one example, the electronic device may also include a communication interface and a bus. The processor, the memory, and the communication interface are connected by a bus and complete communication with each other, as shown in fig. 9.
The communication interface is mainly used for realizing communication among the modules, the devices, the units and/or the equipment in the embodiment of the invention.
The bus includes hardware, software, or both that couple the components of the device to each other. By way of example, and not limitation, the buses may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an infiniband interconnect, a Low Pin Count (LPC) bus, a memory bus, a micro channel architecture (MCa) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a video electronics standards association local (VLB) bus, or other suitable bus, or a combination of two or more of the above. The bus may include one or more buses, where appropriate. Although embodiments of the invention have been described and illustrated with respect to a particular bus, the invention contemplates any suitable bus or interconnect.
Example 4
In addition, in combination with the method for detecting abnormal behaviors of the elderly people in embodiment 1, embodiment 4 of the present invention may also be implemented by providing a computer readable storage medium. The computer readable storage medium has stored thereon computer program instructions; the computer program instructions, when executed by the processor, implement any of the methods for real-time detection of abnormal behavior of the elderly in the above embodiments.
In summary, the embodiment of the invention provides a method, a device, equipment and a storage medium for detecting abnormal behaviors of old people in real time.
It should be understood that the invention is not limited to the particular arrangements and instrumentality described above and shown in the drawings. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present invention are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after appreciating the spirit of the present invention.
The functional blocks shown in the above-described structural block diagrams may be implemented in hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the elements of the invention are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine readable medium or transmitted over transmission media or communication links by a data signal carried in a carrier wave. A "machine-readable medium" may include any medium that can store or transfer information. Examples of machine-readable media include electronic circuitry, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio Frequency (RF) links, and the like. The code segments may be downloaded via computer networks such as the internet, intranets, etc.
It should also be noted that the exemplary embodiments mentioned in this disclosure describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above-described steps, that is, the steps may be performed in the order mentioned in the embodiments, or may be performed in a different order from the order in the embodiments, or several steps may be performed simultaneously.
In the foregoing, only the specific embodiments of the present invention are described, and it will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the systems, modules and units described above may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein. It should be understood that the scope of the present invention is not limited thereto, and any equivalent modifications or substitutions can be easily made by those skilled in the art within the technical scope of the present invention, and they should be included in the scope of the present invention.
Claims (10)
1. The method for detecting the abnormal behavior of the old in real time is characterized by comprising the following steps:
s1: acquiring a real-time video stream under a nursing scene of the aged, and decomposing the real-time video stream into multi-frame real-time images;
S2: extracting the characteristics of each real-time image, and acquiring a target image containing the old people according to the extracted human body characteristic information;
s3: performing motion analysis on each target image, and outputting motion characteristic information of a preset part of the old;
s4: detecting the preset abnormal behavior of the old according to the movement characteristic information, and outputting a detection result;
s5: and according to the detection result, when the abnormal behavior of the old is identified, a safety prompt is sent to the user.
2. The method for detecting abnormal behavior of the elderly according to claim 1, wherein the S2 comprises:
s21: detecting each image according to a target detection algorithm, and outputting upper body position information of a human body;
s22: according to the upper body position information, extracting the characteristics of an upper body region of the human body by utilizing a target classification network, and outputting upper body characteristic information of the human body;
s23: inputting the upper body characteristic information into a pre-trained classifier, and outputting a classification result;
s24: and acquiring the target image when the number of image frames classified as old people in each real-time image is greater than the preset number of image frames according to the classification result.
3. The method for detecting abnormal behavior of the elderly according to claim 1, wherein the step S3 comprises:
s31: tracking the old people in the target image by utilizing a target matching algorithm;
s32: inputting the target image into a pre-trained target detection model, and outputting an image of a preset part of the old, wherein the image of the preset part comprises a hand image and a trunk image;
s33: and carrying out optical flow field calculation on the trunk image and the hand image, and outputting an average optical flow field of the trunk image and a center optical flow field of the hand image as the motion characteristic information.
4. The method for detecting abnormal behavior of the elderly according to claim 3, wherein the S4 comprises:
s41: acquiring preset abnormal behaviors of the old, wherein the abnormal behaviors of the old at least comprise one of the following behaviors: abnormal loitering of the old, incapacity of standing of the old and trembling of the hands of the old;
s42: acquiring a preset time interval, and identifying abnormal loitering behaviors of the old people when the trunk of the old people is judged to have repeated movement according to the variance of the average light flow field in the time interval;
s43: identifying the standing incapacity of the old people according to the first average value of the average light flow field and the second average value of the central light flow field in the time interval;
S44: and identifying the tremble behavior of the hands of the old people when judging that the hands of the old people have weak motions according to the central light flow field in the time interval.
5. The method for detecting abnormal behavior of elderly people according to claim 4, wherein said S42 comprises:
s421: acquiring the time interval and a preset first optical flow field threshold value;
s422: calculating the average optical flow field in the time interval, and outputting the variance of the average optical flow field;
s423: and identifying the abnormal loitering behavior of the old person when the variance is greater than the first optical flow field threshold.
6. The method for detecting abnormal behavior of elderly people according to claim 4, wherein said S43 comprises:
s431: acquiring a preset second optical flow field threshold value and a preset third optical flow field threshold value;
s432: calculating the average optical flow field and the central optical flow field in the time interval, and outputting the first average value and the second average value;
s433: when the first average value is less than the second optical-flow field threshold value and the second average value is greater than the third optical-flow field threshold value, the elderly person is identified as not standing.
7. The method for detecting abnormal behavior of elderly people according to claim 4, wherein said S44 comprises:
S441: acquiring a preset fourth optical flow field threshold value;
s442: performing Fourier transform on the central optical flow field in the time interval, and outputting frequency components corresponding to the central optical flow field;
s443: comparing the frequency components, and identifying the hand tremor behavior when the maximum value of the frequency components is greater than the fourth optical flow field threshold.
8. An apparatus for real-time detection of abnormal behavior of an elderly person, the apparatus comprising:
the real-time image acquisition module is used for acquiring a real-time video stream in an aged care scene and decomposing the real-time video stream into multi-frame real-time images;
the old man identification module is used for extracting the characteristics of each real-time image and acquiring a target image containing the old man according to the extracted human body characteristic information;
the motion characteristic extraction module is used for performing motion analysis on each target image and outputting motion characteristic information of a preset part of the old;
the old man abnormal behavior detection module is used for detecting preset old man abnormal behaviors according to the motion characteristic information and outputting detection results;
and the safety reminding module is used for sending a safety reminding to a user when the detection result is identified as the abnormal behavior of the old people.
9. An electronic device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, which when executed by the processor, implement the method of any one of claims 1-7.
10. A storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310923016.9A CN116883946B (en) | 2023-07-24 | 2023-07-24 | Method, device, equipment and storage medium for detecting abnormal behaviors of old people in real time |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310923016.9A CN116883946B (en) | 2023-07-24 | 2023-07-24 | Method, device, equipment and storage medium for detecting abnormal behaviors of old people in real time |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116883946A true CN116883946A (en) | 2023-10-13 |
| CN116883946B CN116883946B (en) | 2024-03-22 |
Family
ID=88260289
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310923016.9A Active CN116883946B (en) | 2023-07-24 | 2023-07-24 | Method, device, equipment and storage medium for detecting abnormal behaviors of old people in real time |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116883946B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2014026566A (en) * | 2012-07-30 | 2014-02-06 | Pioneer Electronic Corp | Gesture detection device, gesture detection method, and program |
| WO2019157344A1 (en) * | 2018-02-12 | 2019-08-15 | Avodah Labs, Inc. | Real-time gesture recognition method and apparatus |
| CN112232190A (en) * | 2020-10-15 | 2021-01-15 | 南京邮电大学 | Method for detecting abnormal behaviors of old people facing home scene |
| CN112818807A (en) * | 2021-01-27 | 2021-05-18 | 深圳英飞拓科技股份有限公司 | Tumble detection method, tumble detection device, tumble detection apparatus, and storage medium |
| CN114495150A (en) * | 2021-12-06 | 2022-05-13 | 南京信息工程大学 | Human body tumbling detection method and system based on time sequence characteristics |
| CN114469076A (en) * | 2022-01-24 | 2022-05-13 | 南京邮电大学 | Identity feature fused old solitary people falling identification method and system |
| CN114842397A (en) * | 2022-05-19 | 2022-08-02 | 华南农业大学 | Real-time old man falling detection method based on anomaly detection |
| CN115116133A (en) * | 2022-06-14 | 2022-09-27 | 鹏城实验室 | Abnormal behavior detection system and method for monitoring solitary old people |
| CN115719347A (en) * | 2022-11-28 | 2023-02-28 | 北汽福田汽车股份有限公司 | Behavior recognition method and device, electronic equipment and vehicle |
-
2023
- 2023-07-24 CN CN202310923016.9A patent/CN116883946B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2014026566A (en) * | 2012-07-30 | 2014-02-06 | Pioneer Electronic Corp | Gesture detection device, gesture detection method, and program |
| WO2019157344A1 (en) * | 2018-02-12 | 2019-08-15 | Avodah Labs, Inc. | Real-time gesture recognition method and apparatus |
| CN112232190A (en) * | 2020-10-15 | 2021-01-15 | 南京邮电大学 | Method for detecting abnormal behaviors of old people facing home scene |
| CN112818807A (en) * | 2021-01-27 | 2021-05-18 | 深圳英飞拓科技股份有限公司 | Tumble detection method, tumble detection device, tumble detection apparatus, and storage medium |
| CN114495150A (en) * | 2021-12-06 | 2022-05-13 | 南京信息工程大学 | Human body tumbling detection method and system based on time sequence characteristics |
| CN114469076A (en) * | 2022-01-24 | 2022-05-13 | 南京邮电大学 | Identity feature fused old solitary people falling identification method and system |
| CN114842397A (en) * | 2022-05-19 | 2022-08-02 | 华南农业大学 | Real-time old man falling detection method based on anomaly detection |
| CN115116133A (en) * | 2022-06-14 | 2022-09-27 | 鹏城实验室 | Abnormal behavior detection system and method for monitoring solitary old people |
| CN115719347A (en) * | 2022-11-28 | 2023-02-28 | 北汽福田汽车股份有限公司 | Behavior recognition method and device, electronic equipment and vehicle |
Non-Patent Citations (1)
| Title |
|---|
| 褚少微 等: "自拍相机运动手势交互界面的设计与实现", 计算机工程与应用, no. 10, pages 181 - 186 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116883946B (en) | 2024-03-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Liu et al. | A fall detection system using k-nearest neighbor classifier | |
| CN108734055B (en) | Abnormal person detection method, device and system | |
| US9036902B2 (en) | Detector for chemical, biological and/or radiological attacks | |
| Tzeng et al. | Design of fall detection system with floor pressure and infrared image | |
| US20140307076A1 (en) | Systems and methods for monitoring personal protection equipment and promoting worker safety | |
| CN202257856U (en) | Driver fatigue-driving monitoring device | |
| JP6655727B2 (en) | Monitoring system | |
| KR101309366B1 (en) | System and Method for Monitoring Emergency Motion based Image | |
| CN111275910B (en) | Method and system for detecting border crossing behavior of escalator based on Gaussian mixture model | |
| CN109614906A (en) | A kind of security system and security alarm method based on deep learning | |
| CN111616718B (en) | Method and system for detecting fatigue state of driver based on attitude characteristics | |
| CN106295474B (en) | Fatigue detection method, system and the server of deck officer | |
| CN109800715A (en) | A kind of discrepancy garden monitoring method and system based on Internet of Things | |
| CN115797856A (en) | Intelligent construction scene safety monitoring method based on machine vision | |
| CN111387953A (en) | Intelligent body temperature detection system and method | |
| CN103152558A (en) | Intrusion detection method based on scene recognition | |
| CN116883946B (en) | Method, device, equipment and storage medium for detecting abnormal behaviors of old people in real time | |
| Merrouche et al. | Fall detection using head tracking and centroid movement based on a depth camera | |
| Patel et al. | Deep Learning based Driver Drowsiness Detection | |
| CN112070185A (en) | Re-ID-based non-contact fever person tracking system and tracking method thereof | |
| Lee et al. | Automated abnormal behavior detection for ubiquitous healthcare application in daytime and nighttime | |
| CN112562260B (en) | Anti-lost method and device | |
| CN114469076B (en) | Identity-feature-fused fall identification method and system for solitary old people | |
| CN112084899A (en) | Fall event detection method and system based on deep learning | |
| CN114469076A (en) | Identity feature fused old solitary people falling identification method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |