CN117860242B - Infant walking action development detection method, equipment and device - Google Patents

Infant walking action development detection method, equipment and device Download PDF

Info

Publication number
CN117860242B
CN117860242B CN202410282326.1A CN202410282326A CN117860242B CN 117860242 B CN117860242 B CN 117860242B CN 202410282326 A CN202410282326 A CN 202410282326A CN 117860242 B CN117860242 B CN 117860242B
Authority
CN
China
Prior art keywords
infant
video
walking action
frame
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410282326.1A
Other languages
Chinese (zh)
Other versions
CN117860242A (en
Inventor
宋国超
常佳慧
关宏岩
张霆
张永明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Capital Institute of Pediatrics
Original Assignee
Capital Institute of Pediatrics
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Capital Institute of Pediatrics filed Critical Capital Institute of Pediatrics
Priority to CN202410282326.1A priority Critical patent/CN117860242B/en
Publication of CN117860242A publication Critical patent/CN117860242A/en
Application granted granted Critical
Publication of CN117860242B publication Critical patent/CN117860242B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0033—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1116—Determining posture transitions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/23—Recognition of whole body movements, e.g. for sport training
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2503/00—Evaluating a particular growth phase or type of persons or animals
    • A61B2503/04—Babies, e.g. for SIDS detection
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2576/00—Medical imaging apparatus involving image processing or analysis

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Veterinary Medicine (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Pathology (AREA)
  • Public Health (AREA)
  • Biophysics (AREA)
  • Dentistry (AREA)
  • Multimedia (AREA)
  • Physiology (AREA)
  • Theoretical Computer Science (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Physics & Mathematics (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

The invention discloses a method, equipment and a device for detecting the walking action development of an infant, wherein the method comprises the following steps: the acquisition device acquires each infant walking action video to be detected; the uploading device uploads the video to the server; the server designates a cutting platform of each infant walking action video to be detected; dividing the appointed action video by each platform in the distributed cutting platform cluster according to the target walking action type of the baby to obtain a plurality of sub-videos of the target walking action type of each baby to be detected; the detection device determines key points of infant bones in each frame from the sub-video, and determines an angle between the midpoint of the two shoulder joints and the connecting line of the midpoint of the two hip joints and the vertical axis of the video picture; determining a coefficient of variation for the angle based on the angle; and determining the posture stability of each infant to be detected when moving according to the variation coefficient. The invention can efficiently and accurately evaluate the stability of the walking action posture of the infant, and further efficiently and accurately detect the growth condition of the walking action of the infant.

Description

Infant walking action development detection method, equipment and device
Technical Field
The invention relates to the technical field of medical health care informatics, in particular to a method, equipment and a device for detecting the walking action development of an infant.
Background
This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
The research on the action development of infants is in compliance with the great demands of society on infant health care. The assessment of infant motor development is one of important infant health care content, and the assessment result of infant motor development can be used for assessing the basis of the assessment of infant brain development, for example, whether an infant has cerebral palsy or not is judged through the assessment result of infant motor development. Infant walking is an important milestone in the development of infant motor skills. While observational studies indicate that there may be significant functional breaks in a temporally continuous and structurally similar developmental milestone, the development of walking skills from resting support to independent walking is believed to be closely related to an increase in torso stability. The development of walking skills is accompanied by gradual stabilization of the trunk and gradual fluency of the four limbs, thus providing an important foundation for the infants to walk independently.
In recent years, quantitative studies in the field of kinematics have been very well developed. Currently, kinematic evaluation (detection) methods include observational analysis, force plates and pressure sensors, motion capture systems, surface electromyography (sEMG), and the like. The observation and analysis are mainly carried out through a motion test and various measuring meters, the motion track projected on the ground can be monitored through pressure center monitoring, the motion capture system has good precision in a three-dimensional space, sEMG is most valuable for monitoring the muscle activity of the trunk, and the swinging instrument utilizes an accelerometer technology to evaluate the swinging condition of the trunk. Many methods are difficult to apply during infancy including data acquisition, processing and analysis. And it is difficult to measure on an infant with any personal wear item. The main disadvantage of the observation analysis by current clinical tests, such as torso control gauges, is the reliance on subjective assessment of balance control by the inspector. Center of pressure monitoring does not measure the trajectory of motion in a plane other than the ground and is limited to activity without movement when torso stability is analyzed. As for most motion capture systems, it is desirable to use items that are attached to the body, including sensors or retroreflective markers, etc. sEMG requires attaching electrode pads to the body for monitoring trunk muscle activity. The swing is bulky and is overly complex for the infant to wear. Therefore, how to accurately quantify the kinematic index in infant movement (crawling, walking and walking) without using a wearable device is a problem to be solved.
As is clear from the above, there are few methods for detecting the motion and posture stability index of the infant during the movement, and there are observation and analysis, pressure sensors, motion capture systems, sEMG, and the like. The observational analysis is greatly subjectively affected by staff, and other methods require wearing personal care items on infants. The analysis results are subjectively influenced by staff, or close-fitting articles are needed to be worn on the baby, and the precision and the efficiency of the baby walking action development detection are low.
Disclosure of Invention
The embodiment of the invention provides a method for detecting the development of infant walking exercise, which is used for efficiently and accurately evaluating the stability of the posture of the infant walking exercise, and is applied to detection system equipment, and comprises the following steps:
The acquisition device acquires each infant walking action video to be detected;
The uploading device uploads each infant walking action video to be detected to the server;
the server stores the walking action video of each infant to be detected in real time, and designates a cutting platform of the walking action video of each infant to be detected;
Dividing a specified infant walking action video to be detected by each cutting platform in the distributed cutting platform cluster according to the target walking action type of the infant to be detected, and obtaining a plurality of sub-videos of the target walking action types of each infant to be detected;
The detection device downloads each target walking action type sub-video, determines infant skeleton key points in each frame of image from each target walking action type sub-video, and determines angles between the connecting lines of the middle points of two shoulder joints and the middle points of two hip joints in each frame of image in each target walking action type sub-video and the vertical axis of the video picture; determining a variation coefficient of the angle between the connecting line and the vertical axis of the video picture according to the angle; and determining the posture stability of each infant to be detected when moving according to the variation coefficient so as to determine the infant walking action development detection result.
The embodiment of the invention provides a method for detecting the development of infant walking exercise, which is used for efficiently and accurately evaluating the stability of the posture of the infant walking exercise, and is applied to a detection device, and comprises the following steps:
Acquiring a sub-video of each target walking action type; dividing a specified infant walking action video to be detected by each cutting platform in the distributed cutting platform cluster according to the target walking action type of the infant to be detected;
Determining infant skeleton key points in each frame of image from each target walking action type sub-video;
Determining an angle between a connecting line of a midpoint of two shoulder joints and a midpoint of two hip joints in key points of infant bones in each frame of image in each target walking action type sub-video and a vertical axis of a video picture;
determining a variation coefficient of an angle between a connecting line of a midpoint of two shoulder joints and a midpoint of two hip joints in key points of infant bones in each frame of image in each target walking action type sub-video and a vertical axis of a video picture according to the angle between the connecting line and the vertical axis of the video picture;
and determining the posture stability of each infant to be detected when moving according to the variation coefficient so as to determine the infant walking action development detection result.
The embodiment of the invention also provides a device of the infant walking exercise development detection system, which is used for efficiently and accurately evaluating the stability of the infant walking exercise posture, and comprises the following components:
The acquisition device is used for acquiring each infant walking action video to be detected;
The uploading device is connected with the acquisition device and used for uploading the walking action video of each infant to be detected to the server;
the server is connected with the uploading device and used for storing each infant walking action video to be detected in real time and designating a cutting platform of each infant walking action video to be detected;
the distributed cutting platform clusters, wherein each cutting platform in the clusters is used for dividing the designated infant walking action video to be detected according to the target walking action type of the infant to be detected to obtain a plurality of sub-videos of the target walking action types of each infant to be detected;
The detection device is used for downloading each target walking action type sub-video, determining infant skeleton key points in each frame of image from each target walking action type sub-video, and determining angles between the midpoint of two shoulder joints and the connecting line of the midpoint of two hip joints in each frame of image in each target walking action type sub-video and the vertical axis of the video picture; determining a variation coefficient of the angle between the connecting line and the vertical axis of the video picture according to the angle; and determining the posture stability of each infant to be detected when moving according to the variation coefficient so as to determine the infant walking action development detection result.
The embodiment of the invention also provides a device for detecting the infant walking action development, which is used for efficiently and accurately evaluating the stability of the infant walking action posture, and comprises the following components:
the downloading unit is used for acquiring each target walking action type sub-video; dividing a specified infant walking action video to be detected by each cutting platform in the distributed cutting platform cluster according to the target walking action type of the infant to be detected;
the determining unit is used for determining the skeletal key points of the infants in each frame of image from each target walking action type sub-video;
The angle determining unit is used for determining the angle between the midpoint of the two shoulder joints and the connecting line of the midpoint of the two hip joints in the key points of the infant bones in each frame of image in each target walking action type sub-video and the vertical axis of the video picture;
The variation coefficient determining unit is used for determining the variation coefficient of the angle between the connecting line of the midpoint of the two shoulder joints and the midpoint of the two hip joints in each frame of the infant skeleton key points in each frame of the target walking action type sub-video and the vertical axis of the video picture according to the angle between the connecting line and the vertical axis of the video picture;
And the detection unit is used for determining the posture stability of each infant to be detected when moving according to the variation coefficient so as to determine the infant walking action development detection result.
The embodiment of the invention also provides computer equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the infant walking action development detection method when executing the computer program.
The embodiment of the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the infant walking action development detection method when being executed by a processor.
Embodiments of the present invention also provide a computer program product comprising a computer program which, when executed by a processor, implements the infant walking action development detection method described above.
In the embodiment of the invention, the infant walking action development detection scheme is adopted, and the infant walking action development detection scheme works as follows: the acquisition device acquires each infant walking action video to be detected; the uploading device uploads each infant walking action video to be detected to the server; the server stores the walking action video of each infant to be detected in real time, and designates a cutting platform of the walking action video of each infant to be detected; dividing a specified infant walking action video to be detected by each cutting platform in the distributed cutting platform cluster according to the target walking action type of the infant to be detected, and obtaining a plurality of sub-videos of the target walking action types of each infant to be detected; the detection device downloads each target walking action type sub-video, determines infant skeleton key points in each frame of image from each target walking action type sub-video, and determines angles between the connecting lines of the middle points of two shoulder joints and the middle points of two hip joints in each frame of image in each target walking action type sub-video and the vertical axis of the video picture; determining a variation coefficient of the angle between the connecting line and the vertical axis of the video picture according to the angle; and determining the posture stability of each infant to be detected when moving according to the variation coefficient so as to determine the infant walking action development detection result.
The infant walking action development detection scheme provided by the embodiment of the invention has the beneficial technical effects that: the embodiment of the invention realizes the online integrated completion of video acquisition and video appointed segmentation of the infant walking action detection, does not need to wear close-fitting articles for infants in the infant walking action development detection, does not need to carry out autonomous movement of infants in natural environment completely according to the instructions of researchers, and detects the posture stability of the infants during movement according to the angles between the midpoint of two shoulder joints and the midpoint connecting line of two hip joints and the vertical axis of video images after the skeletal key points are determined through computer vision. Therefore, the embodiment of the invention can realize the high-efficiency and accurate assessment of the stability of the walking action posture of the infant, and further can detect the growth condition of the walking action of the infant.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. In the drawings:
FIG. 1 is a schematic flow chart of a method for detecting infant walking action development applied to a detection system according to an embodiment of the invention;
FIG. 2 is a schematic diagram of the detection of infant walking action development in an embodiment of the present invention;
FIG. 3 is an interface diagram of a server specifying a cutting platform for each infant walking motion video to be detected according to an embodiment of the present invention;
FIG. 4 is a schematic illustration of bone keypoints determined in an embodiment of the invention;
FIG. 5 is a flow chart of a method for detecting infant walking action development applied to a detection device according to an embodiment of the invention;
FIG. 6 is a schematic diagram of a device for detecting the movement of an infant walking system according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a device for detecting the movement of an infant walking in an embodiment of the invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be described in further detail with reference to the accompanying drawings. The exemplary embodiments of the present invention and their descriptions herein are for the purpose of explaining the present invention, but are not to be construed as limiting the invention.
The application discloses a method for detecting the walking action development of an infant, which is an information processing method implemented by devices such as a computer and the like in all steps.
The embodiment of the invention provides a development detection scheme for infant walking action, which avoids unreasonable requirements on infant hearing instructions by online acquisition and storage of three-machine-position videos and online appointed cutting of infant walking action according to research purpose videos, collects and processes infant limb key point position data through a BlazePose gesture estimation model of computer vision without a marker, and evaluates infant walking action gesture stability without wearing equipment by the infant. The following describes the infant walking action development detection scheme in detail.
FIG. 1 is a schematic flow chart of a method for detecting the development of infant walking action applied to a detection system according to an embodiment of the present invention, the method is applied to a detection system device, as shown in FIG. 1, and the method includes the following steps:
step 101: the acquisition device acquires each infant walking action video to be detected;
step 102: the uploading device uploads each infant walking action video to be detected to the server;
step 103: the server stores the walking action video of each infant to be detected in real time, and designates a cutting platform of the walking action video of each infant to be detected;
step 104: dividing a specified infant walking action video to be detected by each cutting platform in the distributed cutting platform cluster according to the target walking action type of the infant to be detected, and obtaining a plurality of sub-videos of the target walking action types of each infant to be detected;
step 105: the detection device performs the following level of calculation tasks: downloading each target walking action type sub-video, determining infant skeleton key points in each frame of image from each target walking action type sub-video, and determining angles between the midpoint of two shoulder joints and the connecting line of the midpoint of two hip joints in each frame of image in each target walking action type sub-video and the vertical axis of the video picture; performing the following two-level computing tasks: determining a variation coefficient of the angle between the connecting line and the vertical axis of the video picture according to the angle; and determining the posture stability of each infant to be detected when moving according to the variation coefficient so as to determine the infant walking action development detection result.
In the embodiment of the invention, the infant walking action development detection method comprises the following steps of: the acquisition device acquires each infant walking action video to be detected; the uploading device uploads each infant walking action video to be detected to the server; the server stores the walking action video of each infant to be detected in real time, and designates a cutting platform of the walking action video of each infant to be detected; dividing a specified infant walking action video to be detected by each cutting platform in the distributed cutting platform cluster according to the target walking action type of the infant to be detected, and obtaining a plurality of sub-videos of the target walking action types of each infant to be detected; the detection device performs the following level of calculation tasks: downloading each target walking action type sub-video, determining infant skeleton key points in each frame of image from each target walking action type sub-video, and determining angles between the midpoint of two shoulder joints and the connecting line of the midpoint of two hip joints in each frame of image in each target walking action type sub-video and the vertical axis of the video picture; performing the following two-level computing tasks: determining a coefficient of variation of the angle between the connection line and the vertical axis of the video picture according to the angle; and determining the posture stability of each infant to be detected when moving according to the variation coefficient so as to determine the infant walking action development detection result.
The infant walking action development detection method provided by the embodiment of the invention has the beneficial technical effects that: the embodiment of the invention realizes the online integrated completion of video acquisition and video appointed segmentation of the infant walking action detection, does not need to wear close-fitting articles for infants in the infant walking action development detection, does not need to carry out autonomous movement of infants in natural environment completely according to the instructions of researchers, and detects the posture stability of the infants during movement according to the angles between the midpoint of two shoulder joints and the midpoint connecting line of two hip joints and the vertical axis of video images after the skeletal key points are determined through computer vision. Therefore, the embodiment of the invention can efficiently and accurately evaluate the stability of the walking action posture of the infant, and further efficiently and accurately detect the growth condition of the walking action of the infant. The following describes the method for detecting the infant walking action development in detail with reference to fig. 2 to 4.
The embodiment of the invention realizes the functions of recording, online storage and online appointed video segmentation into a whole through online operation, then determines the key point position of the infant through the BlazePose gesture estimation model determination technology of offline computer vision, and performs calculation and analysis on the target-separated actions of walking so as to evaluate the gesture stability of the infant during movement. As shown in fig. 2, the architecture and principles of the detection system are shown in table 1 below.
TABLE 1
In specific implementation, the computer n+1 and the computer n+2 construct the detection device in the embodiment of the invention.
Taking infant walking as an example, the use process of the detection system is that the infant walking action development detection steps are as follows:
1. As shown in fig. 2, three cameras are fixed by a pi-shaped bracket and a small-sized triangular bracket, and then are connected with a computer 1 through a data line; the camera view is adjusted, in the acquisition device, the shooting direction of the front-shooting camera is vertical to the plane of the n-shaped bracket and is parallel to the ground (namely, the video picture of the front-shooting camera is parallel to the n-shaped bracket and is vertical to the ground at the moment), the shooting direction of the nodding-shooting camera is vertical to the ground (namely, the video picture of the nodding-shooting camera is parallel to the n-shaped bracket and the ground at the moment), and the direction of the side-shooting camera is parallel to the plane of the n-shaped bracket (namely, the video picture of the side-shooting camera is vertical to the n-shaped bracket and is vertical to the ground at the moment); the computer is linked with the network to ensure the smoothness of the network.
In specific implementation, the embodiment of the invention mainly uses the side-shooting position camera in the acquisition device to acquire the infant walking action video to be detected for segmentation and subsequent infant walking action evaluation, of course, the distributed cutting platform can also perform online real-time segmentation on the infant walking action video to be detected acquired by the nodding-shooting position camera and the infant walking action video to be detected acquired by the righting-shooting position camera, the detection device performs detection evaluation based on the segmented sub-videos, the detection device finally obtains the righting-shooting position infant walking action development detection result, the nodding-position infant walking action development detection result and the side-shooting position infant walking action development detection result, and finally can be combined with the detection results of three shooting orientations for fusion, so that a more accurate omnibearing detection result is obtained. Of course, the detection device may not combine the detection results of the three (the forward infant walking movement development detection result, the downward infant walking movement development detection result, and the lateral infant walking movement development detection result), and pick up the detection result corresponding to the group of clear video image as the final infant walking movement development detection result to obtain a more accurate detection result.
2. The online platform system is opened, the video acquisition is clicked, and the three positions are adjusted again.
3. The front of the side shooting position camera is provided with a transparent rectangular table with the height of 100cm multiplied by 50cm, the height of the table top is consistent with that of the side shooting position camera, and the long side of the table is perpendicular to the side shooting position camera.
4. The camera starts recording, and the baby walks along the long side of the long side far away from the camera under proper guidance of researchers.
5. The video is stored on the server on line in real time, and the start and end of the video are recorded in Beijing time to the accuracy of milliseconds.
6. A researcher (user) performs appointed cutting on the video according to research requirements on respective computers (cutting platforms), namely the cutting platforms receive cutting instructions of the user to obtain videos of infant walking actions and limb swinging actions in the air; the start and end of the cut video are specified to include Beijing time and the relative time of the cut video to the original video, and the accuracy is to millisecond.
Infants are in a state of constant activity from birth, such as whole body exercise, and they lack the same mature communication ability as adults, and it is difficult for them to achieve standardized test levels according to the instructions of experimenters. The data collected by the method needs to be further processed into data surrounding a certain fixed action, namely, data cutting work is needed. Some offline and online tools are currently available that can determine the start time of an action by encoding the video, but cannot perform cutting and clipping work around an action on the video. Some video editing tools that are offline on line, but do not encode behavior on video. Therefore, the embodiment of the invention provides a method for video segmentation based on the type of the infant target walking action (the type can be introduced in the following tables 1 and 2).
The specific implementation of splitting video is as follows:
a) Watching video, and recording the starting time and the ending time of the video to be intercepted (taking infant walking as an example) according to the research target of the video;
Clicking play after entering a basic information interface of a website;
The video playing speed can be set to between 0.25-2 times speed, and the video can be progressive at 0.25 times speed. The video is advanced and retracted according to the target motion (the type of the target walk motion, which can be seen in tables 1 and 2 below), with 10ms per click. Finally, determining the video starting time and the video ending time of the target action, wherein the video starting time and the video ending time are the time corresponding to the video, and the time is accurate to ms;
The video start and end times for the target action may be recorded in table 2 below.
TABLE 2
B) Setting corresponding action names in a website (server) evaluation designating interface, setting up not more than 99 action processes according to research requirements, naming, and describing if necessary, as shown in the following table 3;
TABLE 3 Table 3
C) Clicking 'designating' on a video storage page, namely, designating a cutting platform of each infant walking action video to be detected by a server;
d) Selecting the number of times of evaluation, clicking an edit button of a time period, selecting an action to be specified, and inputting the starting time and the ending time corresponding to the action at the time period, as shown in fig. 3;
in the implementation, the appointed and cut video can be obtained at the video appointed interface, and whether the appointed division of the video is correct or not can be checked through the playing function.
7. The main page of the webpage derives the time information of all target videos, and the derived data contains the relative start-stop time with the original video and also contains the start-stop time of the real recorded Beijing time of the target video.
8. Downloading the specified cut video, and determining key points (wrists, ankles, shoulders and hips) by using a BlazePose gesture estimation model of computer vision, wherein the determination result is shown in fig. 4, and in fig. 4:1 represents a left shoulder key point, 2 represents a right shoulder key point, 3 represents a left hip key point, 4 represents a right hip key point, 5 represents a left wrist key point, 6 represents a right wrist key point, 7 represents a left ankle key point, and 8 represents a right ankle key point. That is, in one embodiment, determining the skeletal keys of the infant in each frame of image from each target walking action type sub-video may include: and using a BlazePose gesture estimation model of computer vision to determine the key points of the infant bones in each frame of image from each target walking action type sub-video, thereby further improving the accuracy and efficiency of determining the key points.
9. Obtaining first-level calculation indexes such as an angle (central axes VERTICAL TILT, CAVT) between a connecting line of a middle point of a two-shoulder joint and a middle point of two-hip joint in each video segment and a vertical axis of a video picture, the key point position two-axis speed (v), two-axis acceleration (a), two-axis and tangential and normal jerk (jerk), and distances from a starting point to a finishing point of a wrist joint and an ankle joint of each video segment respectively, namely the first-level calculation indexes are obtained by python calculation;
the calculation process is as follows:
CAVT per frame:
（1）
CAVT n is the angle between the connecting line of the midpoint of the two shoulder joints and the midpoint of the two hip joints in the key points of the infant bones in each frame of image in each target walking action type sub-video and the vertical axis of the video picture; step 2, supporting the sequential identification of each frame of image in the action type sub-video for each target; /(I) Is the horizontal axis coordinate of the left shoulder of the infant,Is the longitudinal axis coordinate of the left shoulder of the infant,/>Is the right shoulder transverse axis coordinate,Is the vertical axis coordinate of the right shoulder,/>For infant left hip transverse axis coordinate,/>For the infant left hip longitudinal axis coordinate,/>Is the right hip transverse axis coordinate,/>Is the right hip longitudinal axis coordinate.
Biaxial velocity and velocity vector sum for each frame:
（2）
（3）
（4）
Biaxial acceleration and acceleration vector sum for each frame:
（5）
（6）
（7）
biaxial jerk and jerk vector sum for each frame:
（8）
（9）
（10）
tangential jerk (TANGENTIAL JERK, tj) for each frame:
（11）
（12）
From the foregoing, in one embodiment, determining an angle between a midpoint of a two-shoulder joint and a midpoint line of two hip joints in each frame of an infant bone key point in each target walking action type sub-video and a vertical axis of a video frame includes: the angle is determined according to the formula (1), and the formula (1) can improve the accuracy of the calculation of the angle and further improve the detection accuracy of the infant walking action.
10. Calculating the variation coefficient (CAVT variation coefficient, coefficient of variation of CAVT, CAVT-C.V.) and the dimensionless tangential jerk (Dimensionless Tangential jerk, dtj) of the angle between the two shoulder joint midpoint and the two hip joint midpoint connecting line in each video segment and the video picture vertical axis, wherein the step is the process of performing a secondary calculation task;
the calculation process is as follows:
CAVT-c.v. per video segment:
（13）
Wherein, CAVT n is the angle between the connecting line and the vertical axis of the video picture and the connecting line between the midpoint of two shoulder joints and the midpoint of two hip joints in the key points of the infant bones in each frame of image in each target walking action type sub-video; /(I)The sequential identification of each frame of image in the action type sub-video is walked for each target.
Dtj per video (sub-video):
（14）
Wherein: : the order of each frame in each video; /(I) ：33.33ms；/>: Duration (ms) of each video; /(I): Maximum value of the speed of each video, jerk: acceleration of acceleration, i.e. 3 rd derivative of distance, dtj: dimensionless Tangential jerk, dimensionless tangential jerk, i.e.: /(I)Step 2, supporting the sequential identification of each frame of image in the action type sub-video for each target; /(I)33.33Ms; /(I)Duration for each sub-video; /(I)For tangential jerk in each frame image,/>Is the maximum value of the speed of each sub-video.
As can be seen from the above, in one embodiment, according to the angle between the midpoint of the two shoulder joints and the midpoint of the two hip joints in the key points of the infant bone in each frame of the image in each target walking action type sub-video, determining the variation coefficient of the angle between the connecting line and the vertical axis of the video frame includes determining the variation coefficient of the angle according to the above formula (13), and the formula (13) can improve the calculation accuracy of the variation coefficient, and further improve the detection accuracy of the infant walking action.
From the foregoing, in one embodiment, the primary computing task further includes: determining tangential jerk of infant walking in each frame of image; the secondary computing task further includes: according to tangential jerk of infant walking in each frame of image, determining dimensionless tangential jerk of infant walking;
Determining the posture stability of each infant to be detected when moving according to the variation coefficient so as to determine the infant walking action development detection result, wherein the method comprises the following steps: and determining the posture stability of each infant to be detected when moving according to the variation coefficient and the dimensionless tangential jerk so as to determine the infant walking action development detection result.
In specific implementation, the embodiment of the invention further combines dimensionless tangential jerk (the dimensionless tangential jerk can further reflect the movement fluency of the limbs of the infant, for example, the smaller the dimensionless tangential jerk is, the more stable the trunk movement is, the smoother the limb movement is, the larger the dimensionless tangential jerk is, the more unstable the trunk is, the more unstable the limb movement is, and the more unsmooth the limb movement is) while considering the gesture stability when detecting the movement of the infant based on the angle between the connecting line of the middle points of the two shoulder joints and the middle points of the two hip joints in each frame of image and the vertical axis of the video image, thereby further improving the precision of the movement development detection of the infant.
From the foregoing, in one embodiment, determining a dimensionless tangential jerk of the infant walker according to the tangential jerk of the infant walker in each frame of images includes:
Determining the biaxial acceleration and the acceleration vector sum of each frame according to the biaxial speed and the speed vector sum of each frame;
determining the biaxial jerk and jerk vector sum of each frame according to the biaxial acceleration and the acceleration vector sum of each frame;
Determining tangential jerk of each frame according to the biaxial jerk and jerk vector sum of each frame;
and determining dimensionless tangential jerk according to the tangential jerk in each frame of image.
In the specific implementation, the embodiment of determining the dimensionless tangential jerk of the infant walking further improves the calculation accuracy of the dimensionless tangential jerk and further improves the detection accuracy of the infant walking action.
In one embodiment, determining the dimensionless tangential jerk of the baby walker according to the tangential jerk of the baby walker in each frame of image includes determining the dimensionless tangential jerk according to the above formula (14), where the formula (14) may improve the accuracy of the calculation of the dimensionless tangential jerk of the baby walker, and further improve the detection accuracy of the baby walker motion.
11. The indexes are statistically analyzed, and scientific problems in the infant walking movement process, such as the relation between the limb movement fluency and the trunk stability, and the like, are explored. The step is a step of evaluating the posture stability of the infant when moving according to the detection result of the first-stage computing second-stage computing task, and then calculating a variation coefficient according to the angle after calculating the angle between the midpoint of the two shoulder joints and the connecting line of the midpoint of the two hip joints and the vertical axis of the video picture, wherein the variation coefficient can reflect the variation range of the angle, for example, the obtained evaluation result (infant walking action development detection result) can be: the smaller the variation range of the variation coefficient of the angle of the infant in the walking process, the posture stability of the infant is as follows: the more stable the trunk, the smoother the limb movement, and the detection result of the infant walking action development is as follows: the infant has good walking action (exercise) development condition; the greater the range of variation in the coefficient of variation, the greater the stability of posture of the infant when moving: the more unstable the trunk, the less fluent the limb movement, and the follow-up determinable infant walking action development detection result is: the infant has poor walking (exercise) development, and thus needs to determine the infant brain development. In specific implementation, the infant walking exercise development detection result may be determined by comparing the variation coefficient changes corresponding to the multi-frame images corresponding to each target walking exercise type sub-video, for example, the exercise types in table 3 above: and (3) walking, leftwards and leftwards, wherein 5 frames of images corresponding to the walking action type sub-video are used for determining the infant walking action development detection result according to the angle change range represented by comparing the variation coefficient corresponding to each frame of image.
In summary, the infant walking action development detection method provided by the embodiment of the invention realizes the following steps:
1. shooting videos at the same time by three machine positions and storing the videos on line;
2. the video specified segmentation can be completed on line so as to meet the research target;
3. Determining key points by adopting a BlazePose gesture estimation model of computer vision, so that the baby does not need to wear close-fitting articles;
4. the embodiment of the invention provides an angle (central axis VERTICAL TILT, CAVT) between the midpoint of the two shoulder joints and the connecting line of the midpoint of the two hip joints and the vertical axis of the video picture for the first time so as to detect the posture stability of the infant when moving, further accurately and efficiently determine the walking action development condition of the infant and further determine the brain development condition of the infant.
In summary, the method for detecting the development of the infant walking action provided by the embodiment of the invention has the beneficial technical effects that: the video acquisition and the video appointed segmentation of the infant walking action research are completed in an online integrated manner, close-fitting articles do not need to be worn on an infant in the infant walking action development research, the infant does not need to move autonomously in a natural environment completely according to the instructions of a researcher, and after key points are determined through a BlazePose gesture estimation model of computer vision, the researcher can perform data mining according to the research purpose so as to evaluate the gesture stability of the infant during movement.
The embodiment of the invention also provides a method for detecting the infant walking action development applied to the detection device, as described in the following embodiment. The principle of the infant walking action development detection method applied to the detection device for solving the problem is similar to that of the infant walking action development detection method applied to the detection system, so that the implementation of the infant walking action development detection method applied to the detection device can be referred to the implementation of the infant walking action development detection method applied to the detection system, and repeated parts are not repeated.
Fig. 5 is a schematic flow chart of a method for detecting the infant walking action development applied to a detection device according to an embodiment of the present invention, as shown in fig. 5, the method includes the following steps:
Step 201: acquiring a sub-video of each target walking action type; dividing a specified infant walking action video to be detected by each cutting platform in the distributed cutting platform cluster according to the target walking action type of the infant to be detected;
Step 202: determining infant skeleton key points in each frame of image from each target walking action type sub-video;
step 203: determining an angle between a connecting line of a midpoint of two shoulder joints and a midpoint of two hip joints in key points of infant bones in each frame of image in each target walking action type sub-video and a vertical axis of a video picture;
Step 204: determining a variation coefficient of an angle between a connecting line of a midpoint of two shoulder joints and a midpoint of two hip joints in key points of infant bones in each frame of image in each target walking action type sub-video and a vertical axis of a video picture according to the angle between the connecting line and the vertical axis of the video picture;
Step 205: and determining the posture stability of each infant to be detected when moving according to the variation coefficient so as to determine the infant walking action development detection result.
In one embodiment, determining an angle between a midpoint of a two-shoulder joint and a midpoint connection of two hip joints in each frame of infant bone key point in each target walking action type sub-video and a vertical axis of the video frame comprises: the angle is determined according to the following formula:
；
CAVT n is the angle between the connecting line of the midpoint of the two shoulder joints and the midpoint of the two hip joints in the key points of the infant bones in each frame of image in each target walking action type sub-video and the vertical axis of the video picture; step 2, supporting the sequential identification of each frame of image in the action type sub-video for each target; /(I) Is the horizontal axis coordinate of the left shoulder of the infant,Is the longitudinal axis coordinate of the left shoulder of the infant,/>Is the right shoulder transverse axis coordinate,Is the vertical axis coordinate of the right shoulder,/>For infant left hip transverse axis coordinate,/>For the infant left hip longitudinal axis coordinate,/>Is the right hip transverse axis coordinate,/>Is the right hip longitudinal axis coordinate.
In one embodiment, determining the coefficient of variation of the angle between the connection line of the midpoint of the two shoulders and the midpoint of the two hips in each frame of the infant bone in each frame of the target walking action type sub-video and the vertical axis of the video frame according to the angle between the connection line and the vertical axis of the video frame comprises determining the coefficient of variation of the angle according to the following formula:
；
Wherein, CAVT n is the angle between the connecting line and the vertical axis of the video picture and the connecting line between the midpoint of two shoulder joints and the midpoint of two hip joints in the key points of the infant bones in each frame of image in each target walking action type sub-video; /(I)The sequential identification of each frame of image in the action type sub-video is walked for each target.
In one embodiment, the method for detecting the infant walking action development further comprises the following steps:
Determining tangential jerk of infant walking in each frame of image;
According to tangential jerk of infant walking in each frame of image, determining dimensionless tangential jerk of infant walking;
Determining the posture stability of each infant to be detected when moving according to the variation coefficient so as to determine the infant walking action development detection result, wherein the method comprises the following steps: and determining the posture stability of each infant to be detected when moving according to the variation coefficient and the dimensionless tangential jerk so as to determine the infant walking action development detection result.
In one embodiment, determining the dimensionless tangential jerk of the baby walker based on the tangential jerk of the baby walker in each frame of image comprises:
Determining the biaxial acceleration and the acceleration vector sum of each frame according to the biaxial speed and the speed vector sum of each frame;
determining the biaxial jerk and jerk vector sum of each frame according to the biaxial acceleration and the acceleration vector sum of each frame;
Determining tangential jerk of each frame according to the biaxial jerk and jerk vector sum of each frame;
and determining dimensionless tangential jerk according to the tangential jerk in each frame of image.
In one embodiment, determining the dimensionless tangential jerk of the infant walker based on the tangential jerk of the infant walker in each frame of images includes determining the dimensionless tangential jerk according to the following formula:
Wherein: step 2, supporting the sequential identification of each frame of image in the action type sub-video for each target; /(I) 33.33Ms; /(I)Duration for each sub-video; /(I)For tangential jerk in each frame image,/>Is the maximum value of the speed of each sub-video.
The embodiment of the invention also provides a device for detecting the infant walking action development, which is described in the following embodiment. Because the principle of the device for solving the problem is similar to that of the infant walking action development detection method applied to the detection system, the implementation of the device can be referred to the implementation of the infant walking action development detection method applied to the detection system, and the repetition is omitted.
FIG. 6 is a schematic structural diagram of a system for detecting the movement of an infant walking in accordance with an embodiment of the present invention, as shown in FIG. 6, the system includes:
the acquisition device 01 is used for acquiring each infant walking action video to be detected;
the uploading device 02 is connected with the acquisition device and is used for uploading each infant walking action video to be detected to the server;
the server 03 is connected with the uploading device and is used for storing each infant walking action video to be detected in real time and designating a cutting platform of each infant walking action video to be detected;
The distributed cutting platform clusters 04, wherein each cutting platform in the clusters is used for dividing the specified infant walking action video to be detected according to the target walking action type of the infant to be detected to obtain a plurality of sub-videos of the target walking action types of each infant to be detected;
The detecting device 05 is used for performing the following first-level computing tasks: downloading each target walking action type sub-video, determining infant skeleton key points in each frame of image from each target walking action type sub-video, and determining angles between the midpoint of two shoulder joints and the connecting line of the midpoint of two hip joints in each frame of image in each target walking action type sub-video and the vertical axis of the video picture; performing the following two-level computing tasks: determining a variation coefficient of the angle between the connecting line and the vertical axis of the video picture according to the angle; and determining the posture stability of each infant to be detected when moving according to the variation coefficient so as to determine the infant walking action development detection result.
In one embodiment, the primary computing task further comprises: determining tangential jerk of infant walking in each frame of image; the secondary computing task further includes: according to tangential jerk of infant walking in each frame of image, determining dimensionless tangential jerk of infant walking;
The detection device is specifically used for: and determining the posture stability of each infant to be detected when moving according to the variation coefficient and the dimensionless tangential jerk so as to determine the infant walking action development detection result.
In one embodiment, the detection device is specifically configured to: the angle is determined according to the following formula:
；
CAVT n is the angle between the connecting line of the midpoint of the two shoulder joints and the midpoint of the two hip joints in the key points of the infant bones in each frame of image in each target walking action type sub-video and the vertical axis of the video picture; step 2, supporting the sequential identification of each frame of image in the action type sub-video for each target; /(I) Is the horizontal axis coordinate of the left shoulder of the infant,Is the longitudinal axis coordinate of the left shoulder of the infant,/>Is the right shoulder transverse axis coordinate,Is the vertical axis coordinate of the right shoulder,/>For infant left hip transverse axis coordinate,/>For the infant left hip longitudinal axis coordinate,/>Is the right hip transverse axis coordinate,/>Is the right hip longitudinal axis coordinate.
In one embodiment, the detection device is specifically configured to: the coefficient of variation of the angle is determined according to the following formula:
；
Wherein, CAVT n is the angle between the connecting line and the vertical axis of the video picture and the connecting line between the midpoint of two shoulder joints and the midpoint of two hip joints in the key points of the infant bones in each frame of image in each target walking action type sub-video; /(I)The sequential identification of each frame of image in the action type sub-video is walked for each target.
In one embodiment, determining the dimensionless tangential jerk of the baby walker based on the tangential jerk of the baby walker in each frame of image comprises:
Determining the biaxial acceleration and the acceleration vector sum of each frame according to the biaxial speed and the speed vector sum of each frame;
determining the biaxial jerk and jerk vector sum of each frame according to the biaxial acceleration and the acceleration vector sum of each frame;
Determining tangential jerk of each frame according to the biaxial jerk and jerk vector sum of each frame;
and determining dimensionless tangential jerk according to the tangential jerk in each frame of image.
In one embodiment, determining the dimensionless tangential jerk of the infant walker based on the tangential jerk of the infant walker in each frame of images includes determining the dimensionless tangential jerk according to the following formula:
Wherein: step 2, supporting the sequential identification of each frame of image in the action type sub-video for each target; /(I) 33.33Ms; /(I)Duration for each sub-video; /(I)For tangential jerk in each frame image,/>Is the maximum value of the speed of each sub-video.
The embodiment of the invention also provides a device for detecting the walking action development of the infant, which is described in the following embodiment. Because the principle of the device for solving the problems is similar to that of the infant walking action development detection method applied to the detection system, the implementation of the device can be referred to the implementation of the infant walking action development detection method applied to the detection system, and the repetition is omitted.
Fig. 7 is a schematic structural diagram of a device for detecting the movement of an infant walking in accordance with an embodiment of the present invention, as shown in fig. 7, the device includes:
a downloading unit 051, configured to obtain each target walking action type sub-video; dividing a specified infant walking action video to be detected by each cutting platform in the distributed cutting platform cluster according to the target walking action type of the infant to be detected;
The determining unit 052 is used for determining the skeletal key points of the infants in each frame of image from each target walking action type sub-video;
The angle determining unit 053 is used for determining an angle between a midpoint of two shoulder joints and a connecting line of the midpoint of two hip joints in key points of infant bones in each frame of image in each target walking action type sub-video and a vertical axis of a video picture;
The variation coefficient determining unit 054 is used for determining a variation coefficient of an angle between a connecting line of a midpoint of two shoulder joints and a midpoint of two hip joints in each frame of skeleton key points of the infant in each frame of image in each target walking action type sub-video and a vertical axis of a video picture according to the angle;
And the detection unit 055 is used for determining the posture stability of each infant to be detected when moving according to the variation coefficient so as to determine the infant walking action development detection result.
In one embodiment, the infant walking movement development detection device may further include:
the tangential jerk determining unit is used for determining tangential jerk of infant walking in each frame of image;
The dimensionless tangential jerk determining unit is used for determining the dimensionless tangential jerk of the infant walking according to the tangential jerk of the infant walking in each frame of image;
The detection unit is specifically used for: and determining the posture stability of each infant to be detected when moving according to the variation coefficient and the dimensionless tangential jerk so as to determine the infant walking action development detection result.
In one embodiment, the angle determining unit is specifically configured to: the angle is determined according to the following formula:
；
CAVT n is the angle between the connecting line of the midpoint of the two shoulder joints and the midpoint of the two hip joints in the key points of the infant bones in each frame of image in each target walking action type sub-video and the vertical axis of the video picture; step 2, supporting the sequential identification of each frame of image in the action type sub-video for each target; /(I) Is the horizontal axis coordinate of the left shoulder of the infant,Is the longitudinal axis coordinate of the left shoulder of the infant,/>Is the right shoulder transverse axis coordinate,Is the vertical axis coordinate of the right shoulder,/>For infant left hip transverse axis coordinate,/>For the infant left hip longitudinal axis coordinate,/>Is the right hip transverse axis coordinate,/>Is the right hip longitudinal axis coordinate.
In one embodiment, the coefficient of variation determining unit is specifically configured to determine the coefficient of variation of the angle according to the following formula:
；
Wherein, CAVT n is the angle between the connecting line and the vertical axis of the video picture and the connecting line between the midpoint of two shoulder joints and the midpoint of two hip joints in the key points of the infant bones in each frame of image in each target walking action type sub-video; /(I)The sequential identification of each frame of image in the action type sub-video is walked for each target.
In one embodiment, determining the dimensionless tangential jerk of the baby walker based on the tangential jerk of the baby walker in each frame of image comprises:
Determining the biaxial acceleration and the acceleration vector sum of each frame according to the biaxial speed and the speed vector sum of each frame;
determining the biaxial jerk and jerk vector sum of each frame according to the biaxial acceleration and the acceleration vector sum of each frame;
Determining tangential jerk of each frame according to the biaxial jerk and jerk vector sum of each frame;
and determining dimensionless tangential jerk according to the tangential jerk in each frame of image.
In one embodiment, determining the dimensionless tangential jerk of the infant walker based on the tangential jerk of the infant walker in each frame of images includes determining the dimensionless tangential jerk according to the following formula:
Wherein: step 2, supporting the sequential identification of each frame of image in the action type sub-video for each target; /(I) 33.33Ms; /(I)Duration for each sub-video; /(I)For tangential jerk in each frame image,/>Is the maximum value of the speed of each sub-video. /(I)
The embodiment of the invention also provides computer equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the infant walking action development detection method when executing the computer program.
The embodiment of the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the infant walking action development detection method when being executed by a processor.
Embodiments of the present invention also provide a computer program product comprising a computer program which, when executed by a processor, implements the infant walking action development detection method described above.
The infant walking action development detection scheme provided by the embodiment of the invention has the beneficial technical effects that: the embodiment of the invention realizes the online integrated completion of video acquisition and video appointed segmentation of the infant walking action detection, does not need to wear close-fitting articles for infants in the infant walking action development detection, does not need to carry out autonomous movement of infants in natural environment completely according to the instructions of researchers, and detects the posture stability of the infants during movement according to the angles between the midpoint of two shoulder joints and the midpoint connecting line of two hip joints and the vertical axis of video images after the skeletal key points are determined through computer vision. Therefore, the embodiment of the invention can efficiently and accurately evaluate the stability of the walking action posture of the infant, and further efficiently and accurately detect the growth condition of the walking action of the infant.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.

Claims (9)

1. A method for detecting infant walking action development, wherein the method is applied to detection system equipment, and the method comprises the following steps:
The acquisition device acquires each infant walking action video to be detected;
The uploading device uploads each infant walking action video to be detected to the server;
the server stores the walking action video of each infant to be detected in real time, and designates a cutting platform of the walking action video of each infant to be detected;
Dividing a specified infant walking action video to be detected by each cutting platform in the distributed cutting platform cluster according to the target walking action type of the infant to be detected, and obtaining a plurality of sub-videos of the target walking action types of each infant to be detected;
The detection device downloads each target walking action type sub-video, determines infant skeleton key points in each frame of image from each target walking action type sub-video, and determines angles between the connecting lines of the middle points of two shoulder joints and the middle points of two hip joints in each frame of image in each target walking action type sub-video and the vertical axis of the video picture; determining a variation coefficient of the angle between the connecting line and the vertical axis of the video picture according to the angle; determining tangential jerk of infant walking in each frame of image; according to tangential jerk of infant walking in each frame of image, determining dimensionless tangential jerk of infant walking; and determining the posture stability of each infant to be detected when moving according to the variation coefficient and the dimensionless tangential jerk so as to determine the infant walking action development detection result.
2. A method for detecting the development of infant walking action, which is applied to a detection device and comprises the following steps:
Acquiring a sub-video of each target walking action type; dividing a specified infant walking action video to be detected by each cutting platform in the distributed cutting platform cluster according to the target walking action type of the infant to be detected;
Determining infant skeleton key points in each frame of image from each target walking action type sub-video;
Determining an angle between a connecting line of a midpoint of two shoulder joints and a midpoint of two hip joints in key points of infant bones in each frame of image in each target walking action type sub-video and a vertical axis of a video picture;
determining a variation coefficient of an angle between a connecting line of a midpoint of two shoulder joints and a midpoint of two hip joints in key points of infant bones in each frame of image in each target walking action type sub-video and a vertical axis of a video picture according to the angle between the connecting line and the vertical axis of the video picture;
Determining tangential jerk of infant walking in each frame of image;
According to tangential jerk of infant walking in each frame of image, determining dimensionless tangential jerk of infant walking;
and determining the posture stability of each infant to be detected when moving according to the variation coefficient and the dimensionless tangential jerk so as to determine the infant walking action development detection result.
3. The method of claim 2, wherein determining an angle between a midpoint of a two-shoulder joint and a midpoint line of two hip joints in each frame of the infant's bone key points in each target walking action type sub-video and a vertical axis of the video frame comprises: the angle is determined according to the following formula:
；
CAVT n is the angle between the connecting line of the midpoint of the two shoulder joints and the midpoint of the two hip joints in the key points of the infant bones in each frame of image in each target walking action type sub-video and the vertical axis of the video picture; step 2, supporting the sequential identification of each frame of image in the action type sub-video for each target; /(I) Is the left shoulder transverse axis coordinate of the infant,/>Is the longitudinal axis coordinate of the left shoulder of the infant,/>Is the right shoulder horizontal axis coordinate,/>Is the vertical axis coordinate of the right shoulder,For infant left hip transverse axis coordinate,/>For the infant left hip longitudinal axis coordinate,/>Is the right hip transverse axis coordinate,/>Is the right hip longitudinal axis coordinate.
4. The method of claim 2, wherein determining the coefficient of variation of the angle between the line of the midpoint of the two shoulders and the midpoint of the two hips in each frame of the infant's bones in each target walking action type sub-video and the vertical axis of the video frame based on the angle between the line and the vertical axis of the video frame comprises determining the coefficient of variation of the angle according to the following formula:
；
Wherein, CAVT n is the angle between the connecting line and the vertical axis of the video picture and the connecting line between the midpoint of two shoulder joints and the midpoint of two hip joints in the key points of the infant bones in each frame of image in each target walking action type sub-video; /(I)The sequential identification of each frame of image in the action type sub-video is walked for each target.
5. An infant walking movement development detection system apparatus, comprising:
The acquisition device is used for acquiring each infant walking action video to be detected;
The uploading device is connected with the acquisition device and used for uploading the walking action video of each infant to be detected to the server;
the server is connected with the uploading device and used for storing each infant walking action video to be detected in real time and designating a cutting platform of each infant walking action video to be detected;
the distributed cutting platform clusters, wherein each cutting platform in the clusters is used for dividing the designated infant walking action video to be detected according to the target walking action type of the infant to be detected to obtain a plurality of sub-videos of the target walking action types of each infant to be detected;
The detection device is used for downloading each target walking action type sub-video, determining infant skeleton key points in each frame of image from each target walking action type sub-video, and determining angles between the midpoint of two shoulder joints and the connecting line of the midpoint of two hip joints in each frame of image in each target walking action type sub-video and the vertical axis of the video picture; determining a variation coefficient of the angle between the connecting line and the vertical axis of the video picture according to the angle; determining tangential jerk of infant walking in each frame of image; according to tangential jerk of infant walking in each frame of image, determining dimensionless tangential jerk of infant walking; and determining the posture stability of each infant to be detected when moving according to the variation coefficient and the dimensionless tangential jerk so as to determine the infant walking action development detection result.
6. A device for detecting the development of infant walking action, comprising:
the downloading unit is used for acquiring each target walking action type sub-video; dividing a specified infant walking action video to be detected by each cutting platform in the distributed cutting platform cluster according to the target walking action type of the infant to be detected;
the determining unit is used for determining the skeletal key points of the infants in each frame of image from each target walking action type sub-video;
The angle determining unit is used for determining the angle between the midpoint of the two shoulder joints and the connecting line of the midpoint of the two hip joints in the key points of the infant bones in each frame of image in each target walking action type sub-video and the vertical axis of the video picture;
The variation coefficient determining unit is used for determining the variation coefficient of the angle between the connecting line of the midpoint of the two shoulder joints and the midpoint of the two hip joints in each frame of the infant skeleton key points in each frame of the target walking action type sub-video and the vertical axis of the video picture according to the angle between the connecting line and the vertical axis of the video picture;
the tangential jerk determining unit is used for determining tangential jerk of infant walking in each frame of image;
The dimensionless tangential jerk determining unit is used for determining the dimensionless tangential jerk of the infant walking according to the tangential jerk of the infant walking in each frame of image;
And the detection unit is used for determining the posture stability of each infant to be detected when moving according to the variation coefficient and the dimensionless tangential jerk so as to determine the infant walking action development detection result.
7. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 4 when executing the computer program.
8. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program which, when executed by a processor, implements the method of any of claims 1 to 4.
9. A computer program product, characterized in that it comprises a computer program which, when executed by a processor, implements the method of any of claims 1 to 4.
CN202410282326.1A 2024-03-12 2024-03-12 Infant walking action development detection method, equipment and device Active CN117860242B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410282326.1A CN117860242B (en) 2024-03-12 2024-03-12 Infant walking action development detection method, equipment and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410282326.1A CN117860242B (en) 2024-03-12 2024-03-12 Infant walking action development detection method, equipment and device

Publications (2)

Publication Number Publication Date
CN117860242A CN117860242A (en) 2024-04-12
CN117860242B true CN117860242B (en) 2024-05-28

Family

ID=90585016

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410282326.1A Active CN117860242B (en) 2024-03-12 2024-03-12 Infant walking action development detection method, equipment and device

Country Status (1)

Country Link
CN (1) CN117860242B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113642525A (en) * 2021-09-02 2021-11-12 浙江大学 Infant neural development assessment method and system based on skeletal points
CN114842459A (en) * 2022-03-31 2022-08-02 上海商汤临港智能科技有限公司 Motion detection method, motion detection device, electronic device, and storage medium
CN115462783A (en) * 2022-10-12 2022-12-13 南昌航空大学 Infant crawling posture analysis system based on skeleton key point detection
KR20230050067A (en) * 2021-10-07 2023-04-14 서울대학교병원 System and method for automatically evaluating complexity of infant movement
CN116721468A (en) * 2023-06-27 2023-09-08 中国传媒大学 Intelligent guided broadcast switching method based on multi-person gesture estimation action amplitude detection
WO2023235777A2 (en) * 2022-06-01 2023-12-07 Rutgers, The State University Of New Jersey Techniques for measuring atypical neurodevelopment in neonates based on short video

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113642525A (en) * 2021-09-02 2021-11-12 浙江大学 Infant neural development assessment method and system based on skeletal points
KR20230050067A (en) * 2021-10-07 2023-04-14 서울대학교병원 System and method for automatically evaluating complexity of infant movement
CN114842459A (en) * 2022-03-31 2022-08-02 上海商汤临港智能科技有限公司 Motion detection method, motion detection device, electronic device, and storage medium
WO2023235777A2 (en) * 2022-06-01 2023-12-07 Rutgers, The State University Of New Jersey Techniques for measuring atypical neurodevelopment in neonates based on short video
CN115462783A (en) * 2022-10-12 2022-12-13 南昌航空大学 Infant crawling posture analysis system based on skeleton key point detection
CN116721468A (en) * 2023-06-27 2023-09-08 中国传媒大学 Intelligent guided broadcast switching method based on multi-person gesture estimation action amplitude detection

Also Published As

Publication number Publication date
CN117860242A (en) 2024-04-12

Similar Documents

Publication Publication Date Title
Wouda et al. Estimation of vertical ground reaction forces and sagittal knee kinematics during running using three inertial sensors
D’Antonio et al. A markerless system for gait analysis based on OpenPose library
Ahmadi et al. 3D human gait reconstruction and monitoring using body-worn inertial sensors and kinematic modeling
KR101488130B1 (en) Running form diagnostic system and method for scoring running form
CN105832343B (en) Multidimensional vision hand function rehabilitation quantitative evaluation system and evaluation method
Wang et al. Quantitative assessment of dual gait analysis based on inertial sensors with body sensor network
CN102679964B (en) Gait parameter measurement system and data processing device and method thereof
CN108338791A (en) The detection device and detection method of unstable motion data
CN109817048B (en) Recognition system for role posture and walking direction in virtual simulation environment
Laribi et al. Human lower limb operation tracking via motion capture systems
Yu et al. Human gait analysis based on OpenSim
Palani et al. Real-time joint angle estimation using mediapipe framework and inertial sensors
Hamill et al. Biomechanics: 40 years on
CN117860242B (en) Infant walking action development detection method, equipment and device
Richter et al. Human Climbing and Bouldering Motion Analysis: A Survey on Sensors, Motion Capture, Analysis Algorithms, Recent Advances and Applications.
Talaa et al. Computer Vision-Based Approach for Automated Monitoring and Assessment of Gait Rehabilitation at Home.
Niu et al. A survey on IMU-and-vision-based human pose estimation for rehabilitation
de la Rubia et al. Natural locomotion based on foot-mounted inertial sensors in a wireless virtual reality system
Chaumeil et al. Agreement between a markerless and a marker-based motion capture systems for balance related quantities
Rizaldy et al. Evaluation of action sport camera optical motion capture system for 3D gait analysis
Rafique et al. Position driven sit-to-stand simulation using human body motion and force capture
Maradani et al. The role of visualization in tele-rehabilitation: a case study
Callejas-Cuervo et al. Review of Optical and Inertial Technologies for Lower Body Motion Capture
Vamsikrishna et al. Classification of head movement patterns to aid patients undergoing home-based cervical spine rehabilitation
Liu et al. Kinematic Analysis of Intra-Limb Joint Symmetry via Multisensor Fusion

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant