CN113229832A - System and method for acquiring human motion information - Google Patents
System and method for acquiring human motion information Download PDFInfo
- Publication number
- CN113229832A CN113229832A CN202110316024.8A CN202110316024A CN113229832A CN 113229832 A CN113229832 A CN 113229832A CN 202110316024 A CN202110316024 A CN 202110316024A CN 113229832 A CN113229832 A CN 113229832A
- Authority
- CN
- China
- Prior art keywords
- data
- electromyographic signal
- motion information
- processing module
- image data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000033001 locomotion Effects 0.000 title claims abstract description 104
- 238000000034 method Methods 0.000 title claims abstract description 13
- 238000012545 processing Methods 0.000 claims abstract description 89
- 210000000988 bone and bone Anatomy 0.000 claims abstract description 33
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 24
- 238000005070 sampling Methods 0.000 claims description 23
- 230000005540 biological transmission Effects 0.000 claims description 22
- 230000003321 amplification Effects 0.000 claims description 19
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 19
- 210000003205 muscle Anatomy 0.000 claims description 18
- 230000000694 effects Effects 0.000 claims description 13
- 238000004891 communication Methods 0.000 claims description 10
- 238000005516 engineering process Methods 0.000 claims description 9
- 239000000284 extract Substances 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 5
- 230000009466 transformation Effects 0.000 claims description 4
- 230000003183 myoelectrical effect Effects 0.000 abstract description 12
- 238000004458 analytical method Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000036544 posture Effects 0.000 description 5
- 206010049565 Muscle fatigue Diseases 0.000 description 4
- 208000006011 Stroke Diseases 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 208000012661 Dyskinesia Diseases 0.000 description 3
- 230000009471 action Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 210000001364 upper extremity Anatomy 0.000 description 3
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 210000002310 elbow joint Anatomy 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 241000282461 Canis lupus Species 0.000 description 1
- WHXSMMKQMYFTQS-UHFFFAOYSA-N Lithium Chemical compound [Li] WHXSMMKQMYFTQS-UHFFFAOYSA-N 0.000 description 1
- 206010061296 Motor dysfunction Diseases 0.000 description 1
- 240000007651 Rubus glaucus Species 0.000 description 1
- 235000011034 Rubus glaucus Nutrition 0.000 description 1
- 235000009122 Rubus idaeus Nutrition 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 238000010219 correlation analysis Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000035622 drinking Effects 0.000 description 1
- 210000003414 extremity Anatomy 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000005021 gait Effects 0.000 description 1
- 210000001503 joint Anatomy 0.000 description 1
- 208000027905 limb weakness Diseases 0.000 description 1
- 231100000861 limb weakness Toxicity 0.000 description 1
- 229910052744 lithium Inorganic materials 0.000 description 1
- 210000003141 lower extremity Anatomy 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000037023 motor activity Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000011164 ossification Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/40—Detecting, measuring or recording for evaluating the nervous system
- A61B5/4076—Diagnosing or monitoring particular conditions of the nervous system
- A61B5/4082—Diagnosing or monitoring movement diseases, e.g. Parkinson, Huntington or Tourette
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
- A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Public Health (AREA)
- Biomedical Technology (AREA)
- Surgery (AREA)
- Molecular Biology (AREA)
- Medical Informatics (AREA)
- Veterinary Medicine (AREA)
- Heart & Thoracic Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Animal Behavior & Ethology (AREA)
- Physiology (AREA)
- Neurology (AREA)
- Artificial Intelligence (AREA)
- Neurosurgery (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Signal Processing (AREA)
- Psychiatry (AREA)
- Developmental Disabilities (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Fuzzy Systems (AREA)
- Dentistry (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The application discloses a system and a method for acquiring human motion information, comprising the following steps: the system comprises a data processing module, a plurality of electromyographic signal acquisition modules and at least two image acquisition modules; the electromyographic signal acquisition module is used for acquiring electromyographic signal data of a user and sending the electromyographic signal data to the data processing module; the image acquisition module is used for acquiring image data of a user and sending the image data to the data processing module; the data processing module is used for acquiring the bone motion characteristics of the user by using a convolutional neural network according to the image data; extracting various electromyographic features of the electromyographic signal data; and determining the motion information of the user according to the bone motion characteristics and the various myoelectricity characteristics. The skeletal motion characteristics of the user are acquired from the image data acquired by the image acquisition module through the convolutional neural network, and then the motion information of the tested person can be accurately and objectively obtained by combining various myoelectric characteristics acquired from the myoelectric signal data acquired by the myoelectric signal acquisition module.
Description
Technical Field
The present application relates to the field of motion detection technologies, and in particular, to a system and a method for acquiring human motion information.
Background
Many different diseases can lead to dyskinesias in patients. Early studies have shown that about 66% of patients after stroke are associated with various motor dysfunctions, such as gait abnormalities and upper limb weakness. In medical treatment or physical treatment of dyskinesia, it is very necessary to evaluate the motor ability of a patient before and after treatment. The motor ability assessment provides guidance for further treatment. The existing scale experiments for evaluating the limb movement function, such as wolf movement function test, action research arm test and the like, require a patient to complete a series of carefully designed tasks according to requirements. Different scores are assigned to the patient's observations at the discretion of the therapist. The pre-ossification assessment of patients requires a trained therapist, however, the number of qualified therapists available is far less than the number of patients and the observed results may vary from person to person.
In view of the foregoing, it is desirable to provide a system and a method for acquiring human body motion information, which can accurately and objectively obtain motion information of a subject.
Disclosure of Invention
To solve the above problems, the present application proposes a system and method for acquiring human motion information.
In one aspect, the present application provides a system for acquiring human motion information, including: the system comprises a data processing module, a plurality of electromyographic signal acquisition modules and at least two image acquisition modules;
the electromyographic signal acquisition module is used for acquiring electromyographic signal data of a user and sending the electromyographic signal data to the data processing module;
the image acquisition module is used for acquiring image data of a user and sending the image data to the data processing module;
the data processing module is used for acquiring the bone motion characteristics of the user by using a convolutional neural network according to the image data; extracting various electromyographic features of the electromyographic signal data; and determining the motion information of the user according to the bone motion characteristics and the various myoelectricity characteristics.
Preferably, the electromyographic signal acquisition module includes: the device comprises an electrode unit, a sampling amplification unit and a wireless transmission unit;
the electrode unit is used for collecting electromyographic signals of a user and sending the electromyographic signals to the sampling amplification unit;
the sampling amplification unit is used for sampling and amplifying the electromyographic signals to obtain electromyographic signal data and sending the electromyographic signal data to the wireless transmission unit;
the wireless transmission unit is used for transmitting the electromyographic signal data to the data processing module.
Preferably, the image acquisition module includes: the device comprises an image acquisition unit and a micro-processing unit;
the image acquisition unit is used for acquiring original image data of a user and sending the original image data to the processing unit;
and the micro-processing unit is used for compressing and preprocessing the original image data to obtain image data with timestamp information, and sending the image data to the data processing module.
Preferably, the data processing module includes: a data interface unit and a processing unit;
the data interface unit is used for receiving the electromyographic signal data and the image data and transmitting the electromyographic signal data and the image data to the processing unit;
the processing unit is used for processing the image data by using a convolutional neural network to acquire bone motion characteristics including joint positions and angles; extracting time domain characteristics, frequency domain characteristics, autoregressive coefficient characteristics and/or wavelet transformation characteristics of the electromyographic signal data; determining a muscle activity index from the features extracted from the electromyographic signal data; and determining the motion information of the user according to the muscle activity index and the skeletal motion characteristic.
Preferably, the wireless transmission unit includes: a module supporting wireless communication technology.
Preferably, the electromyographic signal obtaining module further comprises: a battery; the battery is respectively connected with the sampling amplification unit and the wireless transmission unit.
Preferably, the micro-processing unit further comprises: and the TCP client is used for sending the image data to the data processing module.
Preferably, the data interface unit includes: a wireless communication adapter and a TCP server.
Preferably, the method further comprises the following steps: a cloud server;
the data processing module is further used for sending the received image data and the received electromyographic signal data to the cloud server;
the cloud server is used for storing the motion information or acquiring the bone motion characteristics of the user by using a convolutional neural network according to the image data; extracting various electromyographic features of the electromyographic signal data; and determining the motion information of the user according to the bone motion characteristics and various myoelectricity characteristics, and sending the motion information to the data processing module.
In a second aspect, the present application provides a method for acquiring human motion information, including:
the electromyographic signal acquisition module acquires electromyographic signal data of a user and sends the electromyographic signal data to the data processing module;
the image acquisition module acquires image data of a user and sends the image data to the data processing module;
the data processing module acquires the bone motion characteristics of the user by using a convolutional neural network according to the image data;
the data processing module extracts various electromyographic features of the electromyographic signal data;
and the data processing module determines the motion information of the user according to the bone motion characteristics and various myoelectricity characteristics.
The application has the advantages that: the skeletal motion characteristics of the user are acquired from the image data acquired by the image acquisition module through the convolutional neural network, and then the motion information of the tested person can be accurately and objectively obtained by combining various myoelectric characteristics acquired from the myoelectric signal data acquired by the myoelectric signal acquisition module.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating preferred embodiments and are not to be construed as limiting the application. Also, like reference numerals are used to denote like parts throughout the drawings. In the drawings:
FIG. 1 is a schematic diagram of a system for acquiring human motion information provided herein;
FIG. 2 is a schematic diagram of an image acquisition module and an image acquisition module of a system for acquiring human motion information provided by the present application;
fig. 3 is a schematic diagram of a data processing module and a cloud server of a system for acquiring human motion information provided by the present application;
FIG. 4 is a schematic diagram of a main network structure of a convolutional neural network of a system for acquiring human motion information provided by the present application;
FIG. 5 is a schematic diagram of output data of a system for acquiring human motion information provided by the present application;
fig. 6 is a schematic step diagram of a method for acquiring human motion information provided by the present application.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
In a first aspect, according to an embodiment of the present application, there is provided a system for acquiring human motion information, as shown in fig. 1, including: a data processing module 300, a plurality of electromyographic signal acquisition modules 100, and at least two image acquisition modules 200. And the electromyographic signal acquisition module is used for acquiring electromyographic signal data of the user and sending the electromyographic signal data to the data processing module. And the image acquisition module is used for acquiring image data of a user and sending the image data to the data processing module. The data processing module is used for acquiring the bone motion characteristics of the user by using a convolutional neural network according to the image data; extracting various electromyographic features of the electromyographic signal data; and determining the motion information of the user according to the bone motion characteristics and various myoelectricity characteristics. The exercise information can be used for evaluating muscle fatigue and the like, and can be used for rehabilitation tests of stroke patients. And determining muscle activity indexes according to various myoelectricity characteristics in the myoelectricity signal data, and determining muscle fatigue degrees according to the bone motion characteristics and the muscle activity indexes.
As shown in fig. 2, the electromyographic signal acquisition module is connected to the skin of a user for acquiring electromyographic signal data, and includes: the device comprises an electrode unit, a sampling amplification unit and a wireless transmission unit. The output end of the electrode unit is connected with the input end of the amplification sampling unit, and the output end of the amplification sampling unit is connected with the input end of the wireless transmission unit. And the electrode unit is used for collecting the electromyographic signals of the user and sending the electromyographic signals to the sampling amplification unit. And the sampling amplification unit is used for sampling and amplifying the electromyographic signals to obtain electromyographic signal data and sending the electromyographic signal data to the wireless transmission unit. And the wireless transmission unit is used for transmitting the electromyographic signal data to the data processing module.
The sampling amplification unit can amplify and quantize the electromyographic signals collected by the electrode unit. The sampling rate of each channel of the sampling amplification unit is 250S/S to 16kS/S, the gain is 1 time to 24 times, the number of available channels is at most 8, and the sampling amplification unit can be connected with a high-density electrode array.
The electromyographic signal acquisition module further comprises: and the battery is respectively connected with the sampling amplification unit and the wireless transmission unit. In particular, the battery may include a first voltage output terminal and a second voltage output terminal. The first voltage output end is connected with the sampling amplification unit and the wireless transmission unit respectively, and the second voltage output end is connected with the sampling amplification unit. The first voltage output end is used for outputting 3.3V LDO voltage, and the second voltage output end is used for outputting 5V voltage.
The wireless transmission unit includes: modules supporting wireless communication technologies such as Wi-Fi modules, bluetooth modules, Zigbee modules, UWB modules, modules supporting 802.15.6 technologies, and the like, or wireless communication modules supporting other commercial or self-developed technologies. Preferably, a bluetooth low energy module is used. Since the bluetooth low energy module supports data length extension, the transmission speed is sufficient for the transmission of electromyographic signal data while maintaining the low power consumption characteristic. The electromyographic signal acquisition module consumes 14mA current under the power supply of a 3.7V lithium battery, the size of a sensor board is 2cm multiplied by 3cm multiplied by 1cm, the weight of the sensor board is 16g (including a battery), and the service life of the battery can reach more than 20 hours.
As shown in fig. 2, the image acquiring module is used for acquiring images of user movement from different angles, and comprises: the device comprises an image acquisition unit and a micro-processing unit. And the image acquisition unit is used for acquiring original image data of a user and sending the original image data to the processing unit. And the micro-processing unit is used for compressing and preprocessing the original image data to obtain image data with timestamp information and sending the image data to the data processing module. The micro-processing unit is preferably a microcomputer such as a raspberry pi.
The image acquisition unit can adopt an RGB image sensor to acquire human visual information, and the frame rate is 30fps and the resolution is 1920 multiplied by 1080. The resolution and frame rate of the image data can be configured by the micro-processing unit in different application scenarios, such as step analysis and/or upper limb movement measurements. The micro-processing unit compresses the raw image data in JPEG format. The micro-processing unit may perform local or global preprocessing on the raw data unit, the preprocessing including: median filtering, gamma transformation, etc. The micro-processing unit wirelessly transmits the image data to the data processing module through a TCP protocol. Before the image data is transmitted, the micro-processing unit adds time stamp information to a data frame of the image data so as to synchronize between the different image acquisition modules and the electromyographic signal data acquired in the electromyographic signal acquisition modules.
As shown in fig. 3, the data processing module includes: a data interface unit and a processing unit. And the data interface unit is used for receiving the electromyographic signal data and the image data and transmitting the data to the processing unit. The processing unit is used for processing the image data by using a convolutional neural network to acquire bone motion characteristics including joint positions and angles; extracting time domain characteristics, frequency domain characteristics, autoregressive coefficient characteristics and/or wavelet transformation characteristics of electromyographic signal data; determining a muscle activity index from the features extracted from the electromyographic signal data; and determining the motion information of the user according to the muscle activity index and the bone motion characteristic.
The data interface unit includes: a wireless communication adapter and a TCP server. The wireless communication adapter includes a module supporting wireless communication technology, such as a Wi-Fi module, a bluetooth module, a Zigbee module, a UWB module, a module supporting 802.15.6 technology, etc., or a wireless communication module supporting other commercial or self-developed technologies. Preferably, a bluetooth adapter is adopted for adapting to the wireless transmission unit in the electromyogram signal acquisition module, and preferably, the bluetooth adapter can be simultaneously connected with a plurality of wireless transmission units for data transmission.
The analysis platform runs on the data processing module, and comprises: a data collection and management Graphical User Interface (GUI). On the graphical user interface, the electromyographic signals and the video frames in the image data can be rendered in real time. The electromyographic signals and the image data can be recorded and stored in a data processing module or uploaded to a cloud server. A human posture estimation algorithm based on CNN is designed for extracting human skeleton and motion activity and representing muscle activity by calculating a plurality of characteristics. And the human behavior is quantitatively analyzed by combining the information of the electromyographic sensor node and the image sensor.
The data processing module adopts a convolution neural network, extracts feature points based on interest points from image data, and uses the extracted feature points for calculating bone motion features such as positions, joint angles, bones and the like. The bone motion features may be displayed in the form of a skeletal model.
As shown in fig. 4, the structure of the human body posture estimation algorithm network based on the convolutional neural network according to the embodiment of the present invention can extract skeletal motion features from video frames captured by cameras at different positions in the upper and lower limb assessment tasks of a user. In the convolutional neural network used in the embodiment of the present application, the main network is composed of four convolutional modules. Each module comprises two convolutional layers (Conv) and one max pooling layer (Pool). It is a VGG-based network, so the kernel size of all convolutional layers is set to 3 × 3. The resolution of the video frames in the image data is 224 × 224 as Input to the entire network. After passing through the four convolution modules, the video frame is transmitted to 1024 feature maps. The signature includes two fully connected layers (FCs). The Output (Output) of the convolutional neural network is a vector of length 2 × N, where N is the number of feature points. The vector represents the coordinates of these points. The mean square error of the euclidean distance between the convolutional neural network output and the correct t-label (Ground Truth) is taken as the loss function l (x), which is expressed as:
wherein
An output vector representing the neural network is represented,
representing the correct t-labeled vector. In order to better fit the human body postures of the input video at different visual angles, the image data of the image acquisition modules at two different visual angles are trained by using the model with the same network structure. And carrying out normalization processing on output coordinates of the image data of the two visual angles, and calculating a three-dimensional coordinate result as a characteristic point. And calculating different motion and posture characteristics according to the extracted characteristic points to serve as the bone motion characteristics. The basic parameters of the bone motion characteristics comprise the positions of all joints, the included angles among frameworks, the speed and the angular speed of the frameworks and the like. The data processing module is also capable of calculating detail features from these basic parameters in the bone motion features, e.g. in stepsIn the long analysis, the characteristics comprise step length, step length time, cycle time, standing or swinging and the like, and the characteristics are important reference bases for judging the rehabilitation degree of the stroke patient.
The data processing module extracts a plurality of characteristics and indexes from the received electromyographic signal data to be used as electromyographic characteristics. In electromyographic signal analysis, time domain features are widely used because they directly reflect muscle activity and are easy to calculate. Preferably, the extracted time-domain features include: mean Absolute Value (MAV), standard deviation (STD), waveform accumulation length (WL), waveform symbol change times (ZC), slope symbol change times (SSC), and the like. The extracted frequency domain features include peak frequency, average frequency, median frequency, peak frequency power, and band power/Ratio (Ratio). In addition to the time domain feature and the frequency domain feature, an autoregressive coefficient and a wavelet transform may be extracted as the features. Various electromyographic characteristics and skeletal motion characteristics can be directly used for the joint analysis of the electromyographic signals and the images. The data processing module can also obtain higher-level indexes from the characteristics of fatigue and the like. The exercise information includes muscle fatigue. The frequency domain characteristic, i.e. the median frequency, is a good indicator for judging fatigue. The transition from high frequency to low frequency may reveal the degree of muscle fatigue.
The data processing module can perform mixed analysis on the bone motion characteristics extracted from the image data and indexes calculated by using various electromyographic characteristics extracted from the electromyographic signal data to determine motion information. The motion parameters included in the motion information are alternatives observed by the therapist in the scale experiment. I.e. the motor activity of different patients is almost the same, but much information about their rehabilitation is analyzed by the muscle status. The previously calculated various electromyographic characteristics or indexes can well reflect the state of the muscle. Through the comparison or correlation analysis of different movement stages, various myoelectric characteristics and skeleton movement characteristics are combined, and the human behavior can be further understood.
As shown in fig. 3, the embodiment of the present application further includes: the cloud server 400. The data processing module is also used for sending the received image data and the received electromyographic signal data to the cloud server. The cloud server is used for storing motion information or acquiring the bone motion characteristics of the user by using a convolutional neural network according to the image data; extracting various electromyographic features of the electromyographic signal data; and determining the motion information of the user according to the bone motion characteristics and various myoelectricity characteristics, and sending the motion information to the data processing module. When the data needing to be processed is large, the received image data and the electromyographic signal data can be sent to the cloud server through the data processing module to extract various electromyographic features and various electromyographic features, and the motion information of the user is determined.
As shown in fig. 5, the various electromyographic characteristics and skeletal movement characteristics of different muscles when performing different actions include electromyographic signal data of the muscles, an index of electromyographic characteristic calculation, and an elbow angle. The task was to take a bottle of water from the shelf, drink it, and then replace it. During the experiment, different volumes of water were placed in the bottles and placed on shelves of different heights, wherein the volumes of water included 100mL and 600mL and the heights of the shelves included 85cm and 125cm off the ground or from the plane. Four times of hand stretching and grabbing processes are carried out. Three muscles on the side of the working arm are monitored by using three electromyographic signal acquisition modules. The three muscles recorded were Brachioradialis (BRD), Biceps Brachii (BB) and Triceps Brachii (TB). Two image acquisition modules for acquiring a front view and a side view of a subject. And extracting bones and joint points of the upper arm as an image acquisition module by using the proposed human body posture estimation algorithm based on the convolutional neural network. The elbow joint angle output from the convolutional neural network is further calculated. The execution of each task is divided into stages according to the parameters and the actual meaning of the extracted features. And calculating the MAV (mean absolute value) characteristic of the mean absolute value in the electromyographic signal data as an index of muscle activity, and performing mixed analysis according to the electromyographic characteristic and the skeletal motion characteristic. The skeletal movement characteristics include the angle of the elbow joint and the corresponding myoelectric characteristics (myoelectric signals). The angle of the elbow shows the stages of action including grasping, drinking and returning. As can be seen from the electromyographic features and their mean absolute value MAV features, the Triceps Brachii (TB) is not activated when the height is not high enough. Contraction of the Biceps Brachii (BB) and Brachioradialis (BRD) is also associated with the motor phase.
In a second aspect, according to an embodiment of the present application, there is also provided a method for acquiring human motion information, as shown in fig. 6, including:
s101, acquiring electromyographic signal data of a user by an electromyographic signal acquisition module, and sending the electromyographic signal data to a data processing module;
s102, an image acquisition module acquires image data of a user and sends the image data to a data processing module;
s103, the data processing module acquires the bone motion characteristics of the user by using a convolutional neural network according to the image data;
s104, extracting various electromyographic features of the electromyographic signal data by the data processing module;
and S105, the data processing module determines the motion information of the user according to the bone motion characteristics and various myoelectricity characteristics.
According to the method, the skeletal motion characteristics of the user are acquired from the image data acquired by the image acquisition module through the convolutional neural network, and then the motion information of the tested person can be accurately and objectively obtained by combining various myoelectric characteristics acquired from the myoelectric signal data acquired by the myoelectric signal acquisition module. The image acquisition module and the electromyographic signal acquisition module transmit data to the data processing module in a wireless mode, so that the usability and the flexibility are enhanced. The analysis platform can be used in a cloud server, so that remote support is realized. The image-based human skeletal motion tracking has wide application in biomechanical research, the electromyographic signal data is directly reflected by muscle activity, and the recovery level of dyskinesia patients (such as patients after stroke) can be more effectively and quantitatively evaluated by combining the image data and the electromyographic signal data.
The above description is only for the preferred embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A system for acquiring human motion information, comprising: the system comprises a data processing module, a plurality of electromyographic signal acquisition modules and at least two image acquisition modules;
the electromyographic signal acquisition module is used for acquiring electromyographic signal data of a user and sending the electromyographic signal data to the data processing module;
the image acquisition module is used for acquiring image data of a user and sending the image data to the data processing module;
the data processing module is used for acquiring the bone motion characteristics of the user by using a convolutional neural network according to the image data; extracting various electromyographic features of the electromyographic signal data; and determining the motion information of the user according to the bone motion characteristics and the various myoelectricity characteristics.
2. The system for acquiring human motion information according to claim 1, wherein the electromyographic signal acquisition module comprises: the device comprises an electrode unit, a sampling amplification unit and a wireless transmission unit;
the electrode unit is used for collecting electromyographic signals of a user and sending the electromyographic signals to the sampling amplification unit;
the sampling amplification unit is used for sampling and amplifying the electromyographic signals to obtain electromyographic signal data and sending the electromyographic signal data to the wireless transmission unit;
the wireless transmission unit is used for transmitting the electromyographic signal data to the data processing module.
3. The system for acquiring human motion information of claim 1, wherein the image acquisition module comprises: the device comprises an image acquisition unit and a micro-processing unit;
the image acquisition unit is used for acquiring original image data of a user and sending the original image data to the processing unit;
and the micro-processing unit is used for compressing and preprocessing the original image data to obtain image data with timestamp information, and sending the image data to the data processing module.
4. The system for acquiring human motion information of claim 1, wherein the data processing module comprises: a data interface unit and a processing unit;
the data interface unit is used for receiving the electromyographic signal data and the image data and transmitting the electromyographic signal data and the image data to the processing unit;
the processing unit is used for processing the image data by using a convolutional neural network to acquire bone motion characteristics including joint positions and angles; extracting time domain characteristics, frequency domain characteristics, autoregressive coefficient characteristics and/or wavelet transformation characteristics of the electromyographic signal data; determining a muscle activity index from the features extracted from the electromyographic signal data; and determining the motion information of the user according to the muscle activity index and the skeletal motion characteristic.
5. The system for acquiring human motion information according to claim 2, wherein the wireless transmission unit comprises: a module supporting wireless communication technology.
6. The system for acquiring human motion information according to claim 2, wherein the electromyographic signal acquisition module further comprises: a battery; the battery is respectively connected with the sampling amplification unit and the wireless transmission unit.
7. The system for acquiring human motion information of claim 3, wherein the micro-processing unit further comprises: and the TCP client is used for sending the image data to the data processing module.
8. The system for acquiring human motion information according to claim 4, wherein the data interface unit comprises: a wireless communication adapter and a TCP server.
9. The system for acquiring human motion information of claim 1, further comprising: a cloud server;
the data processing module is further used for sending the received image data and the received electromyographic signal data to the cloud server;
the cloud server is used for storing the motion information or acquiring the bone motion characteristics of the user by using a convolutional neural network according to the image data; extracting various electromyographic features of the electromyographic signal data; and determining the motion information of the user according to the bone motion characteristics and various myoelectricity characteristics, and sending the motion information to the data processing module.
10. A method for obtaining human motion information, comprising:
the electromyographic signal acquisition module acquires electromyographic signal data of a user and sends the electromyographic signal data to the data processing module;
the image acquisition module acquires image data of a user and sends the image data to the data processing module;
the data processing module acquires the bone motion characteristics of the user by using a convolutional neural network according to the image data;
the data processing module extracts various electromyographic features of the electromyographic signal data;
and the data processing module determines the motion information of the user according to the bone motion characteristics and various myoelectricity characteristics.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110316024.8A CN113229832A (en) | 2021-03-24 | 2021-03-24 | System and method for acquiring human motion information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110316024.8A CN113229832A (en) | 2021-03-24 | 2021-03-24 | System and method for acquiring human motion information |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113229832A true CN113229832A (en) | 2021-08-10 |
Family
ID=77130725
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110316024.8A Pending CN113229832A (en) | 2021-03-24 | 2021-03-24 | System and method for acquiring human motion information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113229832A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114093488A (en) * | 2022-01-20 | 2022-02-25 | 武汉泰乐奇信息科技有限公司 | Doctor skill level judging method and device based on bone recognition |
| CN114627559A (en) * | 2022-05-11 | 2022-06-14 | 深圳前海运动保网络科技有限公司 | Exercise plan planning method, device, equipment and medium based on big data analysis |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103584919A (en) * | 2013-11-12 | 2014-02-19 | 中国科学院深圳先进技术研究院 | System and method for synchronously detecting multi-modal biological signals |
| WO2015015385A1 (en) * | 2013-07-31 | 2015-02-05 | Koninklijke Philips N.V. | System and method for guiding the placement of a sensor |
| CN104379056A (en) * | 2012-03-27 | 2015-02-25 | B10尼克斯有限公司 | System for the acquisition and analysis of muscle activity and operation method thereof |
| WO2015150931A1 (en) * | 2014-04-03 | 2015-10-08 | Universiti Brunei Darussalam | Realtime biofeedback mechanism and data presentation for knee injury rehabilitation monitoring and a soft real time intelligent system thereof |
| CN108564599A (en) * | 2018-04-08 | 2018-09-21 | 广东省智能制造研究所 | A kind of human motion speed estimation method |
| CN108597578A (en) * | 2018-04-27 | 2018-09-28 | 广东省智能制造研究所 | A kind of human motion appraisal procedure based on two-dimensional framework sequence |
| CN109800659A (en) * | 2018-12-26 | 2019-05-24 | 中国科学院自动化研究所南京人工智能芯片创新研究院 | A kind of action identification method and device |
-
2021
- 2021-03-24 CN CN202110316024.8A patent/CN113229832A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104379056A (en) * | 2012-03-27 | 2015-02-25 | B10尼克斯有限公司 | System for the acquisition and analysis of muscle activity and operation method thereof |
| WO2015015385A1 (en) * | 2013-07-31 | 2015-02-05 | Koninklijke Philips N.V. | System and method for guiding the placement of a sensor |
| CN103584919A (en) * | 2013-11-12 | 2014-02-19 | 中国科学院深圳先进技术研究院 | System and method for synchronously detecting multi-modal biological signals |
| WO2015150931A1 (en) * | 2014-04-03 | 2015-10-08 | Universiti Brunei Darussalam | Realtime biofeedback mechanism and data presentation for knee injury rehabilitation monitoring and a soft real time intelligent system thereof |
| CN108564599A (en) * | 2018-04-08 | 2018-09-21 | 广东省智能制造研究所 | A kind of human motion speed estimation method |
| CN108597578A (en) * | 2018-04-27 | 2018-09-28 | 广东省智能制造研究所 | A kind of human motion appraisal procedure based on two-dimensional framework sequence |
| CN109800659A (en) * | 2018-12-26 | 2019-05-24 | 中国科学院自动化研究所南京人工智能芯片创新研究院 | A kind of action identification method and device |
Non-Patent Citations (1)
| Title |
|---|
| 王思: "基于多传感器的运动员训练信息融合分析系统设计", 《计算技术与自动化》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114093488A (en) * | 2022-01-20 | 2022-02-25 | 武汉泰乐奇信息科技有限公司 | Doctor skill level judging method and device based on bone recognition |
| CN114093488B (en) * | 2022-01-20 | 2022-04-22 | 武汉泰乐奇信息科技有限公司 | Doctor skill level judging method and device based on bone recognition |
| CN114627559A (en) * | 2022-05-11 | 2022-06-14 | 深圳前海运动保网络科技有限公司 | Exercise plan planning method, device, equipment and medium based on big data analysis |
| CN114627559B (en) * | 2022-05-11 | 2022-08-30 | 深圳前海运动保网络科技有限公司 | Exercise plan planning method, device, equipment and medium based on big data analysis |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Jiang | Combination of wearable sensors and internet of things and its application in sports rehabilitation | |
| CN101773387B (en) | Body feeling network-based sleeveless driven pulse pressure measurement and automatic calibration device | |
| CN103584919B (en) | Multi-modal bio signal synchronous detecting system and method | |
| CN110179643A (en) | A kind of neck rehabilitation training system and training method based on annulus sensor | |
| CN103417218A (en) | Parameter acquisition evaluating system and method of upper limb movement | |
| CN106175747B (en) | Method and system for generating lead electrocardiogram signals using lead differential voltages | |
| CN113229832A (en) | System and method for acquiring human motion information | |
| CN104107134A (en) | Myoelectricity feedback based upper limb training method and system | |
| CN107788976A (en) | Sleep monitor system based on Amplitude integrated electroencephalogram | |
| CN105181998B (en) | A method of the SEMG and movement locus of arm when detection shooting | |
| Boissy et al. | Wireless inertial measurement unit with GPS (WIMU-GPS)—wearable monitoring platform for ecological assessment of lifespace and mobility in aging and disease | |
| CN111803099A (en) | Human upper limb muscle force prediction device and method based on radial basis function neural network | |
| CN102125425A (en) | Digital traditional Chinese medicine pulse-condition diagnosing system based on mobile communication terminal | |
| CN110801212B (en) | BCG signal heart rate extraction method based on neural network | |
| CN110960211A (en) | Embedded-based real-time electrocardio monitoring system | |
| CN109833041A (en) | Exercise induced fatigue intelligent measurement and analysis system | |
| CN106166069A (en) | The microsensor of a kind of myoelectricity collection amplification and sensor-based training method | |
| Tam et al. | A wearable wireless armband sensor for high-density surface electromyography recording | |
| CN107357419A (en) | A kind of motion recognition system and method based on co-contraction rate | |
| CN113283373B (en) | Method for enhancing limb movement parameters detected by depth camera | |
| CN105796078A (en) | Method of body physiology and motion parameter acquisition and transmission system | |
| CN105726016B (en) | Electrocardio output method and device | |
| Dong et al. | Wireless body area sensor network for posture and gait monitoring of individuals with Parkinson's disease | |
| CN110197727A (en) | Upper limb modeling method and motion function assessment system based on artificial neural network | |
| CN104656094A (en) | Portable gait information acquisition device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210826 Address after: No. 30 Shuangqing Road, Haidian District, Beijing 100084 Applicant after: TSINGHUA University Applicant after: Beijing ningju Technology Co.,Ltd. Address before: No. 30 Shuangqing Road, Haidian District, Beijing 100084 Applicant before: TSINGHUA University |
|
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Zhang Fulin Inventor after: Song Wei Inventor after: Wang Xuecheng Inventor after: Ma Xiaoyan Inventor after: Li Xin Inventor after: Pan Yu Inventor before: Song Wei Inventor before: Wang Xuecheng Inventor before: Zhang Fulin Inventor before: Ma Xiaoyan Inventor before: Li Xin Inventor before: Pan Yu |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210810 |