CN108514421A - The method for promoting mixed reality and routine health monitoring - Google Patents
The method for promoting mixed reality and routine health monitoring Download PDFInfo
- Publication number
- CN108514421A CN108514421A CN201810295931.7A CN201810295931A CN108514421A CN 108514421 A CN108514421 A CN 108514421A CN 201810295931 A CN201810295931 A CN 201810295931A CN 108514421 A CN108514421 A CN 108514421A
- Authority
- CN
- China
- Prior art keywords
- motion
- link
- data
- acceleration
- health monitoring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 230000036541 health Effects 0.000 title claims abstract description 24
- 238000012544 monitoring process Methods 0.000 title claims abstract description 16
- 230000001737 promoting effect Effects 0.000 title 1
- 230000033001 locomotion Effects 0.000 claims abstract description 149
- 238000004364 calculation method Methods 0.000 claims abstract description 27
- 230000008569 process Effects 0.000 claims abstract description 27
- 238000012545 processing Methods 0.000 claims abstract description 24
- 238000013473 artificial intelligence Methods 0.000 claims abstract description 20
- 230000009467 reduction Effects 0.000 claims abstract description 19
- 238000007781 pre-processing Methods 0.000 claims abstract description 17
- 238000012795 verification Methods 0.000 claims abstract description 14
- 238000005070 sampling Methods 0.000 claims abstract description 13
- 230000006870 function Effects 0.000 claims abstract description 10
- 230000001133 acceleration Effects 0.000 claims description 50
- 230000005540 biological transmission Effects 0.000 claims description 28
- 230000005021 gait Effects 0.000 claims description 28
- 239000011159 matrix material Substances 0.000 claims description 25
- 238000012706 support-vector machine Methods 0.000 claims description 21
- 239000013598 vector Substances 0.000 claims description 21
- 239000011521 glass Substances 0.000 claims description 20
- 230000009471 action Effects 0.000 claims description 14
- 238000001914 filtration Methods 0.000 claims description 14
- 239000002131 composite material Substances 0.000 claims description 12
- 238000012549 training Methods 0.000 claims description 12
- 230000003068 static effect Effects 0.000 claims description 10
- 230000000007 visual effect Effects 0.000 claims description 10
- 230000008859 change Effects 0.000 claims description 9
- 238000000354 decomposition reaction Methods 0.000 claims description 8
- 230000000694 effects Effects 0.000 claims description 8
- 230000009466 transformation Effects 0.000 claims description 5
- 230000002688 persistence Effects 0.000 claims description 4
- 230000001360 synchronised effect Effects 0.000 claims description 4
- 125000004122 cyclic group Chemical group 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 230000009191 jumping Effects 0.000 claims description 3
- 230000000630 rising effect Effects 0.000 claims description 3
- 125000002015 acyclic group Chemical group 0.000 claims 1
- 230000003044 adaptive effect Effects 0.000 abstract description 3
- 238000006243 chemical reaction Methods 0.000 abstract description 2
- 230000015572 biosynthetic process Effects 0.000 abstract 1
- 230000004069 differentiation Effects 0.000 abstract 1
- 238000000605 extraction Methods 0.000 abstract 1
- 238000003786 synthesis reaction Methods 0.000 abstract 1
- 230000006399 behavior Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000001514 detection method Methods 0.000 description 4
- 208000006011 Stroke Diseases 0.000 description 3
- 230000007547 defect Effects 0.000 description 3
- 230000002068 genetic effect Effects 0.000 description 3
- 230000005484 gravity Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 210000003141 lower extremity Anatomy 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000000276 sedentary effect Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 230000002526 effect on cardiovascular system Effects 0.000 description 2
- 238000005265 energy consumption Methods 0.000 description 2
- 210000000629 knee joint Anatomy 0.000 description 2
- 210000002414 leg Anatomy 0.000 description 2
- 230000035772 mutation Effects 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 208000024172 Cardiovascular disease Diseases 0.000 description 1
- 208000004067 Flatfoot Diseases 0.000 description 1
- 206010061159 Foot deformity Diseases 0.000 description 1
- 206010028347 Muscle twitching Diseases 0.000 description 1
- 208000008589 Obesity Diseases 0.000 description 1
- 206010044565 Tremor Diseases 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000001174 ascending effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000002612 cardiopulmonary effect Effects 0.000 description 1
- 208000026106 cerebrovascular disease Diseases 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 210000004072 lung Anatomy 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 208000030159 metabolic disease Diseases 0.000 description 1
- 235000020824 obesity Nutrition 0.000 description 1
- 230000001575 pathological effect Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 235000020030 perry Nutrition 0.000 description 1
- 238000000611 regression analysis Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 210000000707 wrist Anatomy 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1123—Discriminating type of movement, e.g. walking or running
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/112—Gait analysis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
- A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
- A61B5/6802—Sensor mounted on worn items
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/30—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Public Health (AREA)
- Medical Informatics (AREA)
- Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Physics & Mathematics (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Heart & Thoracic Surgery (AREA)
- Veterinary Medicine (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Physiology (AREA)
- Artificial Intelligence (AREA)
- Dentistry (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Psychiatry (AREA)
- Signal Processing (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Evolutionary Computation (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Epidemiology (AREA)
- Primary Health Care (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The present invention provides the methods that a kind of processing of artificial intelligence motion's hierarchical layered promotes mixed reality and routine health monitoring, on the one hand carry out hierarchical layered modeling in virtual reality movement, reduce time delay and the requirement to processor.Both subtle motion feature is acquired in preprocessing process comprehensively, on the other hand further through adaptive algorithmic match, limited different sensors function operation is only called in different specific differentiation processes, the different specific feature calculations of extraction, analyze link synthesis different characteristic dimensionality reduction sampling, calculation amount is greatly reduced, verification link is calculated by emphasis, in turn ensures correctness；Emphasis verification link is placed on high in the clouds simultaneously, remaining calculating is placed on wearable device end, effectively balances the contradiction of fast reaction and calculation amount, both acquires the minutia of motion state comprehensively, calculation amount and power consumption greatly reduce.
Description
Technical Field
The invention relates to the field of intelligent wearing, the technical field of virtual reality, the technical field of augmented reality, the technical field of mixed reality, the technical field of machine learning, the technical field of neural networks, the technical field of regression analysis, the technical field of support vector machines, the technical field of ant colony algorithms, the technical field of genetic algorithms and the technical field of artificial intelligence, in particular to a method for improving mixed reality and daily health monitoring through artificial intelligence motion hierarchical processing of mixed reality and daily health monitoring.
Background
Regular movement can improve the functions of heart and lung, and reduce the incidence of cardiovascular and cerebrovascular diseases, fat metabolic disorder and other diseases; for some special cases, the physiological parameters may be acquired during a particular gait cycle. For example, the prediction and diagnosis of stroke (stroke) and parkinson are related to the behavior of the action state in different steps of the gait cycle. Therefore, the motion state, period and motion fine characteristics need to be acquired finely.
More and more wearable equipment possesses the meter step function at present, through step rough estimation energy consumption, distance, can't further discern user's activity, and this kind of limitation causes the problem in two aspects:
1) effective exercises cannot be distinguished: effective exercise is an important way for improving the cardio-pulmonary function, enhancing strength and flexibility and improving the health level of people. For example, in the case of a user performing a strenuous activity such as tennis or basketball, the statistics of the existing devices are intermittent steps, and the calculated energy consumption is far below the actual level.
2) Sedentary immobility recognition error: sedentary immobility generally refers to standing still for more than 1 hour, which can cause cardiovascular, cervical and lumbar diseases; studies have shown that the longer the sitting time, the higher the risk of obesity and death; the conventional pedometer may erroneously recognize a non-counting situation such as a long-time riding or riding as sedentary.
Some equipment can carry out the division of motion state, carries out the discernment of motion state through the operation of detailed user activity recognition model, but the technological data volume's that this kind of motion state's discernment must bring the operation increase, if accomplish at wearable intelligent terminal (like bracelet, wrist-watch, waistband) and calculate, must increase the consumption of electric quantity. Due to constraints on volume, weight, and system performance, power supply capacity is one of the most important factors that limit widespread use. At present, wearable power consumption standby time is very limited originally, the detailed characteristics of the motion state are comprehensively collected in the prior art, the calculation amount and power consumption are large, and the cruising ability of the wearable equipment is reduced. If the calculation process is placed in the cloud, the increase of data transmission bandwidth and judgment time delay is inevitably brought, so that the timeliness of some movement responses needing to be judged quickly is influenced.
In conclusion, when the physiological parameters are collected by the existing wearable intelligent equipment, the motion state of the user is not considered, the motion state collection such as simple step counting is only carried out, and the difference of motion links and motion subtle characteristics is not collected. Or, the comprehensive collection aggravates the power consumption of the wearable terminal or increases the data transmission bandwidth and the judgment delay.
Meanwhile, in virtual reality, augmented reality and mixed reality, motion state data are required to be transmitted to a video processor, the data volume transmitted is large due to the fact that software and hardware sensors are involved, physical sign data are comprehensively collected, transmitted and processed through a camera and the sensors in the prior art, and high requirements are placed on transmission bandwidth and processor performance.
In order to ensure that the time delay given to the helmet is small enough, wired transmission with larger bandwidth is often needed, but wireless transmission cannot be used, so that the user experience is influenced; meanwhile, the performance requirement on the hardware processor is high, and the cost is improved.
Therefore, a processing technical scheme capable of reducing the transmission requirement and ensuring the user experience is urgently needed.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a method for improving mixed reality and daily health monitoring by artificial intelligence movement hierarchical processing, which can reduce transmission requirements and ensure user experience, aiming at the defects in the prior art.
According to the invention, the method for improving mixed reality and daily health monitoring by artificial intelligence motion hierarchical processing is provided, and comprises the following steps:
the method comprises the steps that a model for associating the waistband movement of the whole crowd with a video is established, according to the correlation between the movement state judged by the waistband of a common person and the video image requirements of VR glasses, the part of the VR glasses which needs to be displayed most clearly is established, and according to movement prediction, the vision compensation is carried out on the image with the transmission delay exceeding 20ms (threshold time period) within 20ms, and the model for associating the waistband movement with the VR video is established;
the method comprises the steps of establishing a part needing the clearest display in VR glasses according to the relevance of motion state judged by each person waistband and video image requirements of VR glasses of the model, performing visual compensation on images with transmission time delay exceeding 20ms in 20ms according to motion prediction, and establishing a waistband motion and VR video correlation model; because each person has specific user image information such as height, weight, age, gender and the like, and all the persons have attributes of the person in the crowd, the personal model is used as an input factor of the waistband movement and video association model of the whole crowd to influence the model of the corresponding image type of the whole crowd;
the method comprises the steps that VR glasses image pre-judging setting is set, when a person uses the VR glasses image pre-judging setting for the first time and a self model is not established, the individual VR glasses image pre-judging setting is set by using user portrait information such as height, weight, age and gender corresponding to a video correlation model in the movement of a waistband of an entire crowd, a part needing the clearest display in VR glasses is established according to the correlation of the movement state judged by the waistband and the video image requirement of the VR glasses, and the image with the transmission delay exceeding 20ms is subjected to vision compensation within 20ms according to movement prediction; starting for the second time, with personal data, carrying out pre-judgment setting by using the belt movement and video association model of individual groups;
the pre-judging accuracy module is used for establishing a part needing the clearest display in the VR glasses according to the correlation between the motion state judged by the waistband and the video image requirements of the VR glasses, and performing visual compensation within 20ms on the image with the transmission delay exceeding 20ms according to motion prediction; the compensation is checked against data transmitted 20ms later, the consistency is over 50% (predetermined percentage), the accuracy factor is set to 1, and 0 is set below 50%.
The waist belt motion state acquisition module is used for acquiring the waist belt sensor motion data of the user under the specific combination of the motion state and the gait;
executing preprocessing (a dimensionality reduction/typical verification module), combining different levels of data by using an optimized support vector machine in a hierarchical mode, collecting acceleration data output by an acceleration sensor worn by a user at regular time, filtering high-frequency noise through wavelet transformation, only calling limited different sensor functions to work in different specific judging processes through self-adaptive algorithm matching, extracting different specific feature calculation, and integrating different feature dimensionality reduction sampling in an analysis link, so that the calculated amount is greatly reduced, and the correctness is ensured through a key calculation verification link; meanwhile, a key verification link is placed at the cloud end, and other calculations are placed at the wearable equipment end, so that the contradiction between quick response and calculated amount is effectively balanced, the detailed characteristics of the motion state are comprehensively collected, and the calculated amount and the power consumption are greatly reduced;
pre-judging an accuracy training set by a belt movement and video association model, and entering data after movement preprocessing and data obtained by executing preprocessing into a training set;
the deep learning module is used for judging the accuracy of the data after the motion preprocessing and the data obtained by executing the preprocessing;
preferably, the motion state comprises a first cyclical action and a second non-cyclical action; the first cyclical action comprises walking and running; the second non-cyclical action comprises jumping, rising, sitting, and squatting; moreover, the walking includes going upstairs, going downstairs and walking on flat ground.
Preferably, the motion segment determination module is configured to:
acquiring acceleration values a of the acceleration sensor in the directions of the x, y and z axesx、ay、azAnd solving the acceleration signal vector modulus SVMA(ii) a Collecting angular velocity values w of the angular velocity sensor in the directions of three axes x, y and zx、wy、wzAnd solving the vector modulus SVM of the angular velocity signalW(ii) a The above-mentionedVector modulo SVM by acceleration signalAVector mode SVM for angular velocity signalWAnd establishing an identification model which is perfect in self-adaptation, subdividing the motion state and obtaining motion subdivision link information.
Preferably, the motion link judgment module further comprises a filtering unit, wherein the filtering unit performs wavelet transform operation of three steps of wavelet decomposition, high-frequency wavelet coefficient processing and wavelet reconstruction on the acquired data of the acceleration sensor and the angular velocity sensor, discretizes time domain signals in each direction, decomposes mixed signals of various frequency components into different frequency bands, and then processes the mixed signals according to different characteristics of each seed signal in a frequency domain and a frequency band; and acquiring gait data with high signal to noise ratio.
Preferably, the dimension reduction solving module is further configured to:
received data is normalized by subtracting the mean of the column from each element in the proof and dividing by the standard deviation of the column, such that each variable is normalized to a matrix X with a mean of 0 and a variance of 1, i.e., X ═ X1，X2，......Xn]T＝[Xij](n×p)；
Wherein,
to obtain
Solving a correlation coefficient matrix:
where R is a real symmetric matrix (i.e., R)ij＝rji) Wherein r is a correlation coefficient;
solving a correlation coefficient matrix:
if the accumulated contribution rate reaches more than 50%, a ratio height method is adopted, the characteristic value vector of the highest contribution rate is left as a fixed working sample set, and the rest is discarded;
calculating a partial matrix, using the remained characteristic value as a new variable principal component, and calculating a partial matrix F by using the following formula(n×m)＝X(n×p)·U(p×m)；
Wherein X is the original data matrix, U is the principal component load, and the scoring matrix F is the result obtained after dimensionality reduction.
Preferably, the dimension reduction solving module is further configured to:
calculating a first condition, a second condition and a third condition by using the output data of the acceleration sensor, and judging the gait of the human motion by using median filtering;
the first condition is: the accelerometer outputs a synthesized amplitude value, and if the synthesized amplitude value is between the given upper threshold value and the given lower threshold value, the human body is judged to be static; otherwise, judging the motion of the person;
the accelerometer output composite amplitude is:
the upper and lower thresholds are respectively: th (h)a min＝8m/s，tha max＝11m/s；
The first condition is expressed as:
the second condition is: if the local variance output by the accelerometer is lower than a given threshold value, the human body is judged to be static; otherwise, judging the motion of the person;
the local variance of the accelerometer output is:
whereinFor the interval, the accelerometer synthesizes an output average value of the amplitude, and the expression is as follows:
s is the number of half-window samples, and a given threshold is defined as: th (h)σa＝0.5m/s2The second condition is represented as:
the third condition is: the angular velocity sensor outputs an angular velocity composite amplitude, and if the angular velocity composite amplitude is lower than a given threshold value, the human body is judged to be static; otherwise, judging the motion of the person;
define the composite amplitude of the gyroscope output as:
the given thresholds are: th (h)w maxThe third condition is expressed as:
preferably, the sampling rate is adaptively adjusted according to the motion change rate, and the multipath transmission preprocesses the image: the rate of motion of the object being transmitted is proportional to the value of the feature required to restore the image. Such as: the object being photographed is stationary or substantially motionless and the sample rate of the transmission is proportionally reduced. Meanwhile, data is transmitted through multiple paths, the transmission capability of a network can be fully utilized, multiple routes in the route cache are used at the same time, and the setting weight of each link is adjusted according to the change rate of transmission objects of different paths to perform load balancing.
Preferably, when sending a data packet, according to the number of the sub-images decomposed after preprocessing in step 2, allocating the same number of link paths, polling in different paths by a time slice rotation method, polling the amount of time of each link, setting a weight for each link i by the dynamically changed RTT of each link i, and allocating the following relationship in each 20ms time slice, where n is the number of a certain link feature point, and is the acceleration of the feature point in three directions, and each link sends the preprocessed feature value that is currently sampled most recently after sending the previous feature.
Preferably, for the calculation sub-result merging process: respectively reading the output files of all processes in the current time step by the main process, performing splicing processing on the collected multi-channel video streams to generate a panoramic video stream carrying the timestamp, merging and restoring the result according to a regional decomposition algorithm, and temporarily storing the result in an ASCLL format; when a user wears the virtual reality terminal, whether the virtual reality terminal is in a motion state is detected, if so, video frequency frames to be played are adjusted according to the acceleration so as to provide synchronous video information for the user, and the information is displayed in a visual area of the virtual reality terminal.
Preferably, when the intra-frame delay refers to the rotation of the head of the user, the pixel points forming the picture frame jump back to the original point at the end of each frame, at this time, the image of the previous frame and the image of the previous frame are retained by the user persistence phenomenon, a smear phenomenon is generated, the inter-frame delay cannot exceed 20ms, and otherwise, a very obvious smear feeling is generated. To reduce the delay rate; the part exceeding 20ms is combined with the motion state to make up the prediction, thereby achieving the picture pause effect.
Drawings
A more complete understanding of the present invention, and the attendant advantages and features thereof, will be more readily understood by reference to the following detailed description when considered in conjunction with the accompanying drawings wherein:
fig. 1 schematically shows an overall functional block diagram of an intelligent waistband based on artificial intelligence hierarchical motion recognition according to a preferred embodiment of the invention.
Fig. 2 schematically shows a schematic diagram of human motion hierarchy partitioning according to a particularly preferred embodiment of the present invention.
FIG. 3 is a user walking geometry in an embodiment of the present invention.
Fig. 4 is a schematic diagram of three-condition gait detection in an embodiment of the invention.
FIG. 5 is a flow diagram illustrating adaptive refinement of a model in an exemplary embodiment of the invention.
Fig. 6 is a schematic diagram of an embodiment of the present invention.
Fig. 7 is a schematic flow chart of adaptive refinement of a model in an embodiment of the present invention.
It is to be noted, however, that the appended drawings illustrate rather than limit the invention. It is noted that the drawings representing structures may not be drawn to scale. Also, in the drawings, the same or similar elements are denoted by the same or similar reference numerals.
Detailed Description
In order that the present disclosure may be more clearly and readily understood, reference will now be made in detail to the present disclosure as illustrated in the accompanying drawings.
In view of the above defects in the prior art, the technical problem to be solved by the present invention is to provide an intelligent waistband based on an artificial intelligence hierarchical motion recognition method, aiming to solve the defect that the prior art does not consider the motion state of the user when acquiring physiological parameters; the invention identifies the movement links and movement subtle characteristics and acquires corresponding physiological parameter information, thereby facilitating the analysis of the health state of the user by doctors or other personnel. Meanwhile, the motion state is acquired in a grading mode, the data processing operation amount is reduced, the power consumption of the wearable equipment is reduced, and the cruising ability is improved.
Fig. 1 schematically shows an overall functional block diagram of an intelligent waistband based on artificial intelligence hierarchical motion recognition according to a preferred embodiment of the invention.
As shown in fig. 1, the intelligent waistband based on artificial intelligence hierarchical motion recognition according to the preferred embodiment of the invention comprises:
the physiological parameter setting module 10 is used for setting physiological parameters to be acquired under different combinations of motion states and gaits;
the physiological parameter acquisition module 20 is used for acquiring corresponding physiological parameters according to the physiological parameters to be acquired, which are set by the physiological parameter setting module 10, under the specific combination of the motion state and the gait;
the motion state solving module 30 is used for collecting acceleration data output by an acceleration sensor worn by a user at regular time, filtering high-frequency noise through wavelet transformation and dividing the motion state of the user; preferably, as shown in fig. 2, the motion state comprises a first cyclic action and a second non-cyclic action; the first cyclical action comprises walking and running; the second non-cyclical action comprises jumping, rising, sitting, and squatting; moreover, the walking includes going upstairs, going downstairs and walking on flat ground.
A dimensionality reduction solving module 40, configured to use an optimized Support Vector Machine (SVM) to distinguish, by combining hierarchical levels with different levels of data, partial pathological features based on physiological parameters;
and the motion link judging module 50 is configured to subdivide the motion state by combining the angular velocity to obtain motion subdivision link information.
Among the above modules, for example, preferably, the physiological parameter setting module 10 is completed by interaction with the cloud terminal through a smart phone of the user, the processing of the exercise state solving module 30 and the exercise link judging module 50 is completed at the wearable device side, and the processing of the dimensionality reduction solving module 40 is completed at the cloud terminal.
For the application of fall detection and common human behavior analysis, the acceleration sensor can well distinguish the motion and static states of human behaviors, and similar motion behaviors are difficult to distinguish. The wrist angular velocity is combined to further distinguish, but the acceleration and the angular velocity are calculated simultaneously, the calculation amount is large, the timeliness and the electric quantity power consumption are affected, and therefore the calculation amount and the accuracy are considered by adopting a mode of hierarchical classification, dimension reduction classification and key verification. As shown in fig. 2, the human motion may be divided into different levels.
In this embodiment, the motion state solving module firstly uses the acceleration sensor data to perform a first-level motion judgment on the intelligent belt triaxial acceleration sensor, and collects the acting forces in the x, y and z directions according to a sampling frequency of 100Hz (the walking frequency of a person is generally 110 steps/minute (1.8Hz), the running frequency does not exceed 5Hz, and the sampling frequency of 100Hz can accurately reflect the acceleration change.
In this embodiment, the motion state solving module counts the frequency of occurrence of the peak through the pair of trajectories secondly. In horizontal movement of the user, the vertical and forward accelerations may exhibit periodic variations. In the walking and foot-receiving action, the gravity center is upward, and only one foot touches the ground, the vertical acceleration tends to increase in a positive direction, then the gravity center is moved downwards, and the two feet touch the bottom, and the acceleration is opposite. The horizontal acceleration decreases when the foot is retracted and increases when the stride is taken, as shown in fig. 3.
It is worth mentioning that in walking exercise, the acceleration generated by vertical and forward motion is approximately sinusoidal with time and has a peak at a certain point where the acceleration in the vertical direction changes most.
And finally, the motion state solving module is used for filtering the data. Because the electromagnetic interference in the circuit is a main interference source in the acquisition process, the electromagnetic interference is high-frequency noise; the human motion is mainly low-frequency signals within 50Hz, and the wavelet transform threshold method is selected. For such interference, the detection is filtered by adding a threshold and a step frequency judgment, that is, the time interval of two adjacent steps is at least more than 0.2 seconds, and high-frequency noise is filtered.
Further, the motion segment determination module is configured to:
acquiring acceleration values a of the acceleration sensor in the directions of the x, y and z axesx、ay、azAnd solving the acceleration signal vector modulus SVMA(ii) a Collecting angular velocity values w of the angular velocity sensor in the directions of three axes x, y and zx、wy、wzAnd solving the vector modulus SVM of the angular velocity signalW(ii) a The above-mentioned
Vector modulo SVM by acceleration signalAVector mode SVM for angular velocity signalWAnd establishing an identification model which is perfect in self-adaptation, subdividing the motion state and obtaining motion subdivision link information.
Since the acceleration is suitable for the judgment of the motion with definite direction, the judgment of the incapability of falling detection, motion cycle links, splayfoot and the like needs to be carried out by using the angular velocity.
Based on the principle of a kinematic algorithm, four gait event time phases are detected: a gait cycle is divided into two phases, a "support phase" and a "swing phase".
Perry doctors at the national rehabilitation center for RLA, California, USA put forward the RLA staging method according to the occurrence sequence of the walking cycle; the support period is divided into 5 stages; the stride period is broken down into 3 epochs.
1. First touchdown, initialcontact: as the starting point of the walking cycle and the support period; the moment when the heel or other parts of the sole of the foot first contact the ground. The first landing mode for normal people walking is heel landing.
2. Load bearing reaction period, loadingresponse: the foot bottom is in full contact with the ground for a period of time after the heel is grounded; namely, when the heel at one side is grounded and the toe of the lower limb at the opposite side is lifted off; is the process of transferring the center from the heel to the sole. This phase is 0-15% of the gait cycle.
3. Mid-stance, mid-stance: when the finger is lifted from the lower limb at the opposite side to the trunk right above the leg at the side; the center of gravity is now directly above the support surface. This period is 15% -40% of the gait cycle:
4. late stance, terminalstance: the fingers are from the time the support heel lifts off to the time the contralateral lower limb heel lands. This period is 40% -50% of the gait cycle.
5. In the early stage of stepping, pre-swing: the fingers are held for a period of time from heel-strike of the contralateral lower limb until toe-off support. This period is 50% -60% of the gait cycle.
6. Initial step, initial stroke: from the point at which the supporting leg lifts to the point at which the knee joint reaches maximum flexion. This period is 60% -70% of the gait cycle.
7. Mid-swing: from the maximum flexion swing of the knee joint to when the lower leg is perpendicular to the ground. This period is 70% -85% of the gait cycle.
8. At the end of the step, terminating: the lower leg, which is perpendicular to the ground, swings forward until the heel lands again. This period is 85% -100% of the gait cycle.
In this embodiment, the links may be divided by time in the exercise cycle, and certainly, each exercise link may be determined by artificially setting the time of each link.
In the present embodiment, an acceleration signal vector mode and an angular velocity signal vector mode are used as input features of the model.
The analysis module builds a recognition model with perfect self-adaptation, mainly carries out modeling through an acceleration sensor rule, gives a motion state recognition result through the operation of a genetic operator, and is used for analyzing and managing remote health big data.
The handheld device has a low amplitude and a quick twitching state, or what is commonly called hand trembling, or a mischief user wants to simulate walking by quickly and repeatedly shaking the device for a short time, and the accurate value of step counting can be influenced if the interference data are not eliminated.
Carrying out wavelet transformation operations of three steps of wavelet decomposition, high-frequency wavelet coefficient processing and wavelet reconstruction on the collected data in each direction, discretizing time domain signals in each direction, decomposing mixed signals of various frequency components into different frequency bands, and then processing according to different characteristics of each seed signal on a frequency domain and frequency bands; and acquiring gait data with high signal to noise ratio.
Falls are characterized by large acceleration and peak angular velocity because the SVM peak due to collisions with low-lying objects during falls is larger than most common procedures of walking, ascending stairs, etc. in daily activities. However, the process of the human body movement behavior has complexity and randomness, and great misjudgment can be brought by using single acceleration related information to judge the occurrence of the human body falling behavior. Information thresholding using a combination of SVMA and SVMW can distinguish between falls and low intensity motions that produce smaller SVM peaks. Through analyzing the experimental result data SVMA and SVMW in the human body falling process and other daily life behavior processes, the SVMAT 20m/s2 is taken as the acceleration signal vector modulus threshold value and the SVMWT 4rad/s is taken as the angular velocity signal vector modulus threshold value for recognizing falling.
In this embodiment, the motion link determination module further includes a filtering unit, where the filtering unit performs wavelet transform operations of three steps of wavelet decomposition, high-frequency wavelet coefficient processing, and wavelet reconstruction on the acquired data of the acceleration sensor and the angular velocity sensor, discretizes time domain signals in each direction, decomposes a mixed signal of multiple frequency components into different frequency bands, and then processes the mixed signal according to different characteristics of each seed signal in the frequency domain and according to frequency bands; and acquiring gait data with high signal to noise ratio.
The wavelet transform adopts a hard threshold method, and the wavelet coefficient is Cj,kThe threshold is lambda;
the above-mentioned
Further, the dimension reduction solving module is configured to:
acquiring physiological parameter data of a user, and reducing the dimension of the physiological parameter data of the user;
taking the standard deviation of the whole sample population, taking N as the sample size, training a classifier, and identifying gait samples by using the classifier;
comprehensively calculating the course of a certain gait deviating from the arrangement crowdDegree x of
Wherein, ai、bi、ciAcceleration in the x, y, z axis directions respectively,the acceleration of the whole population in the directions of the x axis, the y axis and the z axis of a certain gait link is respectively.
Inputting the personal gait samples of N types registered in the database into a classifier for training, judging which type is (1, N) according to an input value, if the input value exceeds the range of (1, N), newly registering the type of N +1, and then updating the classifier again;
and on the basis of the different motion division, the same motion is subdivided again, and a classification result is determined by adopting a voting mode.
In this embodiment, in order to eliminate the influence of different dimensions and different orders of magnitude between data, the multidimensional signal needs to normalize the raw data to make it comparable, and each variable is normalized to a matrix X with a mean value of 0 and a variance of 1 by subtracting the mean value of the column from each element in the proof and then dividing by the standard deviation of the column.
Further, the dimension reduction solving module is further configured to:
the raw data is normalized by subtracting the mean of the column from each element in the proof and dividing by the standard deviation of the column, such that each variable is normalized to a matrix X with a mean of 0 and a variance of 1, i.e., X ═ X1，X2，......Xn]T＝[Xij](n×p)；
Wherein,
to obtain
Solving a correlation coefficient matrix:
where R is a real symmetric matrix (i.e., R)ij＝rji) Wherein r is a correlation coefficient;
solving a correlation coefficient matrix:
if the accumulated contribution rate reaches more than 50%, a ratio height method is adopted, the characteristic value vector of the highest contribution rate is left as a fixed working sample set, and the rest is discarded;
calculating a partial matrix, using the remained characteristic value as a new variable principal component, and calculating a partial matrix F by using the following formula(n×m)＝X(n×p)·U(p×m)；
Wherein X is an original data matrix, U is a principal component load, and a score matrix F is a result obtained after dimensionality reduction;
the method not only uses the characteristic of high calculation speed of the fixed working sample set method, but also avoids the problems that the number of the vectors exceeds the scale of the working sample set, and the algorithm only optimizes one part of the support vectors and has range limitation. The abnormal people are selected by the method.
Continuous training subdivision is performed in combination with big data: fitness function f (x) of SVM classifieri)＝min(1-g(xi))，And dividing the sample into correct rates for the SVM classifier.
As shown in fig. 4, in the present embodiment, the gait of the movement of the human body can be effectively determined by using the output data of the accelerometer, using a three-condition (C1, C2 and C3) determination algorithm, and using a median filtering method, where the state "0" represents movement and the state "1" represents rest.
Further, the dimension reduction solving module is further configured to:
calculating a first condition, a second condition and a third condition by using the output data of the acceleration sensor, and judging the gait of the human motion by using median filtering;
the first condition is: the accelerometer outputs a synthesized amplitude value, and if the synthesized amplitude value is between the given upper threshold value and the given lower threshold value, the human body is judged to be static; otherwise, judging the motion of the person;
the accelerometer output composite amplitude is:
the upper and lower thresholds are respectively: th (h)a min＝8m/s，tha max＝11m/s；
The first condition is expressed as:
the second condition is: if the local variance output by the accelerometer is lower than a given threshold value, the human body is judged to be static; otherwise, judging the motion of the person;
the local variance of the accelerometer output is:
whereinFor the interval, the accelerometer synthesizes an output average value of the amplitude, and the expression is as follows:
s is the number of half-window samples, and a given threshold is defined as: th (h)σa＝0.5m/s2The second condition is represented as:
the third condition is: the angular velocity sensor outputs an angular velocity composite amplitude, and if the angular velocity composite amplitude is lower than a given threshold value, the human body is judged to be static; otherwise, judging the motion of the person;
define the composite amplitude of the gyroscope output as:
the given thresholds are: th (h)w maxThe third condition is expressed as:
further, the intelligent waistband based on the artificial intelligence layered and graded motion recognition method further comprises a model self-adaptive perfecting module; the model adaptation refinement module is configured to:
reading a new input sample, and calculating the recognition rate of the SVM classifier according to a cross verification method;
if the current recognition rate of the training is higher than or equal to the original recognition rate, setting the parameters of the training as the optimal parameters; otherwise, a selection operation, a crossover operation and/or a mutation operation are executed, and the training parameters are further optimized.
Specifically, with the increase of the sample size, the SVM classifier can be adaptively and continuously optimized and perfected:
(1) sampling calculation of motion state
During the judgment, the standard deviation of one state is larger, and the standard deviation of the other state is smaller and exactly balanced, so that no abnormality is found, and random sampling verification is performed again.
And (4) inputting a new sample every time, and calculating the recognition rate of the SVM classifier according to the cross verification method principle.
(2) And for the characteristic values of the samples which are not found to be abnormal, using an SVM classifier fitness function to divide the accuracy of the samples for the SVM classifier. The parallel execution process is simulated by maintaining a plurality of groups and appropriately controlling the interaction between the groups, thereby improving the execution efficiency of the algorithm even without using a parallel computer.
As shown in fig. 5, each time a new sample is input, the recognition rate of the SVM classifier is calculated according to the principle of the cross validation method, fitness evaluation is performed, the termination value of the genetic algorithm is not set, the termination condition adopts a proportion method, if the recognition rate of training is higher than the existing one, the training parameter is set as the optimal parameter, otherwise, the training parameter is further optimized by performing operations such as selection, crossing and mutation.
Preferably, the sampling rate is adaptively adjusted according to the motion change rate, and the multipath transmission preprocesses the image: the rate of motion of the object being transmitted is proportional to the value of the feature required to restore the image. Such as: the object being photographed is stationary or substantially motionless and the sample rate of the transmission is proportionally reduced. Meanwhile, data is transmitted through multiple paths, the transmission capability of a network can be fully utilized, multiple routes in the route cache are used at the same time, and the setting weight of each link is adjusted according to the change rate of transmission objects of different paths to perform load balancing.
Preferably, when sending a data packet, according to the number of the sub-images decomposed after preprocessing in step 2, allocating the same number of link paths, polling in different paths by a time slice rotation method, polling the amount of time of each link, setting a weight for each link i by the dynamically changed RTT of each link i, and allocating the following relationship in each 20ms time slice, where n is the number of a certain link feature point, and is the acceleration of the feature point in three directions, and each link sends the preprocessed feature value that is currently sampled most recently after sending the previous feature.
Preferably, for the calculation sub-result merging process: respectively reading the output files of all processes in the current time step by the main process, performing splicing processing on the collected multi-channel video streams to generate a panoramic video stream carrying the timestamp, merging and restoring the result according to a regional decomposition algorithm, and temporarily storing the result in an ASCLL format; when a user wears the virtual reality terminal, whether the virtual reality terminal is in a motion state is detected, if so, video frequency frames to be played are adjusted according to the acceleration so as to provide synchronous video information for the user, and the information is displayed in a visual area of the virtual reality terminal.
Preferably, when the intra-frame delay refers to the rotation of the head of the user, the pixel points forming the picture frame jump back to the original point at the end of each frame, at this time, the image of the previous frame and the image of the previous frame are retained by the user persistence phenomenon, a smear phenomenon is generated, the inter-frame delay cannot exceed 20ms, and otherwise, a very obvious smear feeling is generated. To reduce the delay rate; the part exceeding 20ms is combined with the motion state to make up the prediction, thereby achieving the picture pause effect.
The sampling rate is adaptively adjusted according to the motion change rate, and the images are preprocessed through multipath transmission: the rate of motion of the object being transmitted is proportional to the value of the feature required to restore the image. Such as: the object being photographed is stationary or substantially motionless and the sample rate of the transmission is proportionally reduced. Meanwhile, data is transmitted through multiple paths, the transmission capability of a network can be fully utilized, multiple routes in the route cache are used at the same time, and the setting weight of each link is adjusted according to the change rate of transmission objects of different paths to perform load balancing.
When sending data packets, distributing the same number of link paths according to the number of the sub-images to be disassembled after preprocessing in the step 2, polling in different paths by a time slice rotation method, polling the time of staying each link, and setting a weight W for each link i by the dynamically changed RTT of each link ii: the allocation relationship of each 20ms time slice is as follows, wherein n is the number of certain link characteristic points, and Xi，Yi，ZiFor the acceleration of the feature point in three directions, after the previous feature is sent, each link sends the preprocessed feature value which is sampled latest at that time.
And (3) merging the calculation sub-results: respectively reading the output files of all processes in the current time step by the main process, performing splicing processing on the collected multi-channel video streams to generate a panoramic video stream carrying the timestamp, merging and restoring the result according to a regional decomposition algorithm, and temporarily storing the result in an ASCLL format;
when a user wears the virtual reality terminal, whether the virtual reality terminal is in a motion state is detected, if so, video frequency frames to be played are adjusted according to the acceleration so as to provide synchronous video information for the user, and the information is displayed in a visual area of the virtual reality terminal.
The intra-frame delay refers to that when the head of a user rotates, pixel points forming a picture frame jump back to the original point at the end of each frame, at the moment, the image of the previous frame and the image of the previous frame are kept by the user vision persistence phenomenon, the smear phenomenon is generated, the inter-frame delay cannot exceed 20ms, and otherwise, the obvious smear feeling is generated. To reduce the delay rate; the part exceeding 20ms is combined with the motion state to make up the prediction, thereby achieving the picture pause effect.
The invention has the beneficial effects that: the scheme overcomes the two contradictions, on one hand, fine motion characteristics are comprehensively collected in the preprocessing process, on the other hand, only limited different sensor functions are called in different specific judging processes through self-adaptive algorithm matching, different specific characteristic calculation is extracted, different characteristic dimension reduction sampling is integrated in an analyzing link, the amount of exercise is greatly reduced, and the accuracy is guaranteed through a key calculation verification link. Therefore, the detailed characteristics of the motion state are comprehensively collected, the calculated amount and the power consumption are greatly reduced, the medical reference value is improved, and the power consumption standby capability is extracted under the same condition.
On one hand, hierarchical modeling can be performed in virtual reality motion, and time delay and requirements on a processor are reduced. On the other hand, the waistband can also be used for daily health big data service and adjusting sitting posture and spine health. Meanwhile, the health care is based on the familiarity interaction and artificial intelligence health care of the health big data user portrait.
In the invention, on one hand, fine motion characteristics are comprehensively collected in the preprocessing process, on the other hand, only limited different sensor functions are called in different specific judging processes through self-adaptive algorithm matching, different specific characteristic calculation is extracted, and different characteristic dimension reduction sampling is integrated in the analyzing link, so that the calculated amount is greatly reduced, and the accuracy is ensured through the key calculation verification link; meanwhile, the key verification link is placed at the cloud end, and the rest of calculation is placed at the wearable device end, so that the contradiction between quick response and calculation amount is effectively balanced, the accuracy is ensured, and the calculation amount of the wearable device end is not too large.
Therefore, the invention can comprehensively collect the detailed characteristics of the motion state, greatly reduce the calculated amount and the power consumption, improve the medical reference value and extract the power consumption standby capability under the same condition.
In addition, it should be noted that the terms "first", "second", "third", and the like in the specification are used for distinguishing various components, elements, steps, and the like in the specification, and are not used for representing a logical relationship or a sequential relationship between the various components, elements, steps, and the like, unless otherwise specified.
It is to be understood that while the present invention has been described in conjunction with the preferred embodiments thereof, it is not intended to limit the invention to those embodiments. It will be apparent to those skilled in the art from this disclosure that many changes and modifications can be made, or equivalents modified, in the embodiments of the invention without departing from the scope of the invention. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention are still within the scope of the protection of the technical solution of the present invention, unless the contents of the technical solution of the present invention are departed.
Claims (10)
1. A method for improving mixed reality and daily health monitoring through artificial intelligence motion hierarchical processing is characterized by comprising the following steps:
according to the correlation between the motion state judged by the belt of a person and the video image requirements of VR glasses, a part of the VR glasses which needs to be displayed most clearly is established, and according to motion prediction, visual compensation is carried out on the image of which the transmission delay exceeds a threshold time period within the threshold time period range, and an established belt motion and VR video correlation model is established;
according to the correlation between the motion state judged by the personal waistband and the video image requirements of VR glasses, establishing a part needing the clearest display in the VR glasses, and performing visual compensation on the image with the transmission delay exceeding a threshold time period within the threshold time period according to motion prediction, and establishing a waistband motion and VR video correlation model;
the system is used for setting image prejudgment setting of individual VR glasses, establishing a part needing clearest display in the VR glasses according to the correlation between the motion state judged by the waistband and the video image requirement of the VR glasses, and performing visual compensation on the image of which the transmission delay exceeds a threshold time period within the range of the threshold time period according to motion prediction;
according to the correlation between the motion state judged by the waistband and the video image requirements of the VR glasses, establishing the part of the VR glasses which needs to be displayed most clearly, and performing visual compensation on the image of which the transmission delay exceeds a threshold time period within the range of the threshold time period according to motion prediction; verifying the compensation and data transmitted after a threshold time period, setting an accuracy factor to be 1 when the consistency exceeds a preset percentage, and setting the accuracy factor to be 0 when the consistency is lower than the preset percentage;
collecting the belt sensor motion data of the user under the specific combination of the motion state and the gait;
performing preprocessing, combining different levels of data by using an optimized support vector machine in a hierarchical level manner, collecting acceleration data output by an acceleration sensor worn by a user at regular time, filtering high-frequency noise through wavelet transformation, matching through a self-adaptive algorithm, only calling limited different sensor functions to work in different specific judging processes, extracting different specific feature calculation, integrating different feature dimension reduction sampling in an analysis link, and ensuring the correctness through a key calculation verification link; meanwhile, a key verification link is placed at the cloud end, and the rest of calculation is placed at the wearable equipment end;
the data after the motion preprocessing and the data obtained by executing the preprocessing are entered into a training set;
and carrying out accuracy judgment on the data after the motion preprocessing and the data obtained by executing the preprocessing.
2. The method for artificial intelligence motion hierarchy-based hierarchical processing to promote mixed reality and daily health monitoring of claim 1, wherein the motion state includes a first cyclic action and a second acyclic action; the first cyclical action comprises walking and running; the second non-cyclical action comprises jumping, rising, sitting, and squatting; moreover, the walking includes going upstairs, going downstairs and walking on flat ground.
3. The method for improving mixed reality and daily health monitoring through artificial intelligence motion hierarchical processing according to claim 1 or 2, wherein the motion segment judgment module is configured to:
acquiring acceleration values a of the acceleration sensor in the directions of the x, y and z axesx、ay、azAnd solving the acceleration signal vector modulus SVMA(ii) a Collecting angular velocity values w of the angular velocity sensor in the directions of three axes x, y and zx、wy、wzAnd solving the vector modulus SVM of the angular velocity signalW(ii) a The above-mentionedVector modulo SVM by acceleration signalAVector mode SVM for angular velocity signalWAnd establishing an identification model which is perfect in self-adaptation, subdividing the motion state and obtaining motion subdivision link information.
4. The method for improving mixed reality and daily health monitoring through artificial intelligence motion hierarchical processing according to claim 1 or 2, wherein the motion link judgment module further comprises a filtering unit, the filtering unit performs wavelet transformation operation of three steps of wavelet decomposition, high-frequency wavelet coefficient processing and wavelet reconstruction on collected data of the acceleration sensor and the angular velocity sensor, discretizes time domain signals in each direction, decomposes mixed signals of multiple frequency components into different frequency bands, and processes the mixed signals according to different characteristics of each seed signal in a frequency domain and frequency bands; and acquiring gait data with high signal to noise ratio.
5. The method for artificial intelligence motion hierarchy for enhancing mixed reality and daily health monitoring of claim 1 or 2, wherein the dimension reduction solving module is further configured to:
received data is normalized by subtracting the mean of the column from each element in the proof and dividing by the standard deviation of the column, such that each variable is normalized to a matrix X with a mean of 0 and a variance of 1, i.e., X ═ X1，X2，......Xn]T＝[Xij](n×p)；
Wherein,i＝1，2…n，j＝1，2…p；
to obtain
Solving a correlation coefficient matrix:
where R is a real symmetric matrix (i.e., R)ij＝rji) Wherein r is a correlation coefficient;
solving a correlation coefficient matrix:
if the accumulated contribution rate reaches more than 50%, a ratio height method is adopted, the characteristic value vector of the highest contribution rate is left as a fixed working sample set, and the rest is discarded;
calculating a partial matrix, using the remained characteristic value as a new variable principal component, and calculating a partial matrix F by using the following formula(n×m)＝X(n×p)·U(p×m)；
Wherein X is the original data matrix, U is the principal component load, and the scoring matrix F is the result obtained after dimensionality reduction.
6. The method for artificial intelligence motion hierarchy for enhancing mixed reality and daily health monitoring of claim 1 or 2, wherein the dimension reduction solving module is further configured to:
calculating a first condition, a second condition and a third condition by using the output data of the acceleration sensor, and judging the gait of the human motion by using median filtering;
the first condition is: the accelerometer outputs a synthesized amplitude value, and if the synthesized amplitude value is between the given upper threshold value and the given lower threshold value, the human body is judged to be static; otherwise, judging the motion of the person;
the accelerometer output composite amplitude is:
the upper and lower thresholds are respectively: th (h)amin＝8m/s，thamax＝11m/s；
The first condition is expressed as:
the second condition is: if the local variance output by the accelerometer is lower than a given threshold value, the human body is judged to be static; otherwise, judging the motion of the person;
the local variance of the accelerometer output is:
whereinFor the interval, the accelerometer synthesizes an output average value of the amplitude, and the expression is as follows:
s is the number of half-window samples, and a given threshold is defined as: th (h)σa＝0.5m/s2The second condition is represented as:
the third condition is: the angular velocity sensor outputs an angular velocity composite amplitude, and if the angular velocity composite amplitude is lower than a given threshold value, the human body is judged to be static; otherwise, judging the motion of the person;
define the composite amplitude of the gyroscope output as:
the given thresholds are: th (h)wmaxThe third condition is expressed as:
7. the method for improving mixed reality and daily health monitoring through artificial intelligence motion hierarchical processing according to claim 1 or 2, wherein the sampling rate is adaptively adjusted according to the motion change rate, and the images are preprocessed through multipath transmission: the rate of motion of the object being transmitted is proportional to the value of the feature required to restore the image.
8. The method for improving mixed reality and daily health monitoring through artificial intelligence motion hierarchical processing according to claim 1 or 2, characterized in that when a data packet is sent, the same number of link paths are allocated according to the number of the sub-images disassembled after preprocessing in step 2, polling is performed in different paths through a time slice rotation method, the polling duration of each link is the same, a weight is set for each link i by the dynamically changed RTT of each link i, the allocation relation of each 20ms time slice is as follows, wherein n is the number of a certain link feature point, the acceleration of the feature point in three directions is obtained, and after the previous feature is sent by each link, the preprocessed feature value sampled latest at that time is sent.
9. The method for improving mixed reality and daily health monitoring through artificial intelligence motion hierarchical processing according to claim 1 or 2, wherein the calculation sub-results are merged: respectively reading the output files of all processes in the current time step by the main process, performing splicing processing on the collected multi-channel video streams to generate a panoramic video stream carrying the timestamp, merging and restoring the result according to a regional decomposition algorithm, and temporarily storing the result in an ASCLL format;
when a user wears the virtual reality terminal, whether the virtual reality terminal is in a motion state is detected, if so, video frequency frames to be played are adjusted according to the acceleration so as to provide synchronous video information for the user, and the information is displayed in a visual area of the virtual reality terminal.
10. The method according to claim 1 or 2, wherein the intra-frame delay refers to the rotation of the user's head, the pixel points constituting the frame jump back to the origin at the end of each frame, and at this time, the user's persistence of vision retains the previous frame and the image of the previous frame, which causes a smear phenomenon, and the inter-frame delay cannot exceed 20ms, otherwise, a very obvious smear feeling is generated. To reduce the delay rate; the part exceeding 20ms is combined with the motion state to make up the prediction, thereby achieving the picture pause effect.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810295931.7A CN108514421A (en) | 2018-03-30 | 2018-03-30 | The method for promoting mixed reality and routine health monitoring |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810295931.7A CN108514421A (en) | 2018-03-30 | 2018-03-30 | The method for promoting mixed reality and routine health monitoring |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108514421A true CN108514421A (en) | 2018-09-11 |
Family
ID=63431390
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810295931.7A Pending CN108514421A (en) | 2018-03-30 | 2018-03-30 | The method for promoting mixed reality and routine health monitoring |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108514421A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110390565A (en) * | 2019-07-23 | 2019-10-29 | 厦门市佳音在线股份有限公司 | The method and system of intelligent gateway adaptive management are realized by AI edge calculations |
| CN111728618A (en) * | 2020-05-21 | 2020-10-02 | 深圳市千手千眼科技有限公司 | Human body movement gait detection method for personnel positioning |
| CN111916179A (en) * | 2019-05-08 | 2020-11-10 | 北京明熹一品电子商务有限公司 | Method for carrying out 'customized' diet nourishing model based on artificial intelligence self-adaption individual physical sign |
| CN111990967A (en) * | 2020-07-02 | 2020-11-27 | 北京理工大学 | Gait-based Parkinson disease recognition system |
| CN114098719A (en) * | 2021-11-30 | 2022-03-01 | 歌尔科技有限公司 | Action detection method and device and wearable device |
| CN115470364A (en) * | 2022-11-03 | 2022-12-13 | 京华信息科技股份有限公司 | Method and system for monitoring specific personnel based on knowledge graph |
| CN116746910A (en) * | 2023-06-15 | 2023-09-15 | 广州医科大学附属脑科医院 | Gait monitoring method and device based on wearable equipment and wearable equipment |
| CN117011244A (en) * | 2023-07-07 | 2023-11-07 | 中国人民解放军西部战区总医院 | Wrist multispectral image processing method |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5629594A (en) * | 1992-12-02 | 1997-05-13 | Cybernet Systems Corporation | Force feedback system |
| CN101034308A (en) * | 2007-04-10 | 2007-09-12 | 南京航空航天大学 | Helmet display virtual vision time delay compensation system and method based on human body surface myoelectric information |
| CN105137600A (en) * | 2015-09-02 | 2015-12-09 | 胡东海 | Virtual reality helmet and visual sense compensation method |
| CN105511603A (en) * | 2015-11-25 | 2016-04-20 | 小米科技有限责任公司 | Equipment control method and device |
| CN105824416A (en) * | 2016-03-16 | 2016-08-03 | 成都电锯互动科技有限公司 | Method for combining virtual reality technique with cloud service technique |
| CN105976432A (en) * | 2016-07-13 | 2016-09-28 | 顽石运动智能科技（北京）有限公司 | Football virtual system |
| CN106598222A (en) * | 2016-11-14 | 2017-04-26 | 上海斐讯数据通信技术有限公司 | Scene mode switching method and system |
| CN106937531A (en) * | 2014-06-14 | 2017-07-07 | 奇跃公司 | Method and system for producing virtual and augmented reality |
| CN106971059A (en) * | 2017-03-01 | 2017-07-21 | 福州云开智能科技有限公司 | A kind of wearable device based on the adaptive health monitoring of neutral net |
| CN107016411A (en) * | 2017-03-28 | 2017-08-04 | 北京犀牛数字互动科技有限公司 | Data processing method and device |
| CN107015646A (en) * | 2017-03-28 | 2017-08-04 | 北京犀牛数字互动科技有限公司 | The recognition methods of motion state and device |
| CN107018399A (en) * | 2017-03-28 | 2017-08-04 | 北京犀牛数字互动科技有限公司 | Image data transfer method and device |
| CN107102728A (en) * | 2017-03-28 | 2017-08-29 | 北京犀牛数字互动科技有限公司 | Display methods and system based on virtual reality technology |
| CN107102720A (en) * | 2009-12-23 | 2017-08-29 | 谷歌技术控股有限责任公司 | For visual compensation and equipment |
| CN107632710A (en) * | 2013-03-15 | 2018-01-26 | 奇跃公司 | Display system and method |
| KR20180013438A (en) * | 2016-07-29 | 2018-02-07 | 정우락 | Apparatus for Shooting Game in Virtual Reality |
| CN107773254A (en) * | 2017-12-05 | 2018-03-09 | 苏州创捷传媒展览股份有限公司 | A kind of method and device for testing Consumer's Experience |
-
2018
- 2018-03-30 CN CN201810295931.7A patent/CN108514421A/en active Pending
Patent Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5629594A (en) * | 1992-12-02 | 1997-05-13 | Cybernet Systems Corporation | Force feedback system |
| CN101034308A (en) * | 2007-04-10 | 2007-09-12 | 南京航空航天大学 | Helmet display virtual vision time delay compensation system and method based on human body surface myoelectric information |
| CN107102720A (en) * | 2009-12-23 | 2017-08-29 | 谷歌技术控股有限责任公司 | For visual compensation and equipment |
| CN107632710A (en) * | 2013-03-15 | 2018-01-26 | 奇跃公司 | Display system and method |
| CN106937531A (en) * | 2014-06-14 | 2017-07-07 | 奇跃公司 | Method and system for producing virtual and augmented reality |
| CN105137600A (en) * | 2015-09-02 | 2015-12-09 | 胡东海 | Virtual reality helmet and visual sense compensation method |
| CN105511603A (en) * | 2015-11-25 | 2016-04-20 | 小米科技有限责任公司 | Equipment control method and device |
| CN105824416A (en) * | 2016-03-16 | 2016-08-03 | 成都电锯互动科技有限公司 | Method for combining virtual reality technique with cloud service technique |
| CN105976432A (en) * | 2016-07-13 | 2016-09-28 | 顽石运动智能科技（北京）有限公司 | Football virtual system |
| KR20180013438A (en) * | 2016-07-29 | 2018-02-07 | 정우락 | Apparatus for Shooting Game in Virtual Reality |
| CN106598222A (en) * | 2016-11-14 | 2017-04-26 | 上海斐讯数据通信技术有限公司 | Scene mode switching method and system |
| CN106971059A (en) * | 2017-03-01 | 2017-07-21 | 福州云开智能科技有限公司 | A kind of wearable device based on the adaptive health monitoring of neutral net |
| CN107102728A (en) * | 2017-03-28 | 2017-08-29 | 北京犀牛数字互动科技有限公司 | Display methods and system based on virtual reality technology |
| CN107018399A (en) * | 2017-03-28 | 2017-08-04 | 北京犀牛数字互动科技有限公司 | Image data transfer method and device |
| CN107015646A (en) * | 2017-03-28 | 2017-08-04 | 北京犀牛数字互动科技有限公司 | The recognition methods of motion state and device |
| CN107016411A (en) * | 2017-03-28 | 2017-08-04 | 北京犀牛数字互动科技有限公司 | Data processing method and device |
| CN107773254A (en) * | 2017-12-05 | 2018-03-09 | 苏州创捷传媒展览股份有限公司 | A kind of method and device for testing Consumer's Experience |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111916179A (en) * | 2019-05-08 | 2020-11-10 | 北京明熹一品电子商务有限公司 | Method for carrying out 'customized' diet nourishing model based on artificial intelligence self-adaption individual physical sign |
| CN110390565A (en) * | 2019-07-23 | 2019-10-29 | 厦门市佳音在线股份有限公司 | The method and system of intelligent gateway adaptive management are realized by AI edge calculations |
| CN111728618A (en) * | 2020-05-21 | 2020-10-02 | 深圳市千手千眼科技有限公司 | Human body movement gait detection method for personnel positioning |
| CN111990967A (en) * | 2020-07-02 | 2020-11-27 | 北京理工大学 | Gait-based Parkinson disease recognition system |
| CN114098719A (en) * | 2021-11-30 | 2022-03-01 | 歌尔科技有限公司 | Action detection method and device and wearable device |
| CN115470364A (en) * | 2022-11-03 | 2022-12-13 | 京华信息科技股份有限公司 | Method and system for monitoring specific personnel based on knowledge graph |
| CN116746910A (en) * | 2023-06-15 | 2023-09-15 | 广州医科大学附属脑科医院 | Gait monitoring method and device based on wearable equipment and wearable equipment |
| CN116746910B (en) * | 2023-06-15 | 2024-05-28 | 广州医科大学附属脑科医院 | Gait monitoring method and device based on wearable equipment and wearable equipment |
| CN117011244A (en) * | 2023-07-07 | 2023-11-07 | 中国人民解放军西部战区总医院 | Wrist multispectral image processing method |
| CN117011244B (en) * | 2023-07-07 | 2024-03-22 | 中国人民解放军西部战区总医院 | Wrist multispectral image processing method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108514421A (en) | The method for promoting mixed reality and routine health monitoring | |
| CN106971059B (en) | Wearable equipment based on neural network self-adaptation health monitoring | |
| CN108244744B (en) | Motion state identification method, sole and shoe | |
| KR102107379B1 (en) | Method for Prediction Frailty Using Triple Axis Motion Meter, Prediction Frailty System using Triple Axis Motion Meter and Wearable Prediction Frailty Device | |
| CN109069066A (en) | Wearable and connection gait analysis system | |
| CN109805935A (en) | A kind of intelligent waistband based on artificial intelligence hierarchical layered motion recognition method | |
| CN110292378A (en) | Depression remote rehabilitation system based on the monitoring of E.E.G closed loop | |
| Lipperts et al. | Clinical validation of a body-fixed 3D accelerometer and algorithm for activity monitoring in orthopaedic patients | |
| Mokaya et al. | Myovibe: Vibration based wearable muscle activation detection in high mobility exercises | |
| CN109793500B (en) | Knee joint load mechanics analytical equipment | |
| CN110390565A (en) | The method and system of intelligent gateway adaptive management are realized by AI edge calculations | |
| CN108958482B (en) | Similarity action recognition device and method based on convolutional neural network | |
| CN109045682A (en) | A method of it reducing projection mobile phone and interacts body-building game propagation delay time with intelligent shoe | |
| Nguyen et al. | IMU-based spectrogram approach with deep convolutional neural networks for gait classification | |
| CN113974612A (en) | Automatic assessment method and system for upper limb movement function of stroke patient | |
| Tulipani et al. | The sit-to-stand transition as a biomarker for impairment: Comparison of instrumented 30-second chair stand test and daily life transitions in multiple sclerosis | |
| Liu et al. | Deep learning based ground reaction force estimation for stair walking using kinematic data | |
| Datta et al. | Upper limb movement profiles during spontaneous motion in acute stroke | |
| CN108447562A (en) | A kind of user movement capability assessment method and system | |
| Cao et al. | ActiRecognizer: Design and implementation of a real-time human activity recognition system | |
| CN114010453A (en) | Feedback wearing system based on motion signal and preprocessing method | |
| CN115067934A (en) | Hand motion function analysis system based on machine intelligence | |
| Alamäki et al. | Wearable technology supported home rehabilitation services in rural areas: emphasis on monitoring structures and activities of functional capacity. Handbook | |
| CN112022163A (en) | Control method based on waist support waistband, waist support device and system thereof | |
| Lv et al. | Analysis of gait symmetry under unilateral load state |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180911 |
|
| RJ01 | Rejection of invention patent application after publication |