CN114469000B

CN114469000B - Human body fall-reducing and shock-preventing intelligent monitoring method and system based on multi-sensor data reinforcement learning

Info

Publication number: CN114469000B
Application number: CN202210102329.3A
Authority: CN
Inventors: 余海燕; 黄学达; 胡邓; 谢昊飞; 蔡宇翔; 尹彦臻; 邓兴媛; 范国慷; 周杰
Original assignee: Chongqing University of Post and Telecommunications
Current assignee: Chongqing University of Post and Telecommunications
Priority date: 2022-01-27
Filing date: 2022-01-27
Publication date: 2023-08-11
Anticipated expiration: 2042-01-27
Also published as: CN114469000A

Abstract

The invention belongs to the field of sensor data processing, and particularly relates to a human body fall-reducing and shock-preventing intelligent monitoring method and system based on multi-sensor data reinforcement learning, wherein the method comprises the following steps of: monitoring a monitored object in real time, and acquiring perception data of the monitored object by adopting a plurality of sensors; obtaining a human body falling related state according to the perception data; inputting the related state information into a semi-observed Markov decision process model to obtain an optimal execution strategy; executing corresponding actions according to the strategy execution arm system, wherein the actions comprise selecting an execution arm with the maximum protection function from execution arms of candidate parts, activating the execution arm and outputting an inflation command to achieve the effects of reducing the fall and preventing the shock; after the arm action is executed, sensor perception data are collected again, a human body falling related state is obtained, and then a next stage decision support process is carried out; the intelligent airbag intervention mode based on reinforcement learning can pertinently reduce the injury risk of falling for patients with different falling types.

Description

Human body fall-reducing and shock-preventing intelligent monitoring method and system based on multi-sensor data reinforcement learning

Technical Field

The invention belongs to the field of sensor data processing, and particularly relates to a human body fall-reducing and shock-preventing intelligent monitoring method and system based on multi-sensor data reinforcement learning.

Background

Innovations in health technology in the process of population aging have become hot spots in the relevant theoretical world and industry. Population aging begins in the western world of the 20 th century, which is unprecedented in human history. In china, the number of elderly people over 60 years has been on the rise between 2010 and 2050, with an increasing degree of aging of the population. In the case of unintentional injuries of the elderly, fall is an important factor in death of the elderly, since the elderly experience a decrease in both response and balance with age and a decrease in muscle strength. In which fall-related craniocerebral trauma, patient hospitalization rates are four times higher than those of community residents, and health is often worse. Therefore, falling is an important risk event frequently faced in the health care of the elderly, and high-quality and efficient strategic measures are urgently needed to prevent individuals and manage them finely.

At present, for the fall monitoring problem of the old, more scholars propose related solutions at present; if three-dimensional acceleration values of a human body are acquired by using a three-axis acceleration sensor, spatial positioning is performed after abnormal data appear, and finally, the position of the abnormal falling is transmitted in a wireless communication mode; the three-axis acceleration sensor is used for detecting the activity information of the old, the singlechip is used for collecting and storing data, the wavelet means is used for analyzing the data, and when the old is detected to fall, the terminal is automatically positioned and the accurate position is automatically and remotely alarmed in a short message mode. Almeida et al propose a cane with a gyroscope to measure the angular rate of movement of the elderly to determine if the elderly has fallen. However, the crutch falling may be caused by the reasons such as unintentional falling of the old, so that the old is judged to fall by the crutch falling to have a certain false alarm. Bourke AK et al propose a fall detection device wearable in front of the chest, which consists of two orthogonal gyroscopes for measuring the angular velocity and the change in body angle to determine if the elderly has fallen.

However, as can be seen from the existing research technology, the three-axis acceleration sensor is adopted as the core device for monitoring the fall of the old; the sensor is easy to be influenced by environmental factors and factors of the old people in the process of collecting the information of the old people, so that the collected information is in error, and monitoring is difficult; therefore, a solution for realizing effective early warning, timely response, reducing the collision in the falling process and relieving the injury caused by falling is urgently needed when the human body falls.

Disclosure of Invention

In order to solve the problems in the prior art, the invention provides a human body fall-reducing and shock-preventing intelligent monitoring method based on multi-sensor data reinforcement learning, which comprises the following steps:

s1: setting state transition probability and observation probability;

s2: monitoring the monitored object in real time, and sensing state data of the monitored object through a plurality of sensors;

s3: processing the sensing data of the multiple sensors according to the observation probability to obtain the relevant state information of human body falling;

s4: inputting the human body fall related state information, the observation probability and the state transition probability into a semi-observation Markov decision process model to obtain an optimal decision of a monitoring object at the current moment;

s5: inputting the optimal decision into an arm system, and executing a corresponding command by the arm system according to the optimal decision, wherein the command comprises selecting an execution arm with the maximum protection effect (return value) from the execution arms of the candidate part, activating the execution arm and outputting an inflation command;

s6: inflating the intelligent air bag according to the inflation command;

s7: after the action of the execution arm is finished, adopting a plurality of sensors to sense the state data of the monitoring object again, and obtaining the relevant state of human body falling according to the secondarily sensed data; and updating the return value according to the human body fall-related state, and returning to the step S2.

Preferably, the process of setting the state transition probability and the observation probability includes: the calculation process of the state transition probability comprises the steps of obtaining historical monitoring data of a monitored object, and carrying out statistical normalization processing on the historical monitoring data by adopting a statistical method to obtain the state transition probability; the observation probability calculation process includes classifying observation categories of the monitoring object into 9 categories, including: no signal, lumbar, upper left, upper right, head, lower left, lower right, spine sensor signal, and death signal; all observation sets correspond to the sign Ω= { o _N ,o _W ,o _LA ,o _RA ,o _H ,o _LL ,o _RL ,o _S ,o _D O, where o _N Indicating no signal, o _W Represents lumbar sensor signal, o _LA Representing left upper limb sensor signal o _RA Represents the upper right limb sensor signal, o _H Representing head sensor signals, o _LL Represents left lower limb sensor signal o _RL Represents the right lower limb sensor signal, o _S Representing spinal sensor signals, o _D A death signal; and acquiring historical monitoring data of the monitored object, and carrying out statistical normalization processing on the historical monitoring data by adopting a statistical method to obtain the observation probability of each observation category.

Preferably, the multi-sensor includes three hall acceleration sensors for measuring acceleration information of x-axis, y-axis and z-axis of the monitored object, and three angle measurement sensors for measuring attitude angles of the x-axis, y-axis and z-axis accelerations.

Preferably, obtaining the human fall-related status information includes:

s31: calculating the posture information of the monitored object according to the information acquired by the sensor; the posture information comprises four categories of normal walking, high risk of falling, falling and death;

s32: according to the monitored object posture information, determining a target of the monitored object state transition at the current moment by adopting state transition probability;

s33: and determining the state target of the monitored object at the current moment by adopting the observation probability according to the gesture information of the monitored object.

Further, the process of calculating the posture information of the monitored object includes:

step 1: establishing a space rectangular coordinate system by taking the right front of a monitored object as an x axis, the right left side as a y axis and the vertical direction as a z axis; setting an acceleration amplitude threshold, an x-axis acceleration attitude angle threshold, a y-axis acceleration attitude angle threshold and a downward acceleration threshold;

step 2: smoothing the collected acceleration of the x axis, the y axis and the z axis;

step 3: calculating the acceleration amplitude of the monitored object according to the smoothed data;

step 4: comparing the calculated acceleration amplitude with a set acceleration amplitude threshold, if the calculated acceleration amplitude is larger than the set threshold, executing the step 5, otherwise, re-acquiring the gesture information of the monitored object;

step 5: comparing the x-axis acceleration attitude angle and the y-axis acceleration attitude angle with an x-axis acceleration attitude angle threshold and a y-axis acceleration attitude angle threshold respectively; if the x-axis acceleration attitude angle is larger than the set x-axis acceleration attitude angle threshold or the y-axis acceleration attitude angle is larger than the y-axis acceleration attitude angle threshold, executing the step 6, otherwise, re-sensing the attitude information of the monitored object;

step 6: and calculating the acceleration in the vertical direction according to the z-axis acceleration attitude angle, comparing the acceleration in the vertical direction with a set downward acceleration threshold, if the acceleration in the vertical direction is smaller than the downward acceleration threshold, generating signal data if the monitored object is in a falling state, otherwise, re-acquiring the attitude information of the monitored object.

Further, the set acceleration amplitude threshold is 1.9, the x-axis acceleration attitude angle threshold and the y-axis acceleration attitude angle threshold are 65 degrees, and the downward acceleration threshold is 0.6.

Preferably, the process of obtaining the optimal decision using the semi-observed Markov decision process model includes:

step 1: initializing semi-observed Markov decision process model parameters, using the initialized parameters and data of an input model for seven-tuple < S, A, P, omega, O, R, gamma > representation, wherein S represents a set of state sets, A represents a set of action sets, P represents a transition matrix between states, omega represents a set of observation sets, O represents an observation probability, R is a return function, and gamma is a discount factor;

step 2: setting a determined time interval and time, and making a decision at each time; the set time is t= {0, …, T }, where T represents the timeline;

step 3: setting an initial belief state of the monitored object, wherein the belief state represents the knowledge of a decision maker on the current walking state of the monitored object;

step 4: calculating a return value of the monitored object according to the input data, and calculating the expectation of the model according to the return value;

step 5: updating the initial belief state according to the input data;

step 6: according to the updated belief state and the return value, a Belman optimal equation is obtained;

step 7: and calculating an optimal solution of the Belman optimal equation, wherein the optimal solution is a decision of the optimal monitoring object at the current moment.

Further, the belief state update formula is:

where O (o|s') represents the observation probability and pi(s) represents the initial belief state.

Further, the expression of the best bellman equation is:

where s represents the state in the state set of the monitored object, pi represents the belief state of the monitored object, r (s, a) represents the return value, a represents the action performed by the arm system, p (s '|s, a) represents the transition probability, O (o|s') represents the observation probability, and V (s ', pi') represents the bellman equation.

A human body fall-reducing and shock-preventing intelligent monitoring system based on multi-sensor data reinforcement learning comprises a human body acceleration data acquisition component, a micro-processing module, an alarm module and a power supply module; the human acceleration data acquisition assembly, the micro-processing module and the alarm module are all arranged on the PCB;

the human body acceleration data acquisition assembly comprises three Hall acceleration sensors and three angle measurement sensors, wherein the three Hall acceleration sensors are used for measuring acceleration information of an x axis, a y axis and a z axis of a monitored person, and the three angle measurement sensors are used for measuring attitude angles of the acceleration of the x axis, the y axis and the z axis and are connected with the micro-processing module through an OR gate;

the micro-processing module is used for receiving and processing the data transmitted by the human acceleration data acquisition component in real time and sending signal data to the alarm module;

the alarm module is used for acquiring signal data, processing the transmitted signals by adopting a semi-observation Markov decision process model, predicting the falling risk of the monitored object, outputting an inflation command according to the predicted falling risk, inflating the air bag according to the inflation command, and protecting the monitored object;

the power module is connected with the PCB and is used for supplying power to the human body falling-reducing shockproof intelligent monitoring system.

In order to achieve the above object, the present invention further provides a computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any of the above intelligent monitoring methods for human body fall-reducing and shock-preventing based on multi-sensor data reinforcement learning.

In order to achieve the above purpose, the invention also provides a human body fall-reducing and shock-preventing intelligent monitoring device based on multi-sensor data reinforcement learning, which comprises a processor and a memory; the memory is used for storing a computer program; the processor is connected with the memory and is used for executing the computer program stored in the memory, so that the human body fall-reducing and shock-preventing intelligent monitoring device based on multi-sensor data reinforcement learning can execute any one of the human body fall-reducing and shock-preventing intelligent monitoring methods based on the multi-sensor data reinforcement learning.

The invention has the following advantages: for judging the falling state, the invention designs a falling algorithm, and judges the falling state under the condition that three-level judgment is established so as to make protective measures; the invention combines the human body falling process with a semi-observed Markov decision process (POMDP) model to predict the action of an intelligent execution arm so as to pre-judge the human body falling state and execute the intervention of an intelligent air bag; the intelligent airbag intervention mode based on reinforcement learning can pertinently reduce the injury risk of falling for patients with different falling types.

Drawings

FIG. 1 is a schematic diagram of human body posture sensor data collection for human body fall-reducing anti-seismic reinforcement learning

FIG. 2 is a hardware wiring diagram of the human body anti-falling intelligent protection system based on multi-sensor data reinforcement learning disclosed in the embodiment of the invention;

FIG. 3 is a flowchart of a tilt angle threshold determination algorithm according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a data testing device according to an embodiment of the present invention;

FIG. 5 is a flow chart of intelligent control of an execution arm based on reinforcement learning in human body fall-reducing earthquake resistance;

fig. 6 is a schematic structural diagram of a human body anti-falling intelligent protection system based on multi-sensor data reinforcement learning.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

A human body fall-reducing and shock-preventing intelligent monitoring method based on multi-sensor data reinforcement learning is shown in figure 1, and comprises the following steps: the method comprises the steps of monitoring a monitored object in real time, and collecting attitude information of the monitored object by adopting a sensor; processing the acquired attitude information to obtain signal data; setting a prediction threshold value, comparing the set prediction threshold value with signal data, and if the signal data is smaller than or equal to the set prediction threshold value, re-acquiring the posture information of the monitored object, otherwise, inputting the signal data into a semi-observation Markov decision process model, and predicting the falling risk of the monitored object; and outputting an inflation command according to the predicted falling risk, and inflating the air bag according to the inflation command to protect the monitored object.

An important link of the invention is based on a multi-sensor data reinforcement learning concrete implementation mode, combining the human body falling process with a semi-observation Markov decision process (POMDP) model, predicting the action of an intelligent execution arm, so as to pre-judge the human body falling state and execute the intervention of an intelligent air bag. In the POMDP model, elements therein are typically represented by a seven-tuple. The method specifically comprises the following steps:

step S11: a finite set of states is denoted by S, states herein being { normal walking, high risk of falling, death }, denoted as { S1, S2, S3, S4};

step S12: a is used for representing a limited action set, which refers to actions which can be executed by the intelligent execution arm and comprises actions executed at parts such as { non-execution, waist, left upper limb, right upper limb, head, left lower limb, right lower limb, spine }, and the actions are respectively represented as { A0, A1, A2, A3, A4, A5, A6, A7}; the signal data is transmitted to the tilt sensor airbag device; the device is inflated to protect the monitored object.

Step S13: representing the state transition matrix by P, P _a (s ' |s) =p (s ' |s, a) represents the probability that taking action a at time t state s can transition to state s ' at time t+1;

step S14: r is used for indicating the return obtained by the intelligent execution arm at a certain moment when the human body is in the state at the next moment after the intelligent execution arm executes a certain action;

step S15: omega is used for representing an observation result set, and human body observation is mainly obtained through sensors, wherein the sensors are bound at different parts { sole, left lower limb, right lower limb, waist, spine, left upper limb, right upper limb and head (face) } of the human body, signals in the falling process are tested, and the sensors at different parts can be simultaneously or respectively carried out;

step S16: o represents the conditional observation probability, and when the signal of the sensor is received, only part of characteristics of the human body in walking can be observed, so that the probability that the human body is in a certain state needs to be judged by combining the signal; monitoring a monitored object in real time; collecting attitude information of a monitored object; processing the gesture information using an algorithm; calculating to obtain signal data of the attitude state; comparing the obtained signal data with a threshold value; and if the signal data is larger than the threshold value and the three-level judgment is true, judging that the monitored object falls down. And if the signal data is smaller than the threshold value and any one of the three-level judgment is not established, continuing to monitor. The corresponding part signal is judged to be fallen to be 1, otherwise, the corresponding part signal is judged to be 0. When complete, 256 elements are included, such as {1,1,1,1,1,1,1,1} {1,0,0,0,0,0,0}. The signal set of the sensor can be reduced to 9 elements, i.e., o1{0,0,0,0,0,0,0,0}, o2{1,0,0,0,0,0,0,0}, o3{0,1,0,0,0,0,0,0}, o4{0,0,1,0,0,0,0,0}, o5{0,0,0,1,0,0,0,0}, o6{0,0,0,0,1,0,0,0}, o7{0,0,0,0,0,1,0,0}, o8{0,0,0,0,0,0,1,0}, o9{0,0,0,0,0,0,0,1}.

Step S17: the discount factor gamma E [0,1] is denoted by gamma.

Because doctors can only observe partial characteristics of the human body in walking, the human body is required to be judged to be in a certain state by combining signals, and 8 sensors are simultaneously bound on 8 parts to test signals in the falling process of the human body; or 8 tests were performed separately. After the state of the human body is judged, the human body obtains a corresponding return value. Therefore, in order to prevent the human body from falling, the intelligent execution arm is worn on the human body, and corresponding actions are executed when the human body falls. In each falling, among five parts, in order to effectively judge the part of the intelligent air bag which performs the action, two kinds of constraints exist: (1) each time the patient falls, 8 position sensors can sense signals, and modeling analysis of the 8 signals is carried out; (2) each fall, only one-five (positional balloon) can be automatically (model algorithm) executed.

The state transition probability matrix will be described by taking the execution of the operation a1 at the time t as an example. The first column in the table indicates the state of the human body at time t, the first row indicates the state of the human body at time t+1 after the action a1 is performed based on the state at time t, the first row data indicates that the state of the human body at time t is normal, no action (a 1) is performed, the probability that the human body still walks normally at time t+1 is 0.5, the probability of high risk is 0.2, the probability of falling is 0.2, and the probability of death is 0.1. The first column of data indicates that the state of the human body at time t is normal/high risk/fall/death, no action (a 1) is performed, and the probability that the state of the human body at time t+1 is normal is 0.5/0.1/0.01/0 in sequence. The data for each row in this table is added to 1.

Table 1T probability matrix of state transition of execution action a1

The observation probability matrix will be described by taking the execution of the operation a1 at the time t as an example. The first column in the table indicates the state of the human body at time t, the first column indicates the state of the human body at time t, the first row indicates the probability that the signal of the sensor is observed at time t+1 after the action a1 is performed based on the state at time t, the first row data indicates the state of the human body at time t is normal, no action (a 1) is performed, and each signal of the sensor is observed at time t+1. The first column data indicates that the state of the human body at time t is normal/high risk/falling/death, no action (a 1) is performed, and the probability of no signal of the sensor at time t+1 is 0.2/0.1/0/0 in sequence. The data for each row in this table is added to 1.

Table 2t+1 observation probability matrix for executing action a1 at time

The acquired detection data is calculated, and the process is mainly used for cleaning low-quality video images in source data and eliminating data which do not meet specifications. The method comprises the steps of extracting, converting, loading and other data cleaning of input data so as to realize missing data processing, similar repeated object detection, abnormal data processing, logic error detection, inconsistent data processing and the like.

The database design is shown in table 3 and examples of the portion of the characteristic data extracted by the tilt sensor and associated data are shown in table 4.

Table 3 sensor data sheet

Table 4 example of feature data portions sensed by tilt sensor and extracted from associated data

The hardware link of the human body anti-falling intelligent protection system based on multi-sensor data reinforcement learning is shown in fig. 2, and comprises: all devices connected with the singlechip are grounded. The power supply of the motor driver cannot be connected reversely, and a 15A fuse is connected in series at the power interface, and the voltage is between 6.5 and 27V; if the voltage is over-voltage, the drive module can be burnt by power-on; the boosting module inputs 7.4V and outputs 12V; if the output is not 12V, the blue-tone regulator on the module can be turned, and the voltage output can be adjusted. The singlechip controls IN1, IN2 and ENA1 of the motor driving module to realize forward and reverse rotation of the motor, and +5v suggestion of the enabling end is connected with 3.3V; the enable end of the ENA1 device is required to be connected with a high level no matter the enable end is in positive and negative rotation; IN1 inputs high level and IN2 inputs low level then motor full speed forward, IN1 inputs high level and IN2 inputs low level then motor full speed reverse; the motor cannot be commutated when it has not stopped, otherwise the drive may be damaged; when the driving module is powered down, the motor is not required to be directly or indirectly rotated at high speed, otherwise, the electromotive force generated by the motor may burn the driving module. If the motor needs to be rotated at a high speed when the power of the driving module is lost in application, a relay (NO and COM end are connected in series) is connected in series to a motor interface of the driver, and the relay coil and the driver share a power supply, namely, when the power supply is lost, the relay can disconnect the connection between the driver and the motor.

As shown in fig. 3, the process of processing the collected gesture information includes:

step 1: establishing a space rectangular coordinate system by taking the right front of a monitored object as an x axis, the right left side as a y axis and the vertical direction as a z axis; an acceleration amplitude threshold, an x-axis acceleration attitude angle threshold, a y-axis acceleration attitude angle threshold, and a downward acceleration threshold are set.

The human body is only subjected to the action of gravity acceleration in the vertical direction when the human body is at rest. X, y, z axis accelerations and attitude angles pitch, rolly and raw obtained by collecting MPU6050 data.

Step 2: and smoothing the acquired acceleration of the x axis, the y axis and the z axis.

In some cases, some extrinsic factors may have an effect on the acceleration acquired by the sensor. In order to reduce misjudgment possibly caused by noise, the collected acceleration is subjected to five-point multiple-time sliding average within a certain time window, dirty data with larger deviation is removed, and errors caused by enlarging the dirty data during square operation are prevented.

Step 3: and calculating the acceleration amplitude of the monitored object according to the smoothed data.

And calculating an acceleration amplitude value (the acceleration amplitude reflects the intense motion degree of the human body) by using the processed data. Falls belong to a strenuous activity in life, and when falling occurs, the height of a human body relative to the ground can change rapidly, and the acceleration suffered by the human body in the process can also change. This change can be quantified by the acceleration amplitude value and reflect the change in attitude.

Step 4: and (5) comparing the calculated acceleration amplitude with a set acceleration amplitude threshold, if the calculated acceleration amplitude is larger than the set threshold, executing the step (5), otherwise, re-acquiring the posture information of the monitored object.

In order to reduce the influence of dirty data in the data taking process, the algorithm performs five-point multiple smoothing on the axial acceleration data in the x, y and z3 directions in the time window T. The human body falling time is 0.3-0.4 s, so the time window T is about 10 sampling points. And carrying out smoothing treatment on the acquired data, and calculating the acceleration amplitude according to the distance characteristic value. The threshold between the falling and non-falling states is set to 1.9, and if the calculated value is greater than 1.9, the next step of judgment is entered.

Step 5: comparing the x-axis acceleration attitude angle and the y-axis acceleration attitude angle with an x-axis acceleration attitude angle threshold and a y-axis acceleration attitude angle threshold respectively; and if the x-axis acceleration attitude angle is larger than the set x-axis acceleration attitude angle threshold or the y-axis acceleration attitude angle is larger than the y-axis acceleration attitude angle threshold, executing the step 6, otherwise, re-acquiring the attitude information of the monitored object.

A single calculated value cannot mask some disturbing actions, such as a strong movement of the human body, etc., so in order to better judge the falling state, an auxiliary judgment is adopted by downward acceleration and angle judgment of roll angle and pitch angle. When the human body falls back and forth, the pitch angle is larger than 65, and when the roll angle is larger than 65, the next judgment is carried out.

The downward acceleration of the human body is approximately equal to 9.8m/s when the human body is stationary or walks vertically, the human body loses weight when falling down, and the acceleration is reduced. When the downward acceleration is less than 0.6m/s during falling, the next step is carried out. If the three-level judgment is met, judging that the carbon dioxide bottle falls, outputting a forward rotation signal of the motor by the singlechip, and breaking the carbon dioxide bottle by the motor.

The process of obtaining the optimal decision by adopting the semi-observed Markov decision process model comprises the following steps:

step 3: setting a belief state of an initial monitored object, wherein the belief state represents the knowledge of a decision maker on the current walking state of the monitored object;

step 5: updating the initial belief state according to the input data;

Specifically, the time of day is determined and a time line is determined. The time interval and the time moment are determined according to the sampling frequency of the sensor sensing data, and a decision is made at the beginning of each time moment from the condition that the intelligent device is worn by the monitored object. The time instant is denoted herein by t= {0, …, T } where T denotes the time line.

State space. The states of the human body during walking are divided into four categories: normal walking, high risk of falling, death, symbolized by s= { S _N ,s _H ,s _C ,s _D The last state (death) is an absorption state. The state space is partially observableBut death is the only state that can be fully observed. S is used herein _t The S represents the state of the human body at time t.

An action space. Because the mechanical arm performs the actions at different positions, the division of the action space is also based on the different positions, including not performing any actions, the waist/left upper limb/right upper limb/head/left lower limb/right lower limb/spine airbag intelligent machine execution arm, and is expressed as a = { a by a symbol _N ,a _W ,a _LA ,a _RA ,a _H ,a _LL ,a _RL ,a _S Use a herein _t E a represents the action performed at decision time t.

Transition probabilities. P(s) for transition probability _t+1 |s _t ,a _t ) Meaning that the human body is in the current state (s _t Epsilon S) perform action (a _t E A) and then transition to the next time state (s _t+1 E S). These probabilities are statistically derived from sensor data of the anti-fall device worn by the subject, and may also be obtained by simulating more than 1000 walking and falling processes.

Observation and observation probability. At each decision moment, a set of observations (o e Ω) will provide some information about the real presence of the unobserved state of the human body. The model is mainly obtained through a sensor for observation and is obtained according to sensor data of anti-falling equipment worn by a subject. Based on the signals returned by the sensors at different locations, their observations are classified into 9 categories: no signal, { sensor signals for waist, left upper limb, right upper limb, head, left lower limb, right lower limb, spine } are expressed in turn as { (0000000), (1000000), (0100000), (0010000), (0001000), (0000100), (0000010), (0000001) } using two categories, and the other category is death, so that all observation sets correspond to the symbol Ω= { o _N ,o _W ,o _LA ,o _RA ,o _H ,o _LL ,o _RL ,o _S ,o _D }. The sensor signal is not completely accurate, and the probability relation exists between the observed state and the unobserved state, which is represented by a probability matrix O and consists of the observation probability O (o|s)The meaning is the probability that the state o is observed given the patient's true state s and the execution of action a. The observation probability matrix data are obtained through statistics according to sensor data of anti-falling equipment worn by the subject.

Belief status. Let n (S) represent the probability of each state in the state space S, in this case, four states in total, the probability of which is expressed as:

the vector pi is called as belief state and represents the knowledge of the decision maker about the current walking state of the human body. The belief states are represented by a set of probabilities, each probability representing the likelihood of each state, and the belief state of the human body at time t can be represented as:

π _t ＝(π _t (s _N ),π _t (s _H ),π _t (s _C ),π _t (s _D ))

the belief state vector of the human body at the time t is pi= (0.3,0.4,0.4,0), which indicates that the human body has a probability of 30% that the human body is walking normally, a probability of 30% that the human body is in a falling high-risk state and a probability of 40% that the human body has fallen.

A return function. In the POMDP model, its optimal strategy is to maximize the expected return within the decision range T, expressed mathematically as:

wherein gamma represents a discount factor, r _t Indicating the immediate return value at time t. The rewards depend on the state of the human body and the action taken, and the set of possible rewards is derived from a rewards function r (s, a, s '), which is the rewards of performing action a at state s and transitioning to state s'. The return value is obtained by calculating the corresponding statistical probability of the injury caused by various states and the medical expense (normalization processing).

And updating belief states. Given a new observation o ' e Ω, the belief state pi is updated to pi ' according to bayesian rules, and for each state S ' e S, its corresponding updated belief state value can be calculated using the following formula:

wherein O (o|s ') represents the observation probability, pi(s) represents the initial belief state, and s ' and pi ' represent the state and belief state corresponding to the next time, respectively.

The POMDP model can be reformulated by a continuous state MDP, and its optimal strategy is obtained by solving the bellman optimal equation:

wherein s represents the state in the state set of the monitoring object, pi represents the belief state of the monitoring object, r (s, a) represents the return value, a represents the action performed by the arm system, p (s '|s, a) represents the transition probability, O (o|s') represents the observation probability, V (s ', pi') represents the Belman equation value corresponding to the next time, gamma is the discount factor, pi _j Representing the belief state corresponding to state j.

The algorithmic process of the markov decision process based on the observable walking state of the human body portion is shown in table 5.

TABLE 5

As shown in fig. 4, a data testing device, in which a processor performs smoothing processing and acceleration amplitude calculation on collected MPU6050 data, and transmits effective information; the vibration sensor module judges the sensor state; the communication interface transmits information; in a falling state, the anti-falling and anti-seismic air field device is inflated to protect.

As shown in fig. 5, the state transition of the monitoring system and the selection of the execution arm of the intelligent device to execute the action include: four states are normal walking, high risk of falling, falling and death, and the intelligent execution arm can execute actions at the parts including the waist, the left upper limb, the right upper limb, the head, the left lower limb, the right lower limb, the spine and the like. The control signal activates the execution arm at a certain position, and the intelligent execution arm executes a certain action at a certain moment. Will be in the new state and will be rewarded accordingly.

The human body anti-falling intelligent protection system based on multi-sensor data reinforcement learning comprises a human body acceleration data acquisition component, a micro-processing module, an alarm module and a power supply module; the human acceleration data acquisition assembly, the micro-processing module and the alarm module are all arranged on the PCB;

As shown in fig. 6, the general architecture of the human body anti-falling intelligent protection system based on multi-sensor data reinforcement learning includes: the power module turns on a power supply; sensing by an alarm in the vibration sensor module, and judging whether the vibration sensor module is in a vibration or inclination state; closing the carbon dioxide small steel cylinder when the sensor vibrates or tilts; after closing the small carbon dioxide steel cylinder, the motor breaks the small carbon dioxide steel cylinder; after the motor breaks the small carbon dioxide steel cylinder, the anti-falling shock-resistant buffering inflatable bag air bubble column air bag begins to inflate.

In an embodiment of the present invention, a computer readable storage medium is further included, on which a computer program is stored, where the program when executed by a processor implements any one of the above-mentioned intelligent monitoring methods for human body fall-prevention based on multi-sensor data reinforcement learning.

Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the method embodiments described above may be performed by computer program related hardware. The aforementioned computer program may be stored in a computer readable storage medium. The program, when executed, performs steps including the method embodiments described above; and the aforementioned storage medium includes: various media that can store program code, such as ROM, RAM, magnetic or optical disks.

A human body fall-reducing and shock-preventing intelligent monitoring device based on multi-sensor data reinforcement learning comprises a processor and a memory; the memory is used for storing a computer program; the processor is connected with the memory and is used for executing the computer program stored in the memory, so that the human body fall-reducing and shock-preventing intelligent monitoring device based on multi-sensor data reinforcement learning can execute any one of the human body fall-reducing and shock-preventing intelligent monitoring methods based on the multi-sensor data reinforcement learning.

Specifically, the memory includes: various media capable of storing program codes, such as ROM, RAM, magnetic disk, U-disk, memory card, or optical disk.

Preferably, the processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, abbreviated as CPU), a network processor (Network Processor, abbreviated as NP), etc.; but also digital signal processors (Digital Signal Processor, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field programmable gate arrays (Field Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

While the foregoing is directed to embodiments, aspects and advantages of the present invention, other and further details of the invention may be had by the foregoing description, it will be understood that the foregoing embodiments are merely exemplary of the invention, and that any changes, substitutions, alterations, etc. which may be made herein without departing from the spirit and principles of the invention.

Claims

1. A human body fall-reducing and shock-preventing intelligent monitoring method based on multi-sensor data reinforcement learning is characterized by comprising the following steps:

s1: setting state transition probability and observation probability;

s5: inputting the optimal decision into an arm system, and executing a corresponding command by the arm system according to the optimal decision, wherein the command comprises selecting an execution arm with the maximum protection function from the execution arms of the candidate part, activating the execution arm and outputting an inflation command; the execution arm with the maximum protection function is the execution arm with the maximum return value;

s6: inflating the intelligent air bag according to the inflation command;

2. The intelligent human body fall-reducing and shock-preventing monitoring method based on multi-sensor data reinforcement learning of claim 1, which is characterized in thatThe process of setting the state transition probability and the observation probability comprises the following steps: the calculation process of the state transition probability comprises the steps of obtaining historical monitoring data of a monitored object, and carrying out statistical normalization processing on the historical monitoring data by adopting a statistical method to obtain the state transition probability; the observation probability calculation process includes classifying observation categories of the monitoring object into 9 categories, including: no signal, lumbar, upper left, upper right, head, lower left, lower right, spine sensor signal, and death signal; all observation sets correspond to the sign Ω= { o _N ,o _W ,o _LA ,o _RA ,o _H ,o _LL ,o _RL ,o _S ,o _D O, where o _N Indicating no signal, o _W Represents lumbar sensor signal, o _LA Representing left upper limb sensor signal o _RA Represents the upper right limb sensor signal, o _H Representing head sensor signals, o _LL Represents left lower limb sensor signal o _RL Represents the right lower limb sensor signal, o _S Representing spinal sensor signals, o _D A death signal; and acquiring historical monitoring data of the monitored object, and carrying out statistical normalization processing on the historical monitoring data by adopting a statistical method to obtain the observation probability of each observation category.

3. The intelligent human body fall and vibration prevention monitoring method based on multi-sensor data reinforcement learning according to claim 1, wherein the multi-sensor comprises three hall acceleration sensors and three angle measurement sensors, wherein the three hall acceleration sensors are used for measuring acceleration information of an x axis, a y axis and a z axis of a monitored object, and the three angle measurement sensors are used for measuring attitude angles of the x axis, the y axis and the z axis acceleration.

4. The intelligent human body fall and shock prevention monitoring method based on multi-sensor data reinforcement learning of claim 1, wherein obtaining the human body fall related state information comprises:

5. The intelligent human body fall and shock prevention monitoring method based on multi-sensor data reinforcement learning of claim 4, wherein the process of calculating the posture information of the monitored object comprises the following steps:

6. The intelligent human body fall and shock prevention monitoring method based on multi-sensor data reinforcement learning of claim 5 is characterized in that the set acceleration amplitude threshold is 1.9, the x-axis acceleration attitude angle threshold and the y-axis acceleration attitude angle threshold are 65 degrees, and the downward acceleration threshold is 0.6.

7. The intelligent human body fall and shock monitoring method based on multi-sensor data reinforcement learning of claim 1, wherein the process of obtaining the optimal decision by adopting a semi-observed markov decision process model comprises the following steps:

step 5: updating the initial belief state according to the input data;

8. The intelligent monitoring method for human body fall and shock prevention based on multi-sensor data reinforcement learning of claim 7, wherein the belief state update formula is:

9. The intelligent human body fall and shock prevention monitoring method based on multi-sensor data reinforcement learning of claim 7, wherein the expression of the bellman optimal equation is:

10. The human body fall-reducing and shock-preventing intelligent monitoring system based on multi-sensor data reinforcement learning is characterized by comprising a human body acceleration data acquisition assembly, a micro-processing module, an alarm module and a power supply module; the human acceleration data acquisition assembly, the micro-processing module and the alarm module are all arranged on the PCB;

the human acceleration data acquisition assembly includes: the three Hall acceleration sensors are used for measuring acceleration information of an x axis, a y axis and a z axis of a monitored person, and the three angle measurement sensors are used for measuring attitude angles of the acceleration of the x axis, the y axis and the z axis and are connected with the micro-processing module through an OR gate;