JP2016112053A - Walking state determination method, program and device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method for accurately estimating a walking state such as a walking balance between right and left and a deviation from periodicity on the basis of not only information from a sensor commonly used for measuring walking such as an acceleration sensor but also information from an easy-to-install sensor such as an acoustic sensor.SOLUTION: A method for determining a walking state includes: inputting acoustic information such as a foot-step sound of a pedestrian generated at the time of walking, as an electric signal via a sensor such as a microphone; performing a feature extraction process of analyzing the electric signal to be converted into a temporal change amount of a feature amount related to walking; performing a walking cycle extraction process for analyzing the temporal change amount to estimate a walking cycle and its probability thereby to obtain estimation values of the walking cycle and probability; and performing a walking state analysis process for generating information on the walking state of the pedestrian based on the estimation values.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a feature amount extraction method and a walking state determination method for estimating the activity state and health state of a moving organ related to walking.

If the walking state can be automatically measured and the situation can be determined, it is useful for watching the elderly, grasping the living situation, and managing the health of the motor organs.
As existing techniques for measuring the walking state, a pedestrian carries a small device incorporating an acceleration sensor or a gyro sensor (for example, Patent Document 1), or a pressure sensor is installed on the floor (for example, Patent Document 2). ) Is common.

A video camera can be used to directly observe the walking state, but it has many privacy issues and is unsuitable for daily living environments.
Techniques for measuring walking sounds (footsteps) using a microphone have already been reported. Although it is necessary to develop a technique for indirectly estimating the acoustic information that accompanies walking instead of the information obtained by directly observing the walking state, there are many advantages in terms of privacy and installation costs.

JP 2006-118909 A “Pedometer” Japanese Patent Application Laid-Open No. 2014-094070 “Aging Age Evaluation Method” Japanese Patent Application Laid-Open No. 2014-191616 “Method, Device and Service Providing System for Elderly Living Alone” Japanese Patent Laid-Open No. 2002-197437 “Walk Detection System, Walk Detection Device, Device, and Walk Detection Method” JP 2005-342254 A "Walking cycle detection device" Japanese Unexamined Patent Publication No. 2012-058340 “Sound Detection Device, Sound Detection Method, and Sound Detection System” JP 2013-215220 A "Walking state detection device"

Alan de Cheveigne and Hideki Kawahara, “Yin, a fundamental frequency estimator for speech and music,” Journal of the Acoustical Society of America, vol.111, no.4, pp.1917-1930, 2002. Sandra R. Hundzaand, William R. Hook, et al., “Accurate and reliable gait cycle detection in Parkinson's disease,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol.22, no.1, pp.127-137, 2014 .

  As existing techniques for estimating a walking state from walking sounds (footsteps), a pedestrian flow line is estimated using a microphone array in which a plurality of microphones are combined (for example, Patent Document 3), or a pedestrian is identified. However, an object of the present invention is to provide a method for extracting a walking cycle from footsteps and determining a walking state such as a pedestrian's health state and activity state based on the extracted walking cycle.

As a known example for estimating the walking period, there is a method (for example, Patent Documents 1 and 5) that uses information from an acceleration sensor to extract a spectrum peak by Fourier transform, but it can also be applied to information from an acoustic sensor. Another object is to propose a highly accurate period extraction method.
As a known example of extracting a walking cycle from footsteps, for example, Patent Document 6 uses a statistical model in which periodicity is preliminarily assumed, so it is considered unsuitable for detecting a left / right balance or a shift in periodicity. An object of the present invention is to propose a period extraction method that assumes a left-right balance and a shift in periodicity.

In estimating the walking state, it is first determined whether or not the user is walking. This makes it possible to estimate the total amount of walking exercise, which is an index of health status. Moreover, the analysis object for determination of the walking state after that can be limited by this.
Furthermore, from the viewpoint of health management such as deterioration of the joints of walking organs and prevention of accidents in daily life such as whispering during walking, it is important to detect the left and right walking balance as an indicator. is there.
For example, Non-Patent Document 2 is an example of existing research. However, the gyro sensor is used to analyze the movement of the foot in detail, including how to land, and there are problems in terms of ease of mounting and practicality. is there.

Known examples of detecting a walking state from footsteps include, for example, Patent Document 4 and Patent Document 7, but the former uses information in a low frequency band of 100 Hz or less, and the latter is associated with the distance between the target pedestrian and the microphone. Since they are based on changes in volume, it is considered that they are easily affected by vibrations other than footsteps, impact sounds of daily necessities such as furniture, and human voices in daily living environments. It is an object to propose a robust analysis method against the influence of daily sound other than footsteps.
These issues are listed in order to estimate the walking state from acoustic information such as footsteps, but the method for solving the problem is information other than sound that can be obtained from acceleration sensors, distance sensors, etc. It is even better if there is a commonality with respect to the sensor and the method can be expanded to other sensors.

  In the present invention, attention is paid to a walking cycle as a feature quantity that is deeply related to a walking state, and an estimation method thereof and a walking state determination method based thereon are provided. As observation information used for estimating the walking state, acoustic information from a microphone is preferably assumed. However, the method proposed in the present invention can be similarly applied to information other than acoustic information such as acceleration information and joint position information. Yes, it does not limit the type of observation information.

(1)
Acoustic information such as footsteps that occur when a pedestrian walks is input as an electrical signal by a sensor such as a microphone.
A feature extraction process for analyzing the electrical signal and converting it into a temporal change amount of a feature amount related to walking;
A walking cycle extraction process for analyzing the time variation and estimating the walking cycle (for one step and two steps) and its probability;
By applying the walking state analysis process that generates information on the walking state of the pedestrian from the estimated value of the walking cycle and its certainty obtained by applying
Provided is a walking state determination method capable of outputting information on the walking state of the pedestrian.

(2)
The feature extraction processing can reduce the influence of daily sounds other than footsteps and human voices by converting the time change amount of the power (average energy) of the frequency band component of the electrical signal of 8 kHz or higher, A feature extraction method is provided.

(3)
The gait cycle extraction process takes the time variation obtained by the feature extraction process as an input, converts it into any of autocorrelation, cepstrum, and average amplitude error, and further normalizes it. Provided is a walking cycle estimation method characterized by applying an extraction function to estimate a walking cycle and outputting the probability.

(4)
The walking state analysis process is based on the walking cycle obtained by the walking cycle extraction process and the probability value, and is compared with a specific reference value appropriately set in the determination database (determination DB). Provided is a walking action determination method for determining whether or not walking is performed in a time section corresponding to an input electric signal.
The reference value in the determination DB is appropriately set based on the statistical distribution obtained by accumulating the walking period and the probability value obtained by the walking period extraction process and statistically processing the values in advance or sequentially. It is obtained as an appropriately set threshold value corresponding to the function value.

(5)
The walking state analysis processing is based on the walking cycle obtained by the walking cycle extraction processing and the value of the probability, and the probability corresponding to the cycle for each step and the cycle for two steps (a pair of left and right feet). Provided is a walking state estimation method that outputs an estimated value that indicates whether the right and left balance is good or bad during walking by extracting a corresponding probability and converting it into a relative value for comparison.

(6)
A walking abnormality detection method for detecting an abnormality in walking by storing an estimated value obtained by the walking state analysis process over a predetermined period and detecting an abnormal value by comparing the statistical value with a reference value in a determination database. I will provide a.

In the above description, the information input for estimating the walking state is assumed to be acoustic information from a microphone. However, the proposed method can be similarly applied to information other than acoustic information such as acceleration information and joint position information. The type of observation information is not limited.
(7)
For example, an information obtained by converting information obtained by observing a pedestrian's leg using a distance sensor into an electric signal is used as an input signal, and the position of the joint of the pedestrian's leg is determined from the distance information obtained for the leg in the feature extraction process. Provided is a method of detecting a walking state by estimating and converting the movement distance of a leg joint into a temporal change amount based on the estimated position information of the leg joint.

(8)
As yet another example, as an input signal, an acceleration sensor attached to the pedestrian is used to convert pedestrian movement information into an electrical signal. A method for detecting a walking state is provided by converting into a time change amount.
(9)
It is possible to provide a walking state determination program characterized by executing the walking state determination method of (1) to (8) and a storage medium storing the program.
(10)
A walking state determination device comprising at least a sensor device (microphone, distance sensor, acceleration sensor, etc.), an information processing device (including a display device), and executing the walking state determination method of (1) to (8) Can be provided as

In (1), the information about the walking state can be obtained from the acoustic information, so it is not necessary to carry or wear equipment such as sensors. There is an effect that can be grasped more accurately and simply.
Further, in (2), in the form using a microphone as a sensor, only high frequency components of 8 kHz or higher that do not contain almost any audio information except for some consonants are used in the acoustic data, and among them, the sound waveform is directly used. First, since data converted only to power information every several milliseconds is used, there is an effect that the privacy problem is almost solved. Compared with the method using a low frequency component of 100 Hz or less, which is proposed in Patent Document 4, the audio information contained therein is greatly reduced, and the accuracy is also lowered due to the influence of floor vibration and daily noise. Few.
In addition, in (3), by applying a periodicity extraction function including a normalization method optimized for walking cycle extraction, a robust and highly accurate walking cycle can be estimated even in an actual living environment, There is an effect that information on the accuracy of estimation can be obtained at the same time.
In (4), the walking period and other periods can be determined with high accuracy from the observed data using the walking cycle and the probability value as an index. It is effective in obtaining information that can be used as an index for safety and health.
Also, in (5) and (6), the left and right walking balance can be determined, and it becomes possible to detect deterioration of the walking organs such as the deterioration of the ankle joint accompanying the aging of the elderly. This has the effect of making it possible to detect accidents during walking, such as tripping inside.
Also, in (7) and (8), almost the same method can be applied to the form using acceleration information and joint position information instead of acoustic data. There is an effect that a walking state determination method can be used by selecting a mounting method suitable for the user.

Overall view of the process Overall configuration of Example 1 Example of walking cycle extraction from acoustic signal Results of evaluation experiment of walking cycle extraction accuracy Detection example of gait balance breakdown (by acoustic sensor) Detection example of gait balance breakdown (based on joint position information from distance sensor)

The entire processing flow in this method will be described with reference to FIG.
The outline of the processing is as follows. The input signal is converted into a feature value in the feature extraction unit, and after the walking cycle is extracted therefrom, the walking state is analyzed using the result.
In these processes, each process may be executed sequentially by each apparatus, and at least an input device (including an electric signal converter), a processing device or an arithmetic processing device, an output device or a display device, and a storage device are provided. The program may be executed programmatically in the information processing apparatus, or may be incorporated in an embedded system.

  Information obtained by analysis is integrated with information prepared in advance to construct a determination DB, and based on the information in the determination DB, a determination result obtained from the analysis result is output.

By using this method, for example, as a monitoring device for elderly people, by determining abnormalities in the walking cycle, total number of walking, and walking state, it is possible to reduce momentum, change lifestyle, abnormalities in walking organs, etc. Applications that detect accidents such as stumbling are also conceivable.
As an example of such an embodiment, the following two types are taken up and described in detail.

As a first embodiment, a walking state determination apparatus using acoustic information will be described.
The overall configuration will be described with reference to FIG.
Sound information is converted into electrical signals by a microphone installed in the room.
The signal is sent to the walking state analysis processing device for processing.
The output signal is sent to a display device or the like connected by wire or wireless to notify the determination result.

  The details of the processing process inside the walking state analysis processing device will be described below.

An example of signals obtained in each processing step is shown in FIG.
This signal is recorded when an actual elderly person walks in a house, and includes not only footsteps but also furniture collision sounds and human sounds.
The signal was recorded after being quantized to 24 bits at a sampling frequency of 48 kHz.
The waveform is shown in FIG.

Next, the feature extraction unit converts this signal into a feature quantity suitable for analysis.
Here, the characteristic amount is converted into log power in a high frequency band as in the following equation (1).

In Equation (1), S _i (f) is a frequency spectrum obtained by Fourier-transforming the time series signal of the i-th frame (short period) extracted by a window function such as a Hamming window, and F _h and F _l are extracted. Represents the upper limit of the bandwidth range to be used.

As an example of frequency analysis, FIG. 3B shows a power spectrum obtained by Fourier-transforming the above signal with a 512-point FFT using a Hamming window having a window length of 10.7 milliseconds and a period of 5 milliseconds.
The range of the frequency obtained as an analysis result is up to 24 kHz which is the Nyquist frequency, and is the range displayed on the vertical axis of FIG. 3B, but the frequency component of human speech excludes some consonants. Most of them are up to about 4 kHz, and even if they are high, they are hardly included in the band above 8 kHz. On the other hand, the frequency components of footsteps are widely distributed in the high frequency range above the upper half of FIG. You can see that
Therefore, here it adopted 12kHz as F _l, as the F _h was adopted 24 kHz.
Since this frequency range includes almost no human voice band, privacy problems are reduced.
Further, by converting it into an integrated amount as shown in equation (1), it becomes almost impossible to extract voice information therefrom.

FIG. 3C shows the value of P _{i obtained} by converting the signal example of FIG. 3 by this processing.
The frequency range used for the feature value is distributed in a relatively high band when the pedestrian is wearing slippers as in this example, but if the footwear or floor condition is different, it depends on the target footstep. The accuracy can be improved by selecting a suitable range.

Next, the walking cycle extraction unit extracts a cycle from the feature amount obtained by the feature extraction unit.
As a method for extracting the period, a method using an autocorrelation function is generally used, and the period is estimated by calculating the autocorrelation function of the time series signal and extracting the local maximum value as shown in Equation (2). The

In addition, a method for estimating the period by extracting a local maximum value using a cepstrum coefficient as shown in Expression (3) is also widely known.
In Formula (3), F represents a Fourier transform.

Here, instead of the autocorrelation function, an average amplitude difference function (AMDF) as shown in Equation (4) is used.
In Expression (4), M represents the upper limit of the time difference to be calculated, and L represents the length of the time series.
The result obtained by the AMDF is substantially the same as the autocorrelation function, but the calculation efficiency is different from the point that the position of the period is obtained as its minimum value.
The fact that square values are used instead of taking absolute values in equation (4) does not change substantially.

The AMDF value A _j in the signal example of FIG. 3 is shown in FIG.
A position where A _j takes a minimum value is estimated as a period, and the estimated period is obtained by multiplying the value of j by a frame period and converting it into a time amount.
Tracing the time axis direction, i.e., increasing j from 0, the first minimum value corresponds to one step of one foot (one foot cycle), the second minimum value corresponds to two steps on the left and right (cycle period of both feet) ).

Here, as the value of M, 400 corresponding to 2 seconds considered to be the upper limit of the period of both feet was used.
Furthermore, as a periodicity extraction function for enhancing the extraction accuracy by emphasizing the minimum value, a process of normalizing the minimum value based on the maximum value as in the following equation (5) was applied.
This normalization processing is made to correspond to the walking cycle based on a method (Non-patent Document 1) for extracting the fundamental frequency (cycle) of human speech.
The position of the maximum value of B _j is the estimation period. The normalized minimum value B _j in the signal example of FIG. 3 is shown in FIG.

Schematically, in FIG. 3 (e), the first peak from the left corresponds to the one leg period, and the second peak corresponds to the both leg period.
An example of a specific procedure for extracting them will be described.
First, the maximum value U of B _j is obtained according to the following equation (6).

Next, using T _h = K _h U and T _l = K _l U obtained by multiplying U by proportional constants K _h and K _l as threshold values, a search range is set in a continuous interval where B _j takes a value larger than T _l. squeeze.
The maximum value is obtained in each of such areas, and the value of j corresponding to the maximum value is set as a period candidate.
If the maximum value T _h is smaller than at that time is excluded from the candidate. The obtained period candidate values converted into time quantities are obtained as the one leg period M ₁ and the both leg period M _{2 in} ascending order.
At the same time, the respective maximum values are obtained as the probability values L ₁ and L ₂ of the respective estimation periods.

An example of an experimental result for confirming the estimation accuracy of the period is shown in FIG.
Here, 0.5 and 0.25 were used as the values of K _h and K _l .

As an experimental sample, 221 sample data were obtained by cutting out sections containing walking sounds from data recorded with microphones of sounds of living activities of four elderly men and women in a house. In order to give a correct answer to the walking cycle, the sample data was listened to by humans and classified into five categories: {fast, somewhat fast, normal, slightly slow, slow}.
This category corresponds to {400 ms, 450 ms, 500 ms, 550 ms, 600 ms} as a one leg period, respectively. We prepared footsteps and performed comparative listening based on the footsteps.

In FIG. 4, the one-leg period is extracted from the sample data by the proposed method and displayed as a histogram in order from the top for each category. It can be seen that the distribution is generally centered on the correct period, and the period is extracted approximately correctly.
The accuracy rate within an error of 100 milliseconds was 85.5%, and the average error was 59.7 milliseconds.

Next, processing of the walking state determination unit will be described.
As one of the analysis results, it can be determined whether or not the target data is the one during walking.
As a threshold determination process for that purpose, for example, a total of four values (M ₁ , M ₂ , L ₁ , L ₂ ) corresponding to one leg period, both leg periods, and the respective probabilities obtained as described above are used as walking period parameters Obtained by accumulating parameters applied to a large number of samples including footsteps and other sounds in the determination DB, and applying a multivariate statistical analysis method such as discriminant analysis to these data. The coefficient of the discriminant function is stored in the determination DB. With respect to each function value obtained by applying this function to each sample of footsteps and other sounds, a threshold value is set so that the two can be optimally separated and stored in the determination DB. At the time of determination, by comparing a function value obtained by applying input data to the discriminant function with this threshold value, a determination result as to whether or not walking is obtained.

Another output obtained as an analysis result is the determination of the left and right walking balance collapse.
This makes it possible to detect health problems such as a knee joint and hip joint failure associated with aging.
The experimental example will be described with reference to FIG.
In the experiment, in order to simulate abnormalities in the walking organs of healthy subjects, a belt with a weight is wrapped around the ankle of one foot, and two subjects rotate in the range of about 10 square meters counterclockwise, clockwise, and round trip. Walking with a walking pattern for about 30 seconds each, the walking sound at that time was recorded and analyzed by the above method.

In FIG. 5, as an example, in a sample of footsteps when circling counterclockwise, (a) when a weight is not attached, (b) when a weight of 1 kg is attached, and (c) a weight of 2 kg is attached. Show the analysis results. In each result, comparing the probability L ₁ of the one leg period and the probability L _{2 of the} period of both legs, L ₁ is larger than L ₂ in the case (a) where no weight is attached, When weights are attached as in (b) and (c), it can be seen that L ₁ is smaller than L ₂ .

  In other words, when there is no problem in the left and right balance, the cycle of the left and right feet is almost the same, so good periodicity is obtained in one foot unit, whereas when the left and right foot cycles are different, one foot unit This is thought to be due to the fact that the periodicity at the center is broken, and is lower than the periodicity on both feet, with the two left and right steps as a set.

Therefore, as the amount of comparing the L ₁ and L _2, for example, by using a quantity defined as E = L ₁ / L ₂ by the ratio of L ₁ and L _2, it is possible to determine the left and right balance .
By collecting samples of footsteps with abnormal gait organs and footsteps without abnormalities, calculating the value of this E, and incorporating the results statistically learned by discriminant analysis etc. into the determination DB as a gait abnormality determination threshold, It is possible to output a determination result of an abnormality of the walking organ.

  In addition, normal walking sound is input for a long period of time, a large amount of normal values of E are taken over a long period of time, and statistical distribution information such as average values and variances are sequentially calculated, based on this By setting the determined threshold value in the determination DB, when an abnormal value deviating from the distribution is observed, it is detected that there is a possibility of an accident such as a gait organ abnormality or a stumbling, and output it. It becomes possible.

  As a second embodiment, a walking state determination method using joint position information obtained from a distance sensor will be described.

Here, a Kinect (registered trademark) sensor manufactured by Microsoft (registered trademark) was used as the distance sensor. The distance (depth) information of the observation target is obtained from the kinect sensor as distance image data arranged on a two-dimensional plane, and by observing it continuously in time, the three-dimensional shape of the observation target can be obtained with a video camera. It is obtained as moving image data of several tens of frames per second, similar to the one taken.
Further, the positions of the main joints of the human being are estimated from the distance image data and obtained as three-dimensional coordinate values.

Here, of the joint position information, the coordinate values of the right and left ankles _{(x l, y l, z} l) and _{(x r, y r, z} r) with, between them as in the following equation (7) The Euclidean distance was calculated and used as an input signal.

By replacing this value with P _i in the expression (1) of the first embodiment and applying the same processing as that of the first embodiment after that, it is possible to estimate the walking cycle and determine the left / right balance.
An example will be described with reference to FIG. As in Example 1, it can be seen that the right / left balance can be determined by comparing the probability of the one leg period and the both leg period.

In the present embodiment, the positions of the left and right ankles are estimated and the coordinate values thereof are used, but instead, the positions of the left and right knees may be estimated and the coordinate values may be used. Further, these coordinate values may be coordinate values obtained by estimating a leg part in the vicinity (for example, the left and right thighs and the instep of the foot) that moves in conjunction with or in cooperation with the leg part of the ankle or knee during walking.
Further, instead of the distance sensor in this embodiment, even in the case of using other sensors such as an acceleration sensor, for example by using a magnitude of a three-dimensional acceleration vector as P _i, it is similar to the processing methods can be applied.

  Elderly monitoring system that detects abnormalities in living conditions and accidents, early detection of abnormalities and deterioration of the elderly's gait function called locomotive syndrome, grasping the amount of activity for maintaining health in everyday life It can be used in industrial fields related to medical care, nursing care, welfare equipment, housing equipment, security, etc., such as rehabilitation support systems for motor organ disorders.

1 Decrease from maximum value as index of periodicity extraction 2 Maximum value of probability corresponding to cycle for 1 step and its position 3 Maximum value of probability corresponding to cycle for 2 steps and its position

Claims (10)

Acoustic information such as footsteps that occur when a pedestrian walks is input as an electrical signal by a sensor such as a microphone.
A feature extraction process for analyzing the electrical signal and converting it into a temporal change amount of a feature amount related to walking;
A walking cycle extraction process for analyzing the time variation and estimating the walking cycle (for one step and two steps) and its probability;
By applying the walking state analysis process that generates information on the walking state of the pedestrian from the estimated value of the walking cycle and its certainty obtained by applying
A walking state determination method, characterized in that information on the walking state of the pedestrian can be output.   In the feature extraction process, the influence of daily sounds other than footsteps and human voices can be reduced by converting the power (average energy) of the electrical signal in the frequency band of 8 kHz or higher into the time change amount. The walking state determination method according to claim 1, wherein:   Instead of the acoustic information, information obtained by observing a pedestrian's leg using a distance sensor is converted into an electric signal and input, and in the feature extraction process, the pedestrian's leg joint is obtained from the distance information obtained for the leg. 3. The walking state determination method according to claim 2, wherein the position is converted into a time change amount of the movement distance of the joint of the leg based on the estimated position information of the joint of the leg.   Instead of the acoustic information, pedestrian movement information is converted into an electrical signal using an acceleration sensor worn by the pedestrian, and in the feature extraction process, acceleration is converted into a total amount of change over time. The walking state determination method according to claim 2, wherein:   In the gait cycle extraction process, the time variation obtained by the feature extraction process is input, converted into any of autocorrelation, cepstrum, and average amplitude error, and further normalized. The walking state determination method according to any one of claims 2 to 4, wherein the extraction function is applied to estimate a walking cycle and output the probability.   In the walking state analysis process, based on the walking cycle and the probability value obtained in the walking cycle extraction process, by comparing with a certain reference value appropriately set in the determination database (determination DB), 6. The walking state determination method according to claim 5, wherein it is determined whether or not walking is performed in a time section corresponding to the input electric signal.   In the walking state analysis processing, based on the walking cycle and the value of the probability obtained by the walking cycle extraction processing, the probability corresponding to the cycle for each step and the cycle for two steps (a pair of left and right feet) 7. The method according to claim 6, wherein after extracting a corresponding probability, an estimated value representing whether the right and left balance is good or bad during walking is output by converting the two into a relative value for comparison. Walking state determination method.   The estimated value obtained by the walking state analysis process is stored for a certain period or longer, and the abnormal value is detected by comparing the statistical value with a reference value in a determination database, thereby detecting an abnormal walking. The walking state determination method according to claim 7.   A walking state determination program that executes the walking state determination method according to any one of claims 1 to 8, and a storage medium storing the program. A walking state determination characterized by comprising at least a sensor device (microphone, distance sensor, acceleration sensor, etc.) and an information processing device, and executing the walking state determination method according to any one of claims 1 to 8. apparatus.

Images