JP6804908B2 - Estimator, estimation method and computer program - Google Patents

Description

Embodiments of the present invention relate to estimation devices, estimation methods and computer programs.

Currently, as a method of positioning a pedestrian indoors where GPS (Global Positioning System) cannot be used, there is a method of estimating a position from sensor information of a terminal possessed by the pedestrian. However, terminals used for consumer use have a problem that positioning by integrating acceleration and angular velocity is difficult because errors such as noise and offset and timing of zero speed update cannot be detected. Therefore, there is a method of recognizing a pattern of a sensor waveform during walking, estimating the walking motion, the number of steps, and the stride length, and positioning the pedestrian. However, this method has a problem in the accuracy of estimating the walking direction and the moving distance. As described above, in the conventional technique, it may not be possible to accurately estimate information on walking such as walking direction, walking state, and stride length.

Japanese Unexamined Patent Publication No. 2004-264028 Japanese Unexamined Patent Publication No. 2005-114537 Japanese Unexamined Patent Publication No. 2012-145457

An object to be solved by the present invention is to provide an estimation device, an estimation method, and a computer program capable of improving the estimation accuracy of information related to walking.

The estimation device of the embodiment includes an acquisition unit, a posture estimation unit, a coordinate conversion unit, an angle calculation unit, a filter processing unit, a unit motion extraction unit, a motion estimation unit, a direction estimation unit, and a distance estimation unit. And have. The acquisition unit acquires the acceleration data which is the time-series data of the three-axis acceleration vector data of the person and the angular velocity data which is the time-series data of the three-axis angular velocity vector data from the acceleration sensor and the angular velocity sensor. The posture estimation unit estimates the postures of the acceleration sensor and the angular velocity sensor based on the acceleration data and the angular velocity data acquired by the acquisition unit. The coordinate conversion unit performs coordinate conversion of the acceleration data and the angular velocity data based on the postures of the acceleration sensor and the angular velocity sensor estimated by the attitude estimation unit. The angle calculation unit calculates time-series data in the moving direction from the angular velocity data after coordinate conversion by the coordinate conversion unit. The filter processing unit performs a filter process for passing the value of a certain frequency component to the acceleration data and the angular velocity data after the coordinate conversion by the coordinate conversion unit. The unit motion extraction unit detects a range in which the magnitude of the composite vector becomes the shape of one cycle of the triangular function from the acceleration data filtered by the filter processing section as a unit motion section representing the motion performed by the person. Then, the detected acceleration data and angular velocity data in the unit operation section are extracted as estimation time series data for each unit operation section. The motion estimation unit compares the estimation time series data extracted by the unit motion extraction unit with the template pattern of each motion of the person held in advance, and moves and moves without changing the direction of the sensor. Estimate the behavior without. The direction estimation unit estimates the moving direction based on the estimation time series data and the estimation result estimated by the motion estimation unit. The distance estimation unit, the time series data for the estimation, on the basis of the estimation result of the movement-estimating unit has estimated, estimates the movement distance of the person.

The schematic block diagram which shows the functional structure of the estimation apparatus 100 in 1st Embodiment. The schematic block diagram which shows the structure of the motion estimation part 71. The schematic block diagram which shows the structure of the distance estimation part 73. The flowchart which shows the process flow of the estimation apparatus 100 in 1st Embodiment. The figure which shows the sensor coordinate axis. The figure which shows the outline of the global coordinate transformation from a sensor coordinate axis. The figure which shows an example of a low-pass filter. The figure which shows an example of the waveform of a typical step. The figure for demonstrating the effect of applying a low-pass filter. The figure which shows the template pattern of the front walk and the back walk. The figure which shows an example of the traveling direction acceleration. The figure which shows the result of having calculated the DTW distance between the front walk template and the back walk template for each step in a series of walking motions. The figure which shows an example of the waveform pattern of the synthetic norm in the operation without movement. The figure for demonstrating the change of the sensor orientation with walking. Explanatory drawing when estimating the traveling direction based on the data of one step. The figure for demonstrating the method of estimating the traveling direction from the direction of a sensor. The figure which shows an example of a stride model. The figure which shows an example of the automatic selection of a variable. The schematic block diagram which shows the functional structure of the estimation apparatus 100a in 2nd Embodiment. The flowchart which shows the processing flow of the estimation apparatus 100a in 2nd Embodiment. The figure which shows an example of deep learning using convolution. The schematic block diagram which shows the functional structure of the estimation apparatus 100b in 3rd Embodiment. The flowchart which shows the process flow of the estimation apparatus 100b in 3rd Embodiment. The figure which shows an example of the storage layer.

Hereinafter, the estimation device, the estimation method, and the computer program of the embodiment will be described with reference to the drawings.
The estimation device in the embodiment acquires time-series data from a sensor installed in the person or a sensor provided in a terminal possessed by the person, and estimates information about walking of the person based on the acquired data. Here, the sensors are, for example, an acceleration sensor and an angular velocity sensor. In addition, the information related to the walking of a person includes movement, moving direction (also referred to as traveling direction), moving distance, and the like. In the following description, movement, movement direction, and movement distance will be described as examples of information regarding walking of a person. Hereinafter, a plurality of embodiments (first embodiment to third embodiment) will be described as examples of specific configurations.

(First Embodiment)
FIG. 1 is a schematic block diagram showing a functional configuration of the estimation device 100 according to the first embodiment. The estimation device 100 includes a CPU (Central Processing Unit), a memory, an auxiliary storage device, and the like connected by a bus, and executes an estimation program. By executing the estimation program, the estimation device 100 functions as a device including an acquisition unit 10, a posture estimation unit 20, a coordinate conversion unit 30, an angle calculation unit 40, a filter processing unit 50, a unit motion extraction unit 60, and an estimation unit 70. .. All or a part of each function of the estimation device 100 may be realized by using hardware such as ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), and FPGA (Field Programmable Gate Array). The estimation program may also be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a flexible disk, a magneto-optical disk, a portable medium such as a ROM (Read Only Memory) or a CD-ROM, or a storage device such as a hard disk built in a computer system. In addition, the estimation program may be transmitted and received via a telecommunication line.

The acquisition unit 10 receives three-axis acceleration vector time-series data (hereinafter referred to as “acceleration data”) and three-axis angular velocity vector time-series data (hereinafter, “angular velocity”) as time-series data from the acceleration sensor and the angular velocity sensor. Data ") is acquired. At this time, the acquisition unit 10 acquires the acceleration data and the angular velocity data as data with a time stamp.
The attitude estimation unit 20 estimates the attitude of the sensor that acquired each data based on the acceleration data and the angular velocity data acquired by the acquisition unit 10.
The coordinate conversion unit 30 performs coordinate conversion of acceleration data and angular velocity data based on the attitude of the sensor estimated by the attitude estimation unit 20.

The angle calculation unit 40 calculates time series data in the moving direction from the angular velocity data after coordinate conversion.
The filter processing unit 50 performs filter processing on the acceleration data and the angular velocity data after the coordinate conversion.
The unit motion extraction unit 60 obtains data for each unit motion section from the time series data based on the filtered acceleration data and angular velocity data and the time series data in the moving direction calculated by the angle calculation unit 40. Extract. Here, the unit movement represents, for example, the movement of the smallest unit constituting the movement performed by a person. Examples of actions performed by a person include walking three steps, jumping twice, and crouching twice. When the movement performed by the person is a three-step walk, each step is a unit movement. When the movement performed by the person is a jump twice, each of the movements constituting each jump is a unit movement. The movements that make up the jump include, for example, a stepping movement and a flying movement. That is, the double jump includes four unit movements (for example, because the stepping motion and the flying motion are performed twice each). Further, when the movement performed by the person is crouching twice, there are a squatting movement and a standing movement as unit movements. That is, crouching twice includes four unit movements (for example, because the crouching movement and the standing movement are performed twice each). The unit operation interval is a time interval of unit operation. For example, in the case of the above example (when the movement performed by the person is a three-step walk), the unit movement section is a time section of one step.

The estimation unit 70 estimates information (movement, movement direction, movement distance) regarding walking of a person based on the data for each unit movement section extracted by the unit movement extraction unit 60. The estimation unit 70 includes a motion estimation unit 71, a direction estimation unit 72, and a distance estimation unit 73. The motion estimation unit 71 estimates the motion in the data for each unit motion section based on the data for each unit motion section extracted by the unit motion extraction unit 60 and the environmental information. The environmental information is information about the surrounding environment where the sensor is located, for example, the gas concentration such as geomagnetism, atmospheric pressure, temperature, humidity, illuminance, volume, odor, oxygen concentration, and radio wave condition such as beacon. The direction estimation unit 72 estimates the moving direction in the data based on the data for each unit motion section extracted by the unit motion extraction unit 60. For example, the direction estimation unit 72 estimates a movement direction indicating which direction the sensor is moving with reference to the front surface in front of the sensor when walking forward or backward. The distance estimation unit 73 is based on the data for each unit motion section extracted by the unit motion extraction unit 60, the motion estimation result output from the motion estimation section 71, the user parameters, and the calibration data. Estimate the distance traveled in. User parameters represent personal information such as height, gender, and age. The calibration data includes offsets, model parameters, and the like.

FIG. 2 is a schematic block diagram showing the configuration of the motion estimation unit 71. As shown in FIG. 2, the motion estimation unit 71 includes an motion DB 711, a non-walking motion pattern matching section 712, a walking direction data pattern matching section 713, and a motion determination section 714.
The operation DB 711 is configured by using a storage device such as a magnetic hard disk device or a semiconductor storage device. The operation DB 711 stores a template pattern used for estimating a movement operation that does not change the direction of the sensor and an operation that does not involve movement. In the present embodiment, the moving motion that does not change the direction of the sensor is a motion that does not involve rotation of the body due to walking of a person such as forward walking, backward walking, and crab walking (left and right). In the present embodiment, the motion without movement is an motion performed in a place where a person is located, such as crouching, vertical jumping, stepping, body rotation, bending / stretching, back stretching, arm swinging, and raising hands.

The non-walking motion pattern matching unit 712 performs motion matching without movement based on the data for each unit motion section extracted by the unit motion extraction unit 60 and the template pattern stored in the motion DB 711.
The walking direction data pattern matching unit 713 matches the movement motion that does not change the direction of the sensor based on the data for each unit motion section extracted by the unit motion extraction unit 60 and the template pattern stored in the motion DB 711. Do.
The motion determination unit 714 determines the motion based on the matching results of the non-walking motion pattern matching section 712 and the walking direction data pattern matching section 713. The motion determination unit 714 outputs the motion determination result as the motion estimation result.

FIG. 3 is a schematic block diagram showing the configuration of the distance estimation unit 73. As shown in FIG. 3, the distance estimation unit 73 includes a conversion unit 731, a stride model DB 732, and a stride calculation unit 733.
The conversion unit 731 converts the data for each unit operation section extracted by the unit operation extraction unit 60 into information in the frequency domain.
The stride model DB732 is configured by using a storage device such as a magnetic hard disk device or a semiconductor storage device. The stride model DB732 stores the stride model.
The stride calculation unit 733 estimates the stride based on the frequency domain information converted by the conversion unit 731, the stride model stored in the stride model DB732, the user parameters, and the calibration data. The stride estimated by the stride calculation unit 733 is the distance traveled by one step.

FIG. 4 is a flowchart showing a processing flow of the estimation device 100 according to the first embodiment. The acquisition unit 10 acquires acceleration data and angular velocity data as time-series data from the acceleration sensor and the angular velocity sensor (step S101). The acquisition unit 10 outputs the acquired acceleration data and angular velocity data to the attitude estimation unit 20 and the coordinate conversion unit 30. The attitude estimation unit 20 inputs the acceleration data and the angular velocity data output from the acquisition unit 10. The attitude estimation unit 20 estimates the attitude angle of the sensor from which the acceleration data and the angular velocity data are acquired based on the input acceleration data and the angular velocity data (step S102). For the estimation of the attitude angle of the sensor, for example, the attitude angle estimation device described in Reference 1 may be used, or a generally well-known attitude estimation method may be used. (Reference 1: Japanese Unexamined Patent Publication No. 2004-264028) Here, the posture angle uniquely determines the posture of the sensor coordinate system in the global coordinate system. FIG. 5 is a diagram showing sensor coordinate axes. As shown in FIG. 5, three axes of the vertical direction, the traveling direction, and the horizontal direction are used as the sensor coordinate axes. The attitude estimation unit 20 outputs the estimation result of the attitude angle to the coordinate conversion unit 30.

The coordinate conversion unit 30 inputs the acceleration data and the angular velocity data output from the acquisition unit 10 and the estimation result output from the coordinate conversion unit 30. The coordinate conversion unit 30 converts the coordinates of the acceleration data and the angular velocity data (converts from the sensor coordinates to the global coordinates) from the input estimation result of the posture angle (step S103). Specifically, the coordinate conversion unit 30 performs conversion between two coordinate systems using a rotation matrix, a quaternion, or the like. FIG. 6 is a diagram showing an outline of global coordinate transformation from the sensor coordinate axis. For the coordinate transformation, a generally well-known method such as a posture calculation algorithm using a quaternion may be used, or another method may be used. The coordinate conversion unit 30 outputs the angular velocity data after the coordinate conversion to the angle calculation unit 40. Further, the coordinate conversion unit 30 outputs the acceleration data and the angular velocity data after the coordinate conversion to the filter processing unit 50.

The angle calculation unit 40 inputs the angular velocity data after coordinate conversion output from the coordinate conversion unit 30. The angle calculation unit 40 integrates the input angular velocity data after coordinate conversion, and calculates angles for the traveling direction axis, the vertical direction axis, and the lateral direction axis (step S104). The angle of the vertical axis indicates the direction in which the user is moving in the horizontal plane, the angle of the traveling axis indicates the lateral sway of the user, and the angle of the lateral axis indicates the sway of the user back and forth. ..

The filter processing unit 50 inputs the acceleration data and the angular velocity data after the coordinate conversion output from the coordinate conversion unit 30. The filter processing unit 50 performs a filter process for passing the value of a certain frequency component to each of the input coordinate-converted acceleration data and angular velocity data (step S105). For example, the filter processing unit 50 applies a low-pass filter that passes a general frequency (for example, 2 Hz) of walking motion to acceleration data. This is because the clothes vibrate at the time of landing in the walking motion, and the vibration component may be superimposed on the measured value of the sensor. As a result, an error occurs in the detection of the one-step section, which becomes an error in the stride length, and the estimation error of the moving distance may become large. Therefore, the filter processing unit 50 applies a low-pass filter in order to reduce such a problem. Here, an example of a low-pass filter is shown in FIG. Further, for example, the filter processing unit 50 applies a high-pass filter in order to reduce the influence of drift on the angular velocity data. The filter processing unit 50 outputs the acceleration data and the angular velocity data after the filter processing to the unit motion extraction unit 60.

The unit motion extraction unit 60 inputs the angle data output from the angle calculation unit 40, and the acceleration data and the angular velocity data after the filter processing output from the filter processing unit 50. The unit motion extraction unit 60 calculates the norm of the composite vector from the input acceleration data (step S106). The magnitude of the composite vector may be used as the norm of the composite vector. The unit motion extraction unit 60 determines whether or not the unit motion interval can be detected based on the calculated norm of the composite vector (step S107). Since the unit motion is equivalent to the typical one-step motion as shown in FIG. 8, the unit motion extraction unit 60 sets the portion where the magnitude of the composite vector is the shape of one cycle of the trigonometric function as the unit motion section. Detect as. At this time, if the low-pass filter is not applied to the acceleration data, high-frequency components such as vibrations are superimposed on the composite vector as shown in FIG. 9, and the unit operation section cannot be detected correctly.

When the unit operation section cannot be detected (step S107-NO), the estimation device 100 repeatedly executes the processes after step S101.
On the other hand, when the unit motion section can be detected (step S107-YES), the unit motion extraction unit 60 estimates the acceleration data, the angular velocity data, and the angle data in the unit motion section for each unit motion section. Extract as data (step S108). The unit motion extraction unit 60 outputs the extracted time series data for estimation to the motion estimation unit 71, the direction estimation unit 72, and the distance estimation unit 73.

The motion estimation unit 71 inputs the environmental information and the estimation time series data output from the unit motion extraction unit 60. The motion estimation unit 71 estimates the motion in the unit motion section from the input estimation time series data (step S109). Specifically, the motion estimation unit 71 estimates a movement motion that does not change the direction of the sensor and a motion that does not involve movement.

When estimating the movement motion that does not change the orientation of the sensor, the motion estimation unit 71 results in an motion that is closer to the template as compared with the template pattern shown in FIG. FIG. 10 is a diagram showing template patterns for forward walking and backward walking. In FIG. 10, the horizontal axis represents time and the vertical axis represents amplitude. FIG. 10 shows two template patterns 81 and 82. Among the template patterns shown in FIG. 10, the template pattern 81 represents the template pattern for walking forward, and the template pattern 82 represents the template pattern for walking backward. The motion estimation unit 71 holds such a template pattern in the motion DB 711 in advance. Here, the specific processing of the motion estimation unit 71 will be described.

For example, when the acceleration in the traveling direction has the waveform shown on the left side of FIG. 11, the motion estimation unit 71 estimates that the movement is forward walking because the shape is similar to that of the forward walking template of FIG. On the other hand, when the acceleration in the traveling direction has the waveform shown on the right side of FIG. 11, the motion estimation unit 71 estimates that the vehicle is walking backward because the shape is similar to the template for walking backward in FIG. Similarly, the motion estimation unit 71 makes an estimation result in a direction similar to that of the template in the right direction and the left direction in the acceleration in the lateral direction. At this time, the motion estimation unit 71 may determine whether or not they are similar by using the Euclidean distance of the acceleration waveform and the template waveform, may determine by using the DTW (Dynamic Time Warping) distance, and the like. You may use the method of. Then, the motion estimation unit 71 may set the template having the shortest distance to be the most similar. When evaluating the similarity using this distance, the amplitude of each waveform may be standardized and evaluated. For example, the magnitude of the amplitude may be standardized so that the amplitudes of all the waveforms are -1 to 1.

FIG. 12 is a diagram showing the result of calculating the DTW distance between the forward walking template and the backward walking template for each step in a series of walking motions. From this result, since the DTW distance between the acceleration waveform and the forward walking template is smaller than the DTW distance between the acceleration waveform and the backward walking template, the motion estimation unit 71 estimates that the motion is forward walking. As described above, the motion estimation unit 71 may adopt the motion having the most similar characteristics (waveform) to the possessed template. If the distances from the template are all larger than the threshold value, the motion estimation unit 71 can estimate that the template is not moving. FIG. 13 shows an example of the waveform pattern of the synthetic norm in the operation without movement. As shown in FIG. 13, it can be seen that even the operation without movement is composed of a combination of unit operation waveforms. Therefore, the motion estimation unit 71 can estimate the motion even if the motion does not involve movement by estimating while changing the state by pattern matching for each unit motion. Moreover, when the operation is not decided to be one, it may be classified as the operation unknown. In addition, when the operation is unknown, it is possible to estimate whether or not the vehicle is moving by utilizing the environmental information. After that, the motion estimation unit 71 outputs the motion estimation result to the direction estimation unit 72 and an external device.

The direction estimation unit 72 inputs the estimation time series data output from the unit motion extraction unit 60 and the motion estimation result output from the motion estimation unit 71. The direction estimation unit 72 estimates the moving direction based on the input estimation time series data and the motion estimation result (step S110). Specifically, as shown in FIG. 14, the direction estimation unit 72 estimates how much the rotation occurs in one step during forward walking and backward walking. When θ = 0 in FIG. 14, it can be estimated that the motion is always straight and linear. Since humans walk while twisting their hips, the orientation of the sensor is like the waveform of a trigonometric function as shown in FIG. At this time, the orientation of the sensor is usually changed so as to have one cycle in two steps. Therefore, the direction estimation unit 72 first calculates the average angular velocity for the sensor between two steps. Next, the direction estimation unit 72 multiplies the calculation result by the time for one step. This makes it possible to estimate the amount of rotation per step, which is the direction of travel.

In addition, a time delay and a buffer are required to calculate the average angular velocity for two steps. Therefore, as shown in FIG. 15, the traveling direction can be similarly estimated only by the data of one step. Further, as shown in FIG. 16, the direction estimation unit 72 internally divides the time when the composite norm becomes the minimum value and the time when the composite norm becomes the maximum value from the angle data in one target step (FIG. The angle corresponding to the time indicated by the point P at 16) may be estimated as the moving direction. Here, the time for internal division is, for example, a time for internally dividing the minimum value time and the maximum value time at a ratio of 1: 3. The time to be internally divided is the time related to the movement of the person from raising his / her foot to landing. Normally, it is considered to be a relatively robust method because it does not take any action such as standing still with the legs raised when walking, and it is executed with a constant rhythm from the time when the legs are raised until the landing. After that, the direction estimation unit 72 outputs the direction estimation result to an external device.

The distance estimation unit 73 inputs the estimation time series data output from the unit motion extraction unit 60, the motion estimation result output from the motion estimation unit 71, user parameters, and calibration data. The conversion unit 731 calculates the power spectrum by converting the acceleration data in the input estimation time series data into the data in the frequency domain. The conversion unit 731 outputs the calculated power spectrum to the stride calculation unit 733. The stride calculation unit 733 calculates the stride based on the power spectrum output from the conversion unit 731, the user parameters input from the external device, and the stride model parameters read from the stride model DB 732. The stride calculation unit 733 uses the calculated stride as the estimation result of the moving distance (step S111). The model for estimating a general stride is the Weinberg model shown in Equation 1 below.

In the Wienberg model, it is necessary to calibrate the parameter K for each individual by using the difference between the maximum value and the minimum value of the acceleration in the vertical direction. In this method, since the user parameters are input to the model, it is not necessary to calibrate each individual. Further, in this method, since the acceleration is converted into a power spectrum and used, it can be expected that it will be less affected by noise. As the coefficient of the power spectrum used in the stride model, the power spectrum having a frequency closest to the walking frequency of 2 Hz may be used, or the power spectrum having a frequency of 10 Hz or less may be used. The stride model used in this method is shown in FIG. As shown in FIG. 17, the power spectrum coefficient is used in the stride model used in this method. At this time, as shown in FIG. 18, various variable selection methods may be used to select variables and aim at improving generalization performance.

If the stride result estimated using the previous stride model is different from the measured value, the difference may be input as an offset, and the value with the offset added may be output as the estimation result for the subsequent estimation results. Good. You may also update the stride model by entering parameters for the new stride model. After that, the distance estimation unit 73 outputs the estimation result of the travel distance to an external device. The motion estimation unit 71, the direction estimation unit 72, and the distance estimation unit 73 can output the output value every N steps, output every M seconds (M is a real number), and take various output timings. it can. Further, the distance estimation unit 73 may not only output the moving distance but also output the absolute coordinate value by inputting the coordinates of the initial point.

According to the estimation device 100 configured as described above, it is possible to improve the estimation accuracy of the walking direction, the walking state, and the stride length. Specifically, the estimation device 100 detects the unit operation section based on the acquired acceleration data and the angular velocity data, and performs the operation during the unit operation from the acceleration data, the angular velocity data, and the angle data in the detected unit operation section. presume. When estimating the motion, the estimation device 100 estimates the motion motion by comparing the template of the motion motion held and the acceleration data. Further, the estimation device 100 estimates the traveling direction based on the average angular velocity of the sensor for two steps. Further, the estimation device 100 calculates the power spectrum by converting the acceleration data in the time series data into the data in the frequency domain, and calculates the stride based on the calculated power spectrum and the stride model parameter. Therefore, it is possible to improve the estimation accuracy of information related to walking.
Further, the estimation device 100 can reduce the amount of calculation related to positioning, improve robustness against noise such as vibration, and improve the estimation accuracy of the movement direction and the movement distance.

In addition to integrating, the angle calculation unit 40 may use a simple sum of the angular velocity data as the angle, or integrates only the angular velocity data having a certain frequency component in order to eliminate error factors such as drift. The angle may be calculated.

(Second Embodiment)
FIG. 19 is a schematic block diagram showing a functional configuration of the estimation device 100a according to the second embodiment. The estimation device 100a includes a CPU, a memory, an auxiliary storage device, and the like connected by a bus, and executes an estimation program. By executing the estimation program, the estimation device 100a functions as a device including an acquisition unit 10, a posture estimation unit 20, a coordinate conversion unit 30, a filter processing unit 50, a unit motion extraction unit 60, and an estimation unit 70a. In addition, all or a part of each function of the estimation device 100a may be realized by using hardware such as ASIC, PLD and FPGA. The estimation program may also be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a flexible disk, a magneto-optical disk, a portable medium such as a ROM or a CD-ROM, or a storage device such as a hard disk built in a computer system. In addition, the estimation program may be transmitted and received via a telecommunication line.

The estimation device 100a is different from the estimation device 100 in that the unit motion extraction unit 60a and the estimation unit 70a are provided instead of the unit motion extraction unit 60 and the estimation unit 70, and the angle calculation unit 40 is not provided. The estimation device 100a is the same as the estimation device 100 in other configurations. Therefore, the description of the entire estimation device 100a will be omitted, and the unit motion extraction unit 60a and the estimation unit 70a will be described.

The unit motion extraction unit 60a extracts data for each unit motion from the time series data based on the filtered acceleration data and angular velocity data and the time series data in the moving direction calculated by the angle calculation unit 40. To do.
The estimation unit 70a estimates the movement, movement direction, and movement distance of the person based on the data for each unit movement section extracted by the unit movement extraction unit 60a. The estimation unit 70a includes a deep learning unit 74, a learning result storage unit 75, a format conversion unit 76, and a deep learning estimation unit 77.

The deep learning unit 74 inputs angular velocity data and acceleration data in a unit motion section, correct motion, movement direction, and travel distance as learning data from the outside, and deep learning is based on the input learning data. The parameters used in the estimation unit 77 are learned. The deep learning unit 74 performs learning when a learning start signal is input from the outside. The learning start signal is a signal for executing learning by the deep learning unit 74. The learning start signal and learning data may be input using an external input device. The input device is, for example, a keyboard, a mouse, or the like.
The learning result storage unit 75 is configured by using a storage device such as a magnetic hard disk device or a semiconductor storage device. The learning result storage unit 75 stores the parameters learned by the deep learning unit 74.

The format conversion unit 76 converts the time-series data extracted by the unit motion extraction unit 60 and the environment information input from the outside into fixed-length format data.
The deep learning estimation unit 77 walks a person in a unit operation section based on information in the frequency domain of angular velocity data and acceleration data, user parameters input from the outside, and parameters stored in the learning result storage unit 75. Estimate information about.

FIG. 20 is a flowchart showing a processing flow of the estimation device 100a in the second embodiment. In FIG. 20, the same processing as in FIG. 4 is designated by the same reference numerals as in FIG. 4, and the description thereof will be omitted. When the unit motion section can be detected in the process of step S107 (step S107-YES), the unit motion extraction unit 60a determines the acceleration data, the angular velocity data, the angle data, and the angle data in the unit motion section for each unit motion section. Environmental information is extracted as time-series data for estimation (step S201). The unit motion extraction unit 60a outputs the extracted time series data for estimation to the format conversion unit 76.

The format conversion unit 76 inputs the estimation time series data output from the unit motion extraction unit 60a. The format conversion unit 76 calculates frequency information from the input estimation time series data (step S201). Specifically, the format conversion unit 76 needs to make the input data a fixed length for deep learning, and therefore the time series length differs for each step. Therefore, the acceleration data and the angular velocity data in the estimation time series data And, the data length between unit operations is converted into a certain fixed length format by the method of interpolation, extrapolation, etc. of the environmental data. Here, as an example, the format conversion unit 76 calculates each power spectrum and FFT coefficient of each time series data, and deep learning estimation using a fixed number of power spectra and FFT coefficient as fixed length format data. Output to unit 77 (step S202).

The deep learning estimation unit 77 inputs the parameters stored in the learning result storage unit 75 and the fixed-length format data output from the format conversion unit 76. The deep learning estimation unit 77 uses the power spectrum including the walking frequency among the power spectra included in the input fixed-length format data, and uses the power spectrum as the fixed-length power spectrum data in the deep learning model (multilayer neural network). Enter and estimate the movement, movement direction, and movement distance. For the deep learning model, not only a fully connected multi-layer neural network but also a convolution layer and a multi-layer neural network using pooling may be used. Here, the number of convolution layers is determined from the number of phases of one-step movement (phase example: recoil, foot up, foot down, landing), and the filter size of the pooling layer is determined by the power spectrum near the walking frequency component. It may be determined based on the width of. It is necessary to learn the parameters such as the filter coefficient of the deep learning model in advance. For this purpose, the power spectrum, which is input from the outside and is obtained by converting the data during the unit operation period into the frequency domain, and the learning data, which is a set of the corresponding correct operation, movement direction, and movement distance, are input and deep. The learning unit 74 learns the deep learning model, and the result is stored in the deep learning estimation unit 77. FIG. 21 shows an example of deep learning using convolution.

According to the estimation device 100a configured as described above, learning by large-scale walking data becomes possible by applying deep learning. At that time, labor saving can be realized by automating the stride model to be learned in advance, and robustness against individual differences and daily fluctuations can be greatly improved.

(Third Embodiment)
FIG. 22 is a schematic block diagram showing the functional configuration of the estimation device 100b according to the third embodiment. The estimation device 100b includes a CPU, a memory, an auxiliary storage device, and the like connected by a bus, and executes an estimation program. By executing the estimation program, the estimation device 100b functions as a device including an acquisition unit 10, a posture estimation unit 20, a coordinate conversion unit 30, a filter processing unit 50, and an estimation unit 70b. In addition, all or a part of each function of the estimation device 100b may be realized by using hardware such as ASIC, PLD and FPGA. The estimation program may also be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a flexible disk, a magneto-optical disk, a portable medium such as a ROM or a CD-ROM, or a storage device such as a hard disk built in a computer system. In addition, the estimation program may be transmitted and received via a telecommunication line.

The estimation device 100b is different from the estimation device 100 in that the estimation unit 70b is provided instead of the estimation unit 70, and the angle calculation unit 40 and the unit motion extraction unit 60 are not provided. The estimation device 100b is the same as the estimation device 100 in other configurations. Therefore, the description of the entire estimation device 100b will be omitted, and the estimation unit 70b will be described.

The estimation unit 70b estimates information on walking of a person based on acceleration data and angular velocity data filtered by the filtering unit 50. The estimation unit 70b includes a deep learning unit 74b, a learning result storage unit 75, a deep learning estimation unit 77b, and a unit data extraction unit 78.
The deep learning unit 74b inputs the angular velocity data and the acceleration data during a certain period of time, and the correct motion, moving direction, and moving distance as learning data from the outside, and based on the input learning data, the deep learning estimation unit 77b. Learn the parameters used in. The deep learning unit 74b performs learning when a learning start signal is input from the outside.

The learning result storage unit 75 is configured by using a storage device such as a magnetic hard disk device or a semiconductor storage device. The learning result storage unit 75 stores the parameters learned by the deep learning unit 74b.
The unit data extraction unit 78 extracts time-series data at a certain time based on the acceleration data and the angular velocity data after the filter processing output from the filter processing unit 50 and the environmental information.
The deep learning estimation unit 77b is based on angular velocity data and acceleration data during a certain period of time, user parameters input from the outside, and parameters stored in the learning result storage unit 75, and operates and moves in this time. , Estimate the distance traveled.

FIG. 23 is a flowchart showing a processing flow of the estimation device 100b according to the third embodiment. In FIG. 23, the same processing as in FIG. 4 is designated by the same reference numerals as in FIG. When the processing of step S105 is performed, the unit data extraction unit 78 acquires the acceleration data and the angular velocity data after the filtering processing output from the filter processing unit 50. The unit data extraction unit 78 determines whether or not the acceleration data and the angular velocity data for N seconds have been accumulated (step S301). This determination is for inputting to the deep learning estimation unit 77b as time series data for estimation. When the data for N seconds is not accumulated (step S301-NO), the estimation device 100b repeatedly executes the processes after step S101.

On the other hand, when N seconds of acceleration data and angular velocity data are accumulated (step S301-YES), the unit data extraction unit 78 uses the accumulated N seconds of acceleration data and angular velocity data as time series data for estimation. It is output to the deep learning estimation unit 77b. The deep learning estimation unit 77b inputs the parameters stored in the learning result storage unit 75 and the estimation time series data output from the unit data extraction unit 78. The deep learning estimation unit 77b estimates the motion, the moving direction, and the moving distance by using the input estimation time series data, the parameters, and the deep learning model (step S302). In this embodiment, deep learning having a storage layer (recurrent type multi-layer neural network) is used. Since this model has a storage layer, it is possible to input time series data. FIG. 24 shows an example of the storage layer. Even if it has a storage layer, it can be used together because it is possible to apply convolutional deep learning.

According to the estimation device 100b configured as described above, by applying deep learning having a memory function, it is not necessary to cut out the unit operation as in the first embodiment and the second embodiment, and the positioning algorithm Can be carried out within deep learning, and the estimation accuracy can be significantly improved according to the scale of learning data.

According to at least one embodiment described above, the unit motion extraction unit 60 for extracting the data for each unit motion section representing the motion performed by the person obtained from the time series data of the angular velocity data and the acceleration data of the person, and the unit motion By having the estimation unit 70 that estimates the information about the walking of a person in the data based on the data for each unit motion section extracted by the extraction unit 60, the estimation accuracy of the information about walking can be improved.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, as well as in the scope of the invention described in the claims and the equivalent scope thereof.

10 ... acquisition unit, 20 ... attitude estimation unit, 30 ... coordinate conversion unit, 40 ... angle calculation unit, 50 ... filter processing unit, 60, 60a ... unit motion extraction unit, 70, 70a, 70b ... estimation unit, 71 ... motion Estimating unit, 711 ... Motion DB, 712 ... Out-of-walking motion pattern matching unit, 713 ... Walking direction data pattern matching unit, 714 ... Motion determination unit, 72 ... Direction estimation unit, 73 ... Distance estimation unit, 74, 74b ... Deep learning Unit, 75 ... Learning result storage unit, 76 ... Format conversion unit, 77, 77b ... Deep learning estimation unit, 78 ... Unit data extraction unit, 731 ... Conversion unit, 732 ... Stride model DB, 733 ... Stride calculation unit, 74 ... Deep learning unit, 100, 100a, 100b ... Estimator

Claims (7)

An acquisition unit that acquires acceleration data, which is time-series data of three-axis acceleration vector data of a person, and angular velocity data, which is time-series data of three-axis angular velocity vector data, from an acceleration sensor and an angular velocity sensor.
A posture estimation unit that estimates the postures of the acceleration sensor and the angular velocity sensor based on the acceleration data and the angular velocity data acquired by the acquisition unit.
A coordinate conversion unit that performs coordinate conversion of the acceleration data and the angular velocity data based on the postures of the acceleration sensor and the angular velocity sensor estimated by the attitude estimation unit.
An angle calculation unit that calculates time-series data in the moving direction from the angular velocity data after coordinate conversion by the coordinate conversion unit.
A filter processing unit that performs a filter process for passing a certain frequency component value through the acceleration data and the angular velocity data after the coordinate conversion by the coordinate conversion unit.
From the acceleration data filtered by the filtering unit, a range in which the magnitude of the composite vector becomes the shape of one cycle of the triangular function is detected as a unit operation section representing the operation performed by the person, and the detected unit. A unit motion extraction unit that extracts acceleration data and angular velocity data in the motion section as estimation time series data for each unit motion section.
The estimation time-series data extracted by the unit motion extraction unit is compared with the template pattern of each motion of the person held in advance, and the movement motion that does not change the direction of the sensor and the motion that does not involve movement are performed. Motion estimation unit to estimate and
A direction estimation unit that estimates the movement direction based on the estimation time series data and the estimation result estimated by the motion estimation unit.
And time-series data for the estimation, on the basis of the estimation result of the movement-estimating unit has estimated the distance estimation unit that estimates a movement distance of the person,
Estimator equipped with. The estimation device according to claim 1, wherein the movement motion that does not change the direction of the sensor estimated by the motion estimation unit includes forward walking, backward walking, and crab walking of movement to the left and right. The estimation device according to claim 1 or 2, wherein the motion estimation unit estimates that the movement is not moving when the similarity with all the template patterns is smaller than a certain value. The direction estimation unit is described in any one of claims 1 to 3, wherein the direction estimation unit calculates the target step and the average angular velocity one step before the target step, and sets the angle obtained by multiplying the average angular velocity by the time of one step as the moving direction. Estimator. The direction estimation unit estimates the angle corresponding to the time when the norm of the composite vector obtained from the angular velocity data is the minimum value and the time when the maximum value is internally divided in the target step as the moving direction. , The estimation device according to any one of claims 1 to 3. An acquisition step of acquiring acceleration data, which is time-series data of 3-axis acceleration vector data of a person, and angular velocity data, which is time-series data of 3-axis angular velocity vector data, from an acceleration sensor and an angular velocity sensor.
A posture estimation step that estimates the postures of the acceleration sensor and the angular velocity sensor based on the acceleration data and the angular velocity data acquired in the acquisition step.
A coordinate conversion step that performs coordinate conversion of the acceleration data and the angular velocity data based on the postures of the acceleration sensor and the angular velocity sensor estimated in the posture estimation step, and
An angle calculation step for calculating time series data in the moving direction from the angular velocity data after coordinate conversion in the coordinate conversion step, and
A filter processing step that performs a filter process for passing a certain frequency component value through the acceleration data and the angular velocity data after the coordinate conversion in the coordinate conversion step, and
From the acceleration data filtered in the filtering step, a range in which the magnitude of the composite vector becomes the shape of one cycle of the triangular function is detected as a unit operation section representing the operation performed by the person, and the detected unit. A unit motion extraction step for extracting acceleration data and angular velocity data in the motion section as estimation time series data for each unit motion section, and
The estimation time-series data extracted in the unit motion extraction step is compared with the template pattern of each motion of the person held in advance, and the motion motion that does not change the orientation of the sensor and the motion that does not involve movement are performed. Motion estimation step to estimate and
A direction estimation step for estimating the movement direction based on the estimation time series data and the estimation result estimated in the motion estimation step,
And time-series data for the estimation, on the basis of the estimation result estimated in the motion estimation step, a distance estimation step of estimating a moving distance of the person,
Estimating method with. An acquisition step of acquiring acceleration data, which is time-series data of 3-axis acceleration vector data of a person, and angular velocity data, which is time-series data of 3-axis angular velocity vector data, from an acceleration sensor and an angular velocity sensor.
A posture estimation step that estimates the postures of the acceleration sensor and the angular velocity sensor based on the acceleration data and the angular velocity data acquired in the acquisition step.
A coordinate conversion step that performs coordinate conversion of the acceleration data and the angular velocity data based on the postures of the acceleration sensor and the angular velocity sensor estimated in the posture estimation step, and
An angle calculation step for calculating time series data in the moving direction from the angular velocity data after coordinate conversion in the coordinate conversion step, and
A filter processing step that performs a filter process for passing a certain frequency component value through the acceleration data and the angular velocity data after the coordinate conversion in the coordinate conversion step, and
From the acceleration data filtered in the filtering step, a range in which the magnitude of the composite vector becomes the shape of one cycle of the triangular function is detected as a unit operation section representing the operation performed by the person, and the detected unit. A unit motion extraction step for extracting acceleration data and angular velocity data in the motion section as estimation time series data for each unit motion section, and
The estimation time-series data extracted in the unit motion extraction step is compared with the template pattern of each motion of the person held in advance, and the motion motion that does not change the orientation of the sensor and the motion that does not involve movement are performed. Motion estimation step to estimate and
A direction estimation step for estimating the movement direction based on the estimation time series data and the estimation result estimated in the motion estimation step,
And time-series data for the estimation, on the basis of the estimation result estimated in the motion estimation step, a distance estimation step of estimating a moving distance of the person,
A computer program that lets a computer run.

Images