JP2009238196A - Method and system for estimating moving state of portable terminal apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and a system for estimating the moving state of a user in a short period of time and with high accuracy by using a model group for estimating the moving state of the user. <P>SOLUTION: The method estimates the moving state of the portable terminal apparatus which includes at least a CPU, a memory and an environment information acquiring apparatus. The memory stores a group of first symbol strings composed by at least two or more symbols (ID) which represent a prescribed moving state. The CPU is provided with: a step which symbolizes the environment information value which is collected for a prescribed period by the environment information acquiring apparatus; a step which repeats the symbolizing step by shifting the prescribed period by a constant period for at least two or more times and generating a second symbol string which is composed of a plurality of symbolized environment information values which are repeatedly generated; and a step deriving the likelihood of the second symbol string against the first symboll string. By specifying the first symbol string in which the likelifood is maximum, the moving state can be estimated. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

TECHNICAL FIELD OF THE INVENTION

  The present invention relates to a method and system for estimating a user's movement state, and more particularly, to a method and system for automatically estimating a user's movement state in a ubiquitous environment using a mobile terminal device.

  In recent years, there has been a need to detect a user's movement state, or to automatically recognize and identify a user's movement state accurately, and various techniques relating to estimation of the user's movement state have been proposed. However, conventionally, identifying a certain state of the user, for example, a state where the user is stopped, a state where he / she is on a car, a bus, a train, etc. is an estimation of a moving state even if information from the user is measured over a long period of time. The accuracy was low. In particular, it was difficult to estimate the state of being on a car, a bus, or a train. Conventionally, when estimating a user's movement state, one power spectrum is selected at regular intervals using data obtained from an acceleration sensor possessed by the user, and the selected power spectrum and representative power spectrum are obtained. The movement state was estimated by comparison. Accordingly, one selected power spectrum does not include a time-series change, and if the acceleration sensor suddenly changes, the user state may not be estimated correctly. For this reason, it is difficult for conventional power spectrum pattern matching to cope with sudden noise and vibration pattern fluctuations, which leads to a decrease in the estimation accuracy of the moving state, or measurement of a long time window. there were.

  Further, in the conventional system, when selecting the representative power spectrum group from the data group obtained from the acceleration sensor, the representative power spectrum group is selected manually from the acceleration data group or simply by threshold judgment. Therefore, selection of the representative power spectrum by hand or only by simple threshold judgment is synonymous with selecting only one representative power spectrum, and the dynamic setting of the representative power spectrum according to the learning situation is Not done.

  Further, in the conventional system, the state of the user is estimated by comparing one estimation power spectrum that does not include a time series change and the representative power spectrum. However, the form of the power spectrum used to estimate the user's movement state differs depending on the individual user, and changes in time series even for the same user. Therefore, when the user's estimation power spectrum includes a sudden change, the user's movement state cannot be estimated correctly.

Further, in Non-Patent Document 1, acceleration sensors are attached to a plurality of locations on the body, and “sit”, “stand”, “walk”, “up stairs”, “down stairs”, “shake hands”, “write on blackboard”, “keyboard” Motion estimation such as “typing”. In Non-Patent Document 2, 20 types of motion estimation are performed by attaching acceleration sensors to five locations of both wrists, both ankles and thighs. In Non-Patent Document 3, by adding a plurality of sensor information to the mobile terminal device, it is classified into four states of “capturing”, “active”, “free”, and “normal”, and the incoming volume is set to silent mode. An application to be implemented is implemented. In Non-Patent Document 4, research on a smartphone equipped with a plurality of sensors in a mobile terminal device is performed, and the states of “walking”, “running”, and “sitting” are estimated using an acceleration sensor and a gyro sensor. . However, these can estimate various human operating states using various sensors, but a plurality of sensor mounting locations are required, or the mounting method is fixed, and mobile phones with various possessing states exist. In a terminal device, it is not realistic. Non-Patent Document 5 proposes a method of identifying the sensor wearing state using one acceleration sensor and estimating the states of “walking”, “standing”, “sitting”, and “running”. In addition, it was not possible to estimate the movement state including the riding state of bicycles, cars, buses, and the like.
Nicky Kern, "A Model for Human Interruptability: Experimental Evaluation and Automatic Estimation from Wearable Sensors"ISWC'04, pp.158-165 (2004). Stephen S. Intille, "Acquiring In Situ Training Data for Context-Aware Ubiquitous Computing Applications" CHI 2004, ACM Press, pp. 1-9 (2004). Daniel Siewiorek, "SenSay: A Context-Aware Mobile Phone"ISWC'03 (2003). Toshiki Iso, "Personal Context Extractor with Multiple Sensor on a Cell Phone" D. 2 C200525 (2005). Y. Kawahara, "A Context-Aware Content Delivery Service Using Off-the-shelf Sensors" Mobisys2004 (2004). Kenji Kita et al., “Probabilistic Language Model”, 1999 LE Baum et al., “A maximization technique occurring in the statistical analysis of probabilistic function of markov chains”, 1970 , Akaike, “New Approach to Statistical Testing”, 1979

  The present invention aims to solve the above-described problems, and shortens the movement state of a user by using an external environment information acquisition device such as an acceleration sensor mounted on a portable terminal device that fits in the palm of a person. The present invention provides a method and system for automatically estimating with high accuracy in time. Here, the moving state of the user refers to, for example, a state where the user is stopped, walking, running, a state where the user is riding a bicycle, a car, a bus, and a train. Hereinafter, the above-described states are referred to as a stopped state, a walking state, a running state, a bicycle state, an automobile state, a bus state, and a train state, respectively. Note that the mobile terminal device may be a size that fits in the palm of a person or a size smaller than that.

  The present invention generates a model group for estimating a user's moving state, calculates an evaluation value for each model using time series data of the user's moving state, and moves corresponding to the model having the highest evaluation value The purpose is to estimate the movement state of the user.

  Further, when using an acceleration sensor, the power spectrum obtained by simply FF-converting the user's acceleration data value obtained from the acceleration sensor does not have information on the time series change of the acceleration data. The present invention estimates a user's movement state using time series data of a power spectrum. Thereby, in the estimation model group in the present invention, since the user's movement state is estimated using time-series data indicating the user's movement state, even if the user movement state includes a sudden change, It is possible to correctly estimate the movement state of the.

  Furthermore, the present invention uses a power spectrum randomly selected from a group of power spectra for learning, preferably sets a parameter by a genetic algorithm (hereinafter referred to as GA), and uses a GA to obtain a representative power spectrum having a high evaluation value. It relates to the system to discover. That is, an object of the present invention is to provide a learning system apparatus and method for automatically selecting a representative power spectrum having a high evaluation value from a learning power spectrum group.

  Furthermore, in the present invention, it is possible to estimate the user's movement state in a shorter time by changing the length of time required to estimate the user's movement state in multiple stages according to information obtained in a predetermined period. The purpose is to reduce the average time required to estimate the movement state.

  Furthermore, in the present invention, a movement state schema having a movement state group is generated, the movement state schema is used to first identify the movement state group of the user, and then specify the movement state of the user included in the movement state group. Thus, it is an object to estimate the movement state of the user in a shorter time.

  Furthermore, in the present invention, an acceleration sensor, a microphone sensor, and a positioning system receiver are mounted on the mobile terminal device, and the movement state of the user is estimated using these sensors in combination. Thus, the moving state of the user can be estimated without depending on the possessing state of the mobile terminal device. It is another object of the present invention to apply a method suitable for estimation of each movement state step by step, narrow down the estimation candidates, reduce the amount of calculation, and estimate the movement state of the user in a short time. In addition, each sensor can be mounted on a single mobile terminal device, thereby reducing the mounting cost of each sensor and the burden on the user.

  As described above, the present invention is characterized in that the movement state is automatically estimated without depending on the possession state of the mobile terminal device, without depending on the environment where the user is placed, and without requiring any operation by the user. In addition, the sensor data obtained in a relatively short period of time is used to reduce the load required for one estimation process and to detect a change in the movement state of the user in a short time.

  In general, the present invention generates a model for each specific reference movement state, and estimates a user's movement state using a model group which is a set of the models. The model group is preferably a Hidden Markov Model (hereinafter, HMM) group. Here, the HMM group is generated using a power spectrum list for learning in each reference state and a representative power spectrum group selected from the list.

  In the present invention, the learning power spectrum list is obtained by the CPU mounted on the mobile terminal device accessing the heap memory to acquire the learning acceleration data list from the acceleration sensor, and the acquired learning acceleration data list. Is obtained by FF (fast Fourier) transformation. The representative power spectrum list is obtained by the CPU mounted on the mobile terminal device automatically selecting from the obtained power spectrum list for learning using the GA.

  Then, each learning power spectrum in the obtained learning power spectrum list is pattern-matched with the selected representative power spectrum, and the ID of the nearest representative power spectrum is stored in the symbol string. Here, the nearest representative power spectrum is a representative power spectrum having the smallest error from the learning power spectrum. The storage process is performed for each learning power spectrum, and a symbol string in which IDs of a plurality of representative power spectra are stored is generated. HMM learning is performed using the ID of the representative power spectrum stored in the symbol string to create an HMM. The HMM is created for each movement state by the CPU, and an HMM group is created.

  Next, the user's movement state is estimated by acquiring an acceleration data list for estimation from an acceleration sensor possessed by the user, and performing the fast Fourier transform (hereinafter referred to as FF conversion) on the acquired acceleration data list for estimation. Power spectrum list is obtained, and each estimated power spectrum in the obtained estimation power spectrum list is pattern-matched with the representative power spectrum, and the ID of the nearest representative power spectrum is stored in the symbol string To do. The movement state of the user is estimated by the HMM using the ID of the representative power spectrum stored in the symbol string.

  The present invention is a method for estimating a movement state of a mobile terminal device including at least a central processing unit (CPU), a memory, and an environment information acquisition device, wherein the memory represents at least a predetermined movement state. Storing a group of first symbol sequences composed of two or more symbols (IDs), and causing the CPU to symbolize environmental information values collected by the environmental information acquisition apparatus for a predetermined period; and a time window. Repeating the step of shifting to a symbol for a certain period at least twice to generate a second symbol string comprising a plurality of symbolized environment information values generated repeatedly; and Determining the likelihood of the second symbol sequence for a group, and identifying the first symbol sequence with the maximum likelihood It provides a method for estimating a more mobile state.

The environment information acquisition device is an environment information acquisition device including at least one of an acceleration sensor, a microphone sensor, and a position measurement system receiver.
The present invention is also a method for estimating a movement state of a mobile terminal device including at least a central processing unit, a memory, and an acceleration sensor having a predetermined sampling rate, wherein the memory has a predetermined movement Storing a first spectrum list group (representative power spectrum list group) composed of at least two or more spectra representing a state, and causing the CPU to continuously collect acceleration values by the acceleration sensor A time window data generating step of generating a spectrum composed of data of the continuous acceleration values for a predetermined period as time window data in the collected continuous acceleration values, and shifting the time window by a predetermined period to the time The step of generating window data is repeated at least twice, and a plurality of time windows generated repeatedly Generating an estimation data list from the data, fast Fourier transforming the generated estimation data list to generate a second spectrum list (estimation power spectrum list), and Evaluating the likelihood of the second spectrum list for a group of first spectrum lists, and moving by identifying the first spectrum list with the maximum likelihood It features a method for estimating the state.

  The present invention is also a method for estimating a movement state of a mobile terminal device including at least a central processing unit, a memory, and a microphone sensor having a predetermined sampling rate, wherein the memory has a predetermined movement A first vector list group (representative vector list group) composed of at least two or more vectors representing a state is stored, and the CPU continuously generates a sound for a predetermined period every predetermined period by the microphone sensor. A time window data generating step of collecting data and generating a spectrum composed of the collected continuous sound data as time window data; and a step of generating the time window data by shifting the time window by the predetermined period A step of generating an estimation data list from a plurality of time window data repeatedly generated two or more times. Performing a fast Fourier transform on the generated estimation data list to generate a second spectrum list (estimation vector list); a constant frequency from the generated second spectrum list; Calculating an average value of the amplitude of the second spectrum every time, generating a second vector list, and obtaining a likelihood of the second vector list for the group of the first vector lists And a method of estimating the movement state by specifying the first vector list having the maximum likelihood.

  The present invention is also a method for estimating a moving state of a mobile terminal device including at least a central processing unit, a memory, and a position positioning system receiver, wherein the memory represents at least a predetermined moving state. A first speed data list group (representative speed data list group) composed of two or more speed data is stored, and the CPU causes the position positioning system receiver to collect position data every predetermined period. A step of accumulating in the memory, and a step of sequentially specifying the nth position data from the nth position data among the position data accumulated in the memory, wherein n is from 1 to N And an average speed data generation step for generating an average speed between two points using the nth position data and the (n + 1) th position data. And the step of generating the average speed data is repeated at least twice in the specific order, and a second speed data list (estimation speed data list) is generated from the plurality of average speed data generated repeatedly. And determining the likelihood of the second speed data list for the group of the first speed data list, wherein the first speed data with the maximum likelihood is It is characterized by a method of estimating a movement state by specifying a list.

  Further, in the present invention, the step of generating the second speed data list according to claim 5 includes repeating the step of generating the average speed data at least twice in the specific order, Generating an average speed data list from the average speed data, and using the n-th to n + m-th second average speed data from the generated average speed data list, m average speed data A step of calculating an average value of the steps, wherein m is an integer from 1 to M and M is a value smaller than N, and the step of calculating the average value is repeated at least twice in the specific order. And generating a second velocity data list from the plurality of average values generated repeatedly. 6. The method according to claim 5, further comprising: To.

  In the present invention, when the movement state estimated by the method according to claim 3 is the first or second predetermined movement state, the first step of specifying the estimated movement state as the movement state When the movement state estimated by the method according to claim 3 is not the first or second predetermined movement state, the movement state is further estimated by the method according to claim 4, and the estimated movement state 5 is a third predetermined movement state, the second step of identifying the estimated movement state as the movement state, and the movement state estimated by the method of claim 4 is a third predetermined movement state. In the case of not being in a moving state, the method further comprises a method of estimating a moving state by the method according to claim 5 and specifying the estimated moving state as a moving state.

  The present invention also relates to a method for estimating a movement state of a mobile terminal device including at least a central processing unit, a memory, and an acceleration sensor having a predetermined sampling rate, wherein the memory includes a predetermined movement state. Storing a first spectrum list group (representative power spectrum list group) composed of at least two or more spectra representing the above, and causing the CPU to continuously collect acceleration values by the acceleration sensor; A time window data generating step for generating, as time window data, a spectrum composed of data of the continuous acceleration values for a predetermined period in the collected continuous acceleration values; and shifting the time window by a predetermined period to the time window The step of generating data is repeated at least twice, and a plurality of time window data generated repeatedly are generated. Generating an estimation data list from the data, a fast Fourier transform of the generated estimation data list to generate a second spectrum list, and a group of the first spectrum list Selecting the first spectrum nearest to the second spectrum in the predetermined period by comparing each one spectrum with each second spectrum in the second spectrum list. And a step of selecting the first spectrum by repeating a predetermined number of times to identify a first spectrum list having the highest frequency, and a method of estimating a moving state (minimum error estimation method) And

  According to the present invention, in the first step according to claim 7, when the movement state estimated by the method according to claim 8 is the first predetermined movement state, the estimated movement state is moved. When the movement state specified by the method according to claim 8 is not the first movement state, the movement state is estimated by the method according to claim 3, and the estimated movement state is the second state. The method is characterized in that the estimated movement state is specified as the movement state in the case of the predetermined movement state.

  The present invention is also a system for estimating a movement state of a portable terminal device including at least a central processing unit, a memory, and an environment information acquisition device, wherein the memory represents at least a predetermined movement state. Storing a group of first symbol sequences composed of two or more symbols, wherein the CPU symbolizes environmental information values collected by the environmental information acquisition apparatus for a predetermined period, and shifts a time window for a predetermined period to The step of symbolizing is repeated at least twice, a second symbol sequence including a plurality of symbolized environment information values generated repeatedly is generated, and the second symbol for the group of the first symbol sequences is generated. The present invention relates to a system that obtains a likelihood of a sequence and estimates a moving state by specifying the first symbol sequence that maximizes the likelihood.

The environment information acquisition device is characterized by an environment information acquisition device including at least one of an acceleration sensor, a microphone sensor, and a position positioning system receiver.
The present invention is also a system for estimating a movement state of a mobile terminal device including at least a central processing unit, a memory, and an acceleration sensor having a predetermined sampling rate, wherein the memory includes a predetermined movement Storing a first spectrum list group composed of at least two or more spectra representing a state, and causing the CPU to continuously collect acceleration values by the acceleration sensor, and the collected consecutive acceleration values , A spectrum composed of the data of the continuous acceleration values for a predetermined period is generated as time window data, the time window is shifted by a certain period, and the generation of the time window data is repeated at least twice, and is repeatedly generated. The estimation data list is generated from a plurality of time window data, and the generated estimation data list is generated at high speed. The second spectrum list is generated, and the likelihood of the second spectrum list is evaluated with respect to the group of the first spectrum list. The first likelihood is maximized. The present invention relates to a system for estimating a movement state by specifying a spectrum list.

  Further, the present invention is a system for estimating a movement state of a mobile terminal device including at least a central processing unit, a memory, and a microphone sensor having a predetermined sampling rate, wherein the memory includes a predetermined movement A group of a first vector list composed of at least two vectors representing a state is stored, and the CPU continuously collects sound data for a predetermined period every predetermined period by the microphone sensor. A spectrum composed of the continuous sound data is generated as time window data, the time window is shifted by the predetermined period, and the generation of the time window data is repeated at least twice, and the plurality of time window data repeatedly generated is generated. From which the estimation data list is generated, the generated estimation data list is fast Fourier transformed, and the second spectrum is obtained. A second spectrum list, calculating an average value of the amplitude of the second spectrum for each fixed frequency from the generated second spectrum list, generating a second vector list, The present invention relates to a system for obtaining a likelihood of the second vector list with respect to a group of vector lists and estimating a movement state by specifying the first vector list having the maximum likelihood.

  The present invention is also a system for estimating a moving state of a mobile terminal device including at least a central processing unit, a memory, and a positioning system receiver, wherein the memory represents a predetermined moving state. Storing a first speed data list group composed of at least two or more speed data, wherein the CPU collects the position data at a predetermined period by the position measurement system receiver and stores the position data in the memory; The n-th position data among the position data stored in step S is used as the n-th position data, and the N-th position data is sequentially specified, where n is an integer from 1 to N, and the n-th position data Using the (n + 1) th position data, an average speed between two points is generated, and the generation of the average speed data is repeated at least twice in the specific order. Generating a second speed data list from the plurality of generated average speed data, and determining the likelihood of the second speed data list for the group of the first speed data list, The present invention relates to a system for estimating a moving state by specifying the maximum first speed data list.

  Further, in the present invention, the generation of the second speed data list according to claim 14 is performed by repeating the generation of the average speed data at least twice in the specific order, and the plurality of average speed data generated repeatedly. An average speed data list is generated from the average speed data, and an average value of m average speed data is calculated using the nth to n + mth average speed data from the generated average speed data list, where m is an integer from 1 to M, and M is a value smaller than N. The calculation of the average value is repeated at least twice in the specific order, and the second speed is calculated from the plurality of average values generated repeatedly. 15. A system according to claim 14 for generating a data list.

  Furthermore, the present invention specifies the estimated movement state as the movement state when the movement state estimated by the system according to claim 12 is the first or second predetermined movement state, and When the movement state estimated by the system according to claim 12 is not the first or second predetermined movement state, the movement state is further estimated by the system according to claim 13, and the estimated movement state is the third movement state. If the movement state estimated by the system according to claim 13 is not the third predetermined movement state, the estimated movement state is specified as the movement state when the movement state is a predetermined movement state, and further claimed. 14. A system that estimates a movement state by the system described in 14, and identifies the estimated movement state as a movement state.

  The present invention is also a system for estimating a movement state of a mobile terminal device including at least a central processing unit, a memory, and an acceleration sensor having a predetermined sampling rate, wherein the memory includes a predetermined movement Storing a first spectrum list group composed of at least two or more spectra representing a state, and causing the CPU to continuously collect acceleration values by the acceleration sensor, and the collected consecutive acceleration values , A spectrum composed of the data of the continuous acceleration values for a predetermined period is generated as time window data, the time window is shifted by a certain period, and the generation of the time window data is repeated at least twice, and is repeatedly generated. The estimation data list is generated from a plurality of time window data, and the generated estimation data list is generated at high speed. A second spectrum list is generated, and one spectrum in the first spectrum list group is compared with each second spectrum in the second spectrum list. The first spectrum nearest to the second spectrum in the predetermined period is selected, and the selection of the first spectrum is repeated a predetermined number of times to identify the first spectrum list with the highest frequency. Thus, the present invention relates to a system for estimating a moving state.

  In the present invention, when the estimated movement state according to claim 16 is the first or second predetermined movement state, the identification of the movement state is estimated by the system according to claim 17. When the movement state estimated is the first predetermined movement state, the estimated movement state is specified as the movement state, and the movement state estimated by the system according to claim 17 is not the first movement state. A system which is performed by estimating a movement state by the system according to claim 12, and specifying the estimated movement state as a movement state when the estimated movement state is a second predetermined movement state. It is characterized by.

  The present invention also relates to a method for estimating a movement state of a mobile terminal device including at least a central processing unit, a memory, and an acceleration sensor having a predetermined sampling rate, wherein the memory has a predetermined movement state. Storing a first spectrum list group (representative power spectrum list group) composed of at least two representative spectra, and causing the CPU to continuously collect acceleration values by the acceleration sensor; A time window data generating step for generating a spectrum composed of data of the continuous acceleration values for a predetermined period as time window data, and a step of generating time window data by shifting the time window for a certain period. Is repeated at least twice, and the estimation data is generated from the plurality of time window data generated repeatedly. Generating a data list; performing a fast Fourier transform on the generated estimation data list to generate a second spectrum list (estimation power spectrum list); and Evaluating a likelihood of the second spectrum list for a group, wherein the moving state is estimated by specifying the first spectrum list having the maximum likelihood And

  In the method, the likelihood evaluation step comprises comparing and adapting the second spectrum list to a group of the first spectrum list by the CPU to form the first spectrum list. Among the spectra to be selected, each of the spectra nearest to the spectrum constituting the second spectrum list is selected, and a set (symbol sequence) of each selected first spectrum is passed through a function consisting of HMM. The method of evaluating the likelihood of the HMM for each movement state.

  Further, in the above method, the first spectrum list includes a step in which the CPU continuously collects learning acceleration values by an acceleration sensor, and the collected continuous acceleration values for learning include a predetermined period of time. A learning time window data generation step for generating a learning spectrum consisting of continuous learning acceleration value data as learning time window data, and generating a learning time window data by shifting the time window by a certain period. Repeating the step of generating at least two times, generating a learning data list from the plurality of learning time window data repeatedly generated, and subjecting the generated learning data list to a fast Fourier transform, and learning power Generate a spectral list and a genetic algorithm for the training power spectrum list And calculating an evaluation value, wherein the list having the highest calculated evaluation value is determined as the first spectrum list. Characterized by the method described in.

  Further, in the step of evaluating the likelihood of the HMM according to claim 20, the CPU further performs the selection of the selected first spectrum set through a function comprising the HMM with a predetermined time width. A method is provided that includes evaluating the likelihood of the HMM for each movement state.

  According to the present invention, in addition to the steps recited in claim 19, the CPU further evaluates the likelihood of the HMM according to claim 22 for each movement state, and the likelihood. A second step of identifying a moving state in which the maximum is obtained, and when the HMM corresponding to the maximum likelihood obtained in the predetermined time width is not the predetermined moving state, the predetermined time width The method according to claim 20, wherein the moving state is estimated by sequentially specifying the moving state by repeating the first step and the second step with a longer predetermined time width. Estimation method).

The step of collecting acceleration values according to claim 19 further comprises the step of causing the CPU to further collect acceleration values per unit time continuously by the acceleration sensor, wherein the likelihood is maximized. The method of estimating a movement state according to claim 19, wherein the movement state estimated by specifying the first spectrum list is stored in a memory as estimation data per unit time. It is characterized by.

  In another aspect, the method according to claim 24, wherein a data structure (movement state schema) in which an estimated movement state and a movement state having a high frequency of being mistaken for the estimated movement state are grouped together is the same group. Each of which includes one or more movement state groups each including one or more movement states, and the movement state group is a method for estimating a movement state using a data structure including one basic movement state, the CPU Reading the estimated data per unit time from the memory, identifying the type of movement state group of the data structure using the estimated data per unit time, and the estimated data per unit time Obtaining a predetermined period and generating an estimated data log, and using the estimated data log, the movement by a majority estimator It has a step of estimating the movement of a 1 encompassed by state group, and a method for estimating the movement.

26. The method according to claim 25, wherein the unit time is a time required to acquire a predetermined number of estimated data required to identify the type of movement state group of the data structure. 26. The step of estimating the movement state by a majority voting method using the estimation data log according to claim 25, wherein the most frequent movement state included in the estimation data log is the type of the identified movement state group The highest-frequency movement state is identified as the estimated movement state, and the highest-frequency movement state included in the estimated data log does not belong to the type of the identified movement state group, 26. The movement state is estimated by specifying a basic movement state of the identified movement state group as an estimated movement state. Or a method according to 26.

  The present invention is a system for estimating a moving state of a mobile terminal device including at least a CPU, a memory, and an acceleration sensor having a predetermined sampling rate, wherein the memory represents the predetermined moving state. A group of a first spectrum list composed of at least two or more spectra is stored, and the CPU continuously collects acceleration values by the acceleration sensor, and the collected continuous acceleration values have a predetermined period. The step of generating a spectrum composed of the continuous acceleration value data as time window data, shifting the time window for a certain period and generating the time window data is repeated at least twice, and the plurality of times generated repeatedly An estimation data list is generated from the window data, and the generated estimation data list is Transforming to generate a second spectrum list and evaluating the likelihood of the second spectrum list for the group of the first spectrum list, wherein the first likelihood is maximized. It features a system that estimates movement states by specifying a spectrum list.

  In the system, the likelihood is evaluated by comparing and matching the second spectrum list with the first spectrum list by the CPU by using the first spectrum list. Among the constituting spectra, each of the nearest spectra is selected with respect to the spectra constituting the second spectrum list, and the selected first set of spectra is passed through the function consisting of HMM, The system according to claim 28, wherein the likelihood of the HMM is evaluated for each movement state.

  Further, in the system, the first spectrum list is obtained by causing the CPU to continuously collect learning acceleration values by the acceleration sensor, and in the collected continuous learning acceleration values, for a predetermined period of time. At least two steps of generating a learning spectrum composed of the continuous learning acceleration value data as learning time window data, and generating the learning time window data by shifting a time window by a predetermined period. It repeats more than once, a learning data list is generated from the plurality of learning time window data generated repeatedly, the generated learning data list is subjected to fast Fourier transform, and a learning power spectrum list is generated. , Applying a genetic algorithm to the learning power spectrum list, calculating an evaluation value, Wherein the system of claim 28 or 29, characterized in that the evaluation value is determined by the highest list to the first spectrum list.

  30. The likelihood evaluation of the HMM according to claim 29, wherein the CPU further converts the selected first spectrum set into the HMM through a function comprising the HMM over a predetermined time width. A system is provided that includes evaluating the likelihood of each movement state.

  Furthermore, in the present invention, the CPU according to claim 28 further includes a first step of evaluating the likelihood of the HMM according to claim 31 for each movement state, and a movement state in which the likelihood is maximum. And a predetermined time width longer than the predetermined time width when the HMM corresponding to the maximum likelihood obtained in the predetermined time width is not in a predetermined moving state. 30. The system according to claim 29, wherein the moving state is estimated by repeating the first step and the second step to sequentially specify the moving state.

  Further, in the acceleration value collection according to claim 28, the CPU further causes the acceleration sensor to continuously collect acceleration values per unit time by the acceleration sensor, and the first spectrum list that maximizes the likelihood. 29. The system for estimating a movement state according to claim 28, characterized in that the movement state estimated by specifying is stored in a memory as estimation data per unit time.

  In another aspect, the system according to claim 33 has one or more data structures in which the estimated movement state and a movement state that is frequently mistaken for the estimated movement state are in the same group. Including one or a plurality of movement state groups including a movement state, the movement state group being an estimation of a movement state using a data structure including one basic movement state, wherein the CPU estimates the unit time Reading data from the memory, identifying the type of movement state group of the data structure using the estimated data per unit time, obtaining the estimated data per unit time for a predetermined period, generating an estimated data log, Using the estimated data log to estimate one movement state included in the movement state group by a majority estimation method, The system for estimating the dynamic state characterized.

  35. The system according to claim 34, wherein the unit time is a time required to acquire a predetermined number of estimated data required to identify the type of movement state group of the data structure.

  Moreover, in another aspect, the estimation of the movement state of the majority estimator using the estimation data log according to claim 33, wherein the movement state with the highest frequency included in the estimation data log is the identified movement state. If it belongs to a group type, the highest-frequency moving state is specified as an estimated moving state, and the highest-frequency moving state included in the estimated data log does not belong to the identified moving state group type 36. The system according to claim 34 or 35, wherein a movement state is estimated by specifying a basic movement state of the identified movement state group as an estimated movement state.

  FIG. 1 is a diagram illustrating a system configuration of a user movement state estimation method according to an aspect of the present invention. The user movement state estimation method is mounted on, for example, a portable terminal device that fits in the palm of a person. The portable terminal device (A1) includes at least an acceleration sensor (A50), a memory, a central processing unit (CPU) (A10), and a battery (A60). including. The memory may include a heap memory (A20) and a flash memory (A30). Each component is driven by a battery. Here, the acceleration sensor acquires acceleration data and stores the acceleration data in a memory space in the sensor. For example, the CPU instructs to load the program from the flash memory and store it in the heap memory. For example, the flash memory stores a program such as a data acquisition function. For example, the heap memory temporarily stores a program stored in the flash memory.

  The present invention creates an HMM from a learning acceleration data list acquired in a specific reference state, and estimates the user's movement state using the HMM created for each movement state.

  FIG. 2 is a flowchart of learning model creation processing. Learning acceleration data in a specific reference state is acquired, and a learning power spectrum list is generated by FF conversion (B1). More specifically, the CPU reads in advance a learning model creation function (consisting of an FF conversion function, a representative power spectrum group selection function, a pattern matching function, a symbol string generation function, and an HMM learning processing function) from the flash memory, and a heap. Load into memory. The CPU calculates the learning power spectrum list by the FF conversion function in the learning model creation function, and stores the calculation result in the heap memory.

  Next, a representative power spectrum list is selected from the generated learning power spectrum list (B2). More specifically, the CPU selects the representative power spectrum list by the representative power spectrum list selection function in the learning model creation function, and writes the selection result in the flash memory. Steps B1 and B2 are repeated for each movement state to generate a representative power spectrum group (B3).

  Next, each representative power spectrum in the generated representative power spectrum group is subjected to pattern matching with the learning power spectrum by the pattern matching function in the learning model creation function (B4).

  Then, the ID of the nearest representative power spectrum is accumulated as a learning symbol in the heap memory, and the CPU generates a learning symbol string by the symbol string generation function and stores it in the heap memory (B5).

With the HMM learning function, the CPU reads from the heap memory into the learning symbol string, performs HMM learning (B6), creates an HMM learning result (probability table), writes it into the flash memory, and creates an HMM (B7). The above steps B1 to B6 are performed for each movement state of the learning target to generate an HMM group.
Estimation of User Movement State FIG. 3 is a flowchart of a user movement state estimation process. Acceleration data is acquired from an acceleration sensor possessed by the user, and an estimation power spectrum list is generated by FF conversion (C1). More specifically, the CPU reads in advance a movement state estimation function (FF conversion function, pattern matching function, symbol string generation function, HMM estimation function) from the flash memory and loads it onto the heap memory. Then, by the FF conversion function, the CPU reads the estimation acceleration data from the heap memory, calculates the estimation power spectrum list, and stores the calculation result in the heap memory.

  Then, each representative power spectrum of the representative power spectrum group obtained in step B3 and the estimation power spectrum are pattern-matched by the pattern matching function (C2).

  Next, the nearest representative power spectrum ID with respect to the estimation power spectrum is selected (step C3) and stored in the heap memory. For example, if the shift width is set to 0.5 seconds by calculation by the CPU, one representative power spectrum ID is accumulated in 0.5 seconds. Then, the CPU performs the above accumulation process for each estimation power spectrum in the estimation power spectrum list. Here, the symbolization of acceleration data refers to a series of processes from steps C1 to C3.

  Next, the representative power spectrum ID selected by the CPU is read from the heap memory by the symbol string generation function, and sequentially stored in the estimation symbol string to generate an estimation symbol string (C4).

  Next, HMM estimation is performed using a function made up of HMMs using the generated estimation symbol string. To estimate the user's movement state, the representative power spectrum ID stored in the estimation symbol string is input to the HMM in each movement state, and the likelihood for each HMM is obtained. The movement state is estimated by identifying the movement state type corresponding to the HMM having the highest likelihood among the obtained likelihoods (HMM estimation). Specifically, using the HMM estimation function, the CPU sequentially reads the symbols (ID of the representative power spectrum) in the estimation symbol string from the heap memory in time series, and calculates the evaluation value (likelihood) for each HMM. (C5) The result is written into the heap memory. The movement state corresponding to the HMM having the highest likelihood is specified, and an estimation result is obtained (C6).

  In conventional pattern matching, one estimation power spectrum that does not include a time-series change is compared with a representative power spectrum, the nearest representative power spectrum is selected, and the user's movement state is estimated. Therefore, when a sudden change in the user's state occurs, if the selected power spectrum contains a lot of noise, an incorrect movement state is specified. Therefore, it is difficult to accurately estimate the movement state by the conventional pattern matching.

  In the present invention, the symbol string in which the ID of the representative power spectrum is stored includes time-series data. Therefore, even if noise is included in one power spectrum among a plurality of time-series power spectra, the moving state is estimated in consideration of the power spectrum conditions before and after the noise. Therefore, the present invention makes it possible to estimate a user's state more accurately without being influenced by a sudden external environment.

As described above, by estimating the user's movement state using the time series data of the power spectrum, the performance of the estimation performance evaluation with respect to the time required for the estimation is improved as compared with the conventional method.
Details of each step will be described individually below.
Generation of Learning Power Spectrum List FIG. 4 is a flowchart of the learning power spectrum list generation process. The power spectrum list for learning is data used for learning the HMM, and is generated by acquiring reference state data in a specific movement state for a certain period.

  First, learning acceleration data (x, y, z) in a reference state is acquired from the acceleration sensor (D1). Here, x, y, and z are accelerations on three axes, but they may be accelerations on one or two axes. More specifically, the CPU pre-loads a learning power spectrum list generation function (acceleration value calculation, time interval determination, acceleration value time window data calculation function, FF conversion function), and a device driver of the acceleration sensor. Read from memory and load onto heap memory. Then, the CPU activates the acceleration sensor by the device driver and starts measurement. The device driver reads the acceleration sensor measurement values (learning acceleration data) and accumulates them in the heap memory.

Next, the CPU reads the learning acceleration data from the heap memory by the acceleration value calculation function, and generates the acceleration value α = (x ² + y ² + z ² ) ^1/2 from the learning acceleration data by calculation by the CPU. (D2), storing in the heap memory.

  Then, the CPU generates the acceleration value α over a predetermined period (for example, 2 seconds) by using the acceleration value time window data calculation function, generates time window data for learning the acceleration (D3), and sends the result to the heap memory. accumulate. Here, for example, when the sampling rate is 128 Hz, the time window data for learning the acceleration consists of 256 acceleration values α in 2 seconds. If the acceleration sensor has a sampling rate of 256 Hz, the predetermined period may be 1 second.

  Next, the CPU reads out the previous learning power spectrum generation time from the heap memory by the time interval determination function, measures the elapsed time, determines whether the threshold is exceeded, and if it does not exceed the threshold, it is for learning the acceleration. The step of generating the time window data is repeated (D4). If it exceeds the threshold, the process moves to step D5.

  Next, the CPU reads a bundle of acceleration time window data (here, the obtained bundle of time window data is referred to as a time-series data list) from the heap memory by the FF conversion function, and converts each time window data into the time window data. The power spectrum for learning is calculated by FF conversion (D5), and is stored in the heap memory. Of the learning power spectrum obtained by FF conversion of the time window data, a spectrum in a predetermined frequency band, for example, a spectrum of 1 to 10 Hz can be used. The interval at which the step of acquiring acceleration time window data is repeated may be the predetermined period, or may be longer or shorter. In this embodiment, for example, the shift width of the time window for 2 seconds is set to 0.5 seconds.

Thus, the list of learning power spectrum stored as training power spectrum list P _a (e.g. a = 1200).
Selection of Representative Power Spectrum Group Selection of the representative power spectrum list from the learning power spectrum list is performed using GA in the preferred embodiment. The representative power spectrum list is selected by generating a population from the learning power spectrum list by the CPU, calculating an evaluation value for each individual of the generated population, and selecting each individual based on the evaluation value. The evaluation is performed by performing generation change a predetermined number of times and selecting an individual having a high evaluation value. That is, calculate the evaluation value of each individual, sort the population in the order of evaluation value based on the evaluation value, repeat the operations such as crossover, inheritance, mutation etc. of the population a predetermined number of times, The individual having the highest evaluation value in the selected individual group is selected as the optimum solution. A process including a series of these processes is shown in FIG. In performing the series of processes, the CPU loads a representative power spectrum selection program stored in advance in the flash memory onto the heap memory. Then, the CPU registers instructions of the representative power spectrum selection program in the CPU register, performs program processing, and accumulates the obtained values in the heap memory.

FIG. 5 is a flowchart of representative power spectrum list selection processing. In FIG. 5, first, an initial representative power spectrum list (not a group) is R (R ₁ , R ₂ ,... R _n ) (for example, n = 32). Here, R ₁ ... R _n are representative power spectra selected in the selection process. In the representative power spectrum list R, a NULL value is set as an initial value. The CPU reads the representative power spectrum selection function (GA function) from the flash memory in advance and loads it onto the heap memory. Then, the CPU randomly extracts n power spectra (genes) from the learning power spectrum list P _a (here, a = 1200) by the GA function, and repeats this extraction process m times to obtain n A predetermined number (for example, m = 30) of initial individuals having one gene is generated, and an individual group l _m is generated (E1), and stored in the heap memory.

Next, the CPU reads two individuals from the individual group l _{m at} random from the heap memory using the GA function, and crosses them to generate one child individual. This process is repeated m times to generate a child population C _m (E2) and accumulate it in the heap memory. Here, crossing means that one individual is generated from two individuals, and a parent individual A (R _a1 , R _a2 , ... R _an ) and a parent individual B (R _b1 , R _b2 , ... R _bn ) The gene is selected at random, and a child individual C, for example, (R _a1 , R _b2 ,... R _bn−1 , R _an ) is generated.

Next, the CPU calculates an evaluation value of each parent individual and each child individual using the GA function (E3), and stores the calculation result in the heap memory. CPU, for each power spectrum of learning for power spectrum list P _a (gene), the same to select the representative power spectrum of the nearest neighbor (gene) from the representative power spectrum-list R. Here, the representative power spectrum, because it is selected for each learning power spectrum constituting the P _a, the power spectrum list consisting of a number of power spectrum is selected. Each the power spectrum of the selected nearest neighbor of the power spectrum list, obtained as an evaluation value the reciprocal of the sum of the error between each of the power spectrum of the training power spectrum list P _a. Here, for example, an evaluation value is calculated for 60 parent individuals and 30 child individuals in total. The nearest power spectrum refers to a spectrum having the smallest distance (error) from the representative power spectrum, and the distance is a sum of absolute values of amplitude value errors for each frequency.

  Next, the CPU reads the evaluation value group of the parent individual and the child individual from the heap memory by using the GA function, searches for the best evaluation value, and the best evaluation value of the next generation (child generation) individual group is the previous generation (parent generation). (E4), and the result is stored in the heap memory. In this embodiment, it is determined whether or not one best evaluation value has evolved compared to the best evaluation value of the previous generation. For example, whether or not the entire population has evolved from the previous generation. Can also be determined. If the calculated evaluation value is larger than the evaluation value of the previous generation, it means that the next generation has evolved from the previous generation. Here, in the first time, 0 is set as the evaluation value.

  And when the next generation population has evolved from the previous generation, based on the evaluation value of each individual, select a predetermined number of elite individuals and surviving individuals, and classify them as elite individuals, normal individuals, dead individuals, Change generations. In the generational change, the elite individual is inherited as it is by the next generation, and the same next generation individual is generated. On the other hand, dead individuals are not inherited by the next generation, and new mutant individuals are selected instead of dead individuals.

  More specifically, when it is determined in step E4 that the next generation population has evolved (the evaluation value is large), the CPU heaps the evaluation value calculated for each individual (parent individual, child individual). Read from the memory, select a plurality of upper (for example, 9) individuals as elite individuals (E5), and store the result in the heap memory. The selected elite individuals are inherited by the next generation.

  Then, the CPU generates a roulette that is proportionally distributed to the evaluation values, and selects a certain percentage (for example, about 80%) of the remaining individual group from the roulette strategy, from which a specified number of individuals (here, 18) are selected (E6) and stored in the heap memory. The individual selected here is inherited by the next generation.

Next, the remaining next generation individuals (three in this case) disappear as dead individuals, and instead the CPU randomly generates mutant individuals (E7) and accumulates the results in the heap memory. The generated mutant individuals are incorporated into the next generation. Here, mutations are generated in the same procedure as the initial population generated (step E1, E2) by CPU, i.e. are chosen randomly from the power spectrum list P _a for learning.

  Then, the CPU reads the individual group selected / generated in steps E5 to E7 from the heap memory, inherits it as the next generation individual group (here, 30) (E8), and stores it in the heap memory.

  On the other hand, as a result of the determination in step E4, when the next generation evaluation value is the same as or smaller than the previous generation evaluation value, the process proceeds to step E9. This means that the next generation population is in an evolutionary state. The CPU reads the number of non-evolutionary states from the heap memory, and determines whether the non-evolutionary state continues for a predetermined number of times (E9). If it has not continued for a predetermined number of times, 1 is added to the number of non-evolution states, and the result is overwritten and accumulated in the heap memory. Then, steps E5 to E8 are repeated a predetermined number of times, for example, 50 times (for 50 generations) in this embodiment.

  When the number of non-evolution states reaches the predetermined number, the CPU reads the individual with the highest evaluation value from the final individual group from the heap memory, and selects the gene information as the representative power spectrum list R (E10). ). The result is written to the flash memory.

As described above, the representative power spectrum list selection process is performed for all the movement states, and the representative power spectrum group is selected. For example, if a representative power spectrum list R is selected for each of seven movement states, 32 representative power spectra are selected for each list, and a representative power spectrum group including a total of 224 representative power spectra is selected. The
In the representative power spectrum group selected for pattern matching in the learning power spectrum list , an ID is individually assigned to each representative power spectrum (224 in this case). Then, for each learning power spectrum of the time series stored in the learning power spectrum list P _a, it performs each representative power spectrum and pattern matching of the representative power spectrum groups, ID of the representative power spectrum nearest Are sequentially stored in the heap memory, and a learning symbol string is generated. Here, the symbol string is a data string in which IDs of representative power spectra in a predetermined period are stored in time series.

As described above, the process of generating the learning symbol string is performed on the movement states of all reference states.
HMM Learning The present invention enables the user's movement state to be accurately recognized even when the user's movement state greatly fluctuates by using the HMM for estimating the user's movement state. An HMM is generally a set of states connected to each other by transitions. The HMM is composed of several internal states and generates an observable output for each step of each transition from the initial state to the final state. It is a state transition. Details of the HMM are described in, for example, Kenji Kita, 1999, “Probabilistic Language Model”. Details of the Baum-Welch algorithm are described in LE Baum et al., “A maximization technique occurring in the statistical analysis of probabilistic function of markov chains”. AIC (Akaike Information Criterion) is described in Akaike's book, “New Approach to Statistical Testing”.
Creation of HMM FIG. 6 shows an HMM learning process flow. In one embodiment, the HMM is composed of an initial state probability Xi, a state transition probability Aij, and a signal output probability Bi (k). In the HMM, several states i are prepared, the probability of starting from each state is the initial state probability Xi, the transition probability from a certain state i to each state j is the state transition probability Aij, and the symbol k in each state i A probability of appearance (here, IDs of 224 representative spectra) is defined as a signal output probability Bi (k). The HMM can output a symbol string probabilistically based on the values of the initial state probability Xi, the state transition probability Aij, and the symbol output probability Bi (k). To estimate the user's movement state, an HMM is prepared for each movement state in advance, and the HMM probability table, Xi, Aij, bi (k) is learned so that each HMM outputs the learning symbol string with the highest probability. Is done.

  More specifically, first, the initial values of the number of initial states constituting the HMM, initial state probabilities Xi, state transition probabilities Aij, and signal output probabilities Bi (k) are appropriately determined (F1). Here, for example, the number of initial states is 3, and the initial values of the probabilities Xi, Aij, and Bi (k) are set to equal values.

Next, the initial state probability Xi, the state transition probability Aij, and the signal output probability Bi (k) are calculated by calculation by the CPU using the learning symbol string generated for HMM learning (F2).
Next, the likelihood calculation of the HMM is preferably performed by a forward algorithm using the values Xi, Aij, Bi (k) obtained in step F2 (F3).

Next, it is determined whether the obtained likelihood value has an improvement of a certain level or more compared with the previous likelihood value (F4). When the obtained likelihood value has a certain improvement, the process returns to step F2.
If the likelihood does not increase beyond a certain level, an AIC is calculated (F5). Here, the AIC is an index for evaluating the goodness of the statistical model, and is used to balance the complexity of the model and the degree of fitness with the data.

  Next, it is determined whether the value of AIC is smaller than a previously calculated value by a certain amount or more (E6). When the value is smaller than the previously calculated value by a certain amount or more, the number of states constituting the HMM is increased by 1 (F7), and the process returns to Step E1.

When the value of AIC is not smaller than the previously calculated value by a certain amount or more, the initial state probability Xi, state transition probability Aij, and signal output probability Bi (k) are stored in the heap memory (F8), and the HMM learning function This completes the HMM learning process. As a result, the configuration of the HMM such as the number of states and the state transition probability that is most likely to output a learning symbol string in a specific movement state is determined. The CPU performs the above steps F1 to F8 for each learning target movement state, and individually configures the HMM. Therefore, in this embodiment, an HMM is generated for each stop state, walking state, running state, bicycle state, automobile state, bus state, and train state, which are the movement states of the learning target, and an HMM group is generated.
Generation of the estimation power spectrum list The estimation power spectrum list used for estimating the movement state of a user in a certain movement state is performed by the same process as the selection of the learning power spectrum list. That is, acceleration data for estimation is acquired from an acceleration sensor possessed by a user in a certain moving state over a predetermined period (here, 2 seconds), and time window data of acceleration is generated from the obtained acceleration data for estimation.

Then, the generation of the estimation acceleration time window data is repeated a predetermined number of times to obtain a time series data list (estimation data list) of the estimation acceleration. Note that the generation of the estimation acceleration time window data is repeated a predetermined number of times until the number of data necessary for estimating the user's movement state is obtained. Then, the time window data of the plurality of estimation accelerations are each FF-converted by calculation by the CPU, and an estimation power spectrum list including a plurality of estimation power spectra is calculated.
Multi-stage estimation method using variable-length symbols In a further aspect of the present invention, the user's movement state is estimated according to information obtained in a predetermined period by calculating a symbol length (time length) required for estimating the movement state. The present invention relates to estimating a user's movement state in a shorter time by changing in multiple stages (multistage estimation method). FIG. 7 is a processing flow diagram according to a multistage estimation method using variable-length symbol sequences. By applying this method, it is possible to further reduce the average estimation time required for the above estimation method and estimate the user's movement state.

  Specifically, a moving state estimation method using a multistage estimation method will be described with reference to the processing flow of FIG. First, the CPU reads in advance an HMM estimation function used in the multistage estimation method from the flash memory and loads it onto the heap memory. Then, the previous estimation acceleration data for the first time width (for example, 3 seconds) required to identify the user's movement state is acquired from the acceleration sensor of the user in a certain movement state, and the estimation is performed. An estimation symbol string is generated based on the power spectrum list (G1).

  Next, the CPU reads an estimation symbol string from the heap memory using the HMM estimation function (FIG. 3, step C5), and estimates the movement state (G2). Then, it is determined whether or not the estimation result is the running state (G3). If the estimation result is the running state, the estimation result is accumulated in the heap memory (G4), and the estimation process is terminated. The length of the estimation symbol string required for estimating the user's movement state differs for each user's movement state. Therefore, the user's movement state (for example, running state) that can be estimated in the shortest time (here, 3 seconds) is first estimated.

  On the other hand, if it is determined in step G3 that the estimation result is not the running state, a second time width (for example, 30) longer than the first time width required for further identifying the user's moving state from the acceleration sensor of the user. Acceleration data for the past over a second) is acquired, an estimation power spectrum list is generated, and an estimation symbol sequence is generated (G5). Next, the CPU reads an estimation symbol string from the heap memory using the HMM estimation function, and estimates the movement state (G6). At this time, the HMM in the moving state (running state in this case) that has already been estimated in the processing of steps G1 to G3 is excluded from the estimation target. As described above, the multi-stage estimation method estimates the user's movement state in multiple stages by using the fact that the symbol string length required for identification of the movement state differs depending on the type of movement state. Further, the multistage estimation method is characterized by reducing the load on the CPU for the estimation process and increasing the estimation process speed by removing the already estimated HMM in the moving state from the estimation target.

  Next, whether or not the result estimated by the HMM estimation function using the extended symbol string is a movement state (for example, a walking state or a bicycle state) that can be estimated in the second time width as in step G3. Is determined (G7). When the estimation result is the walking state or the bicycle state, the estimation result is accumulated in the heap memory (G8), and the estimation process is terminated. When the estimation result is not the walking state or the bicycle state, the past sensor over the third time width (for example, 180 seconds) longer than the second time width required for identifying the user's moving state is further obtained from the acceleration sensor of the user. The estimation acceleration data is acquired, the estimation power spectrum list is generated, and the estimation symbol string is generated (G9). In the present embodiment, the acceleration data for estimation of the third time width is necessary for identifying, for example, a stop state, a car state, a train state, and a bus state. Here, the third time width refers to a time width required for estimating all moving states.

  Next, the CPU reads the estimation symbol string from the heap memory by the HMM estimation function, and performs HMM estimation (G10). At this time, the HMM in a state that can be estimated by the processing in steps G1 to G6 (here, the running state, the bicycle state, and the walking state) is excluded from the estimation target. When the movement state (stop state, car state, bus state, train state) is estimated, the estimation result is stored in the heap memory (G11), and the estimation process is terminated.

As described above, in the multistage estimation method, the movement state is estimated by calculation by the CPU using the estimation acceleration data with the longest time width in which all the movement states can be estimated. Therefore, when it is possible to estimate with a time width shorter than the longest time width, the time required for the average moving state estimation is shortened by estimating such a moving state first (multi-stage estimation). For example, in the case of a traveling state that is a moving state that can be estimated in the shortest time, it is estimated in only 3 seconds, and the process ends without estimating whether it is another moving state. Can be reduced. In the multistage estimation method, the user's movement state is estimated using past estimation acceleration data, and the movement state estimation time can be further shortened according to the processing capability of the CPU. As described above, in this embodiment, the time required for estimating the moving state using the present multistage estimation method is about half that in the case where the present method is not used.
Majority Estimation Method Using Moving State Schema A further aspect of the present invention relates to a majority estimation method that further reduces the time required to estimate the movement state without reducing the estimation accuracy of the multistage estimation method described above. In the majority decision estimation method, by using the movement state schema, it is possible to shorten the processing time required for the estimation of the movement state and reduce the processing load required for one estimation.

  FIG. 8 shows a movement state schema in this embodiment. Here, when the movement state estimated by the HMM estimation is different from the actual movement state of the user, the movement state schema is obtained from the ease of difference (easy to make a mistake). More specifically, as a result of estimating the moving state using the estimation symbol string in a predetermined period, the estimated moving state of the user may be different from the actual moving state of the user. Each movement state in the movement state group is obtained by paying attention to an estimation error between such an actual state and an estimated state. That is, as a result of estimating the movement state using a function composed of HMMs, the estimated movement state and the movement state that is frequently mistaken for the estimated movement state are grouped into the same group. The movement state schema is composed of the grouped one or more movement state groups. Each movement state group includes one or more movement states.

  As described above, the moving state is divided into, for example, a stopped state, a walking state, a traveling state, a bicycle state, an automobile state, a bus state, and a train state. Each moving state group has one basic state, and a set of one or more basic states is a basic state group. On the other hand, each movement state group may have one or a plurality of derivation states, or may not have a derivation state. Here, the derived state is a state that always transits to the basic state and then transitions to it. Movement states within the movement state group other than the identified basic state are derived states. When only one movement state is included in the movement state group, the movement state becomes a basic state, and no derivative state exists.

  In this embodiment, for example, the basic state is a walking state, a stopped state, and a running state. In addition, a derived state derived from the walking state is a bicycle state, these states are a walking group, a derived state derived from the stopped state is a bus state, a train state, and an automobile state, and these states are a stop group. The traveling state is a traveling group, and the traveling state has no derivative state. In addition, the walking state, the stopped state, and the running state, which are basic states, belong to the basic state group.

  As described above, the movement state schema expressing the movement state transition is generated. By using the created movement state schema, the movement state of the user can be identified in a short period of time as long as it is identification between groups.

  The majority estimator is composed of a majority estimator using an estimated data log (FIG. 9) and an estimated data log acquisition process (FIG. 10). Is done. In the majority voting estimation method, HMM learning (FIG. 6) is performed in advance through a function composed of an HMM in order to create an HMM necessary for processing according to the present method. Further, the estimated data is an ID indicating the movement state corresponding to the HMM by evaluating the likelihood of the HMM through a function composed of the HMM by the calculation by the CPU, specifying the HMM having the maximum likelihood. It is. The movement state ID is assigned to each movement state.

  Hereinafter, the majority estimation processing flow using the estimation data log will be described with reference to FIG. (First, the estimated data for the previous unit time is acquired from the estimated data log accumulated in the heap memory (H1). Here, the unit time is, for example, a predetermined number required to identify the type of the movement state group The estimated data log is a log of estimated data obtained by acquiring estimated data over a predetermined period.In this embodiment, the unit time The estimation result of the moving state is accumulated as an estimation data log every 3 seconds, and the immediately preceding estimation data is the estimation data for the last 3 seconds in the present embodiment. The width is set to 0.5 seconds, and an estimated data log consisting of 6 estimated data is obtained in 3 seconds.

  Next, it is determined to which moving state group the immediately preceding estimated data belongs. Here, it is first determined whether or not it is a moving state group (in this case, a traveling group) that can estimate the moving state in the shortest estimation time among the moving state groups (H2). When the immediately preceding estimated data belongs to the traveling group, the traveling state estimation result is set as the traveling state (H3), and the estimation result is stored in the heap memory (H4). In the present embodiment, since only the traveling state is included in the traveling group, the estimation result is accumulated as the traveling state.

  Next, if the immediately preceding estimated data does not belong to the traveling group, whether or not the immediately preceding estimated data belongs to a group (here, a walking group) capable of estimating the moving state in the next shortest estimated time after the traveling state Is determined (H5). Here, whether or not the immediately preceding estimated data belongs to the walking group is determined by determining whether the estimated data is a walking state or a bicycle state in the walking group. If it is determined to belong to the walking group, estimated data for the past first predetermined period (for example, for the past 30 seconds, 10 times) is acquired, and the majority of the estimation results is taken (H6). Then, as a result of the majority decision, it is determined whether or not the most frequently moved state is a walking state or a bicycle state (H7), and if it is either a walking state or a bicycle state, the result (majority result) is estimated. The result is (H8). When it is neither the walking state nor the bicycle state, the walking state is set as the estimation result (H9). Then, the estimation result is accumulated on the heap memory (H4).

  Even if it is determined in step H5 that the immediately preceding estimated data is a walking group, as a result of taking a majority decision from the estimated data log for the past 30 seconds in step H6, the movement state of the group different from the walking group (for example, bus State) may be obtained as the most frequent state in the first predetermined period. In this case, even if movement states of different groups are obtained, an estimation result is obtained assuming that the movement state in the first predetermined period is the walking state that is the basic state of the walking group (H9). This is because the bus state does not belong to the walking group according to the movement state schema (FIG. 8). Therefore, by using the movement state schema, when the movement state estimated using the immediately preceding estimated data and the movement state estimated in the first predetermined period do not belong to the same movement state group, the first By ignoring the estimation result of the movement state estimated during the predetermined period, a more accurate estimation result can be obtained in a short time.

  On the other hand, if it is determined in step H4 that it does not belong to the walking group, estimated data for a second predetermined period (for example, the past 180 seconds, 30 times) is acquired, and the majority of the estimation results is taken (H10). Then, as a result of the majority decision, it is determined whether or not the most frequent moving state is a stopped state, an automobile state, or a bus state (H11), and if it is any one of a stopped state, an automobile state, or a bus state, The result (majority decision result) is set as an estimation result (H12). When it is neither the stop state, the car state, nor the bus state, the stop state is set as the estimation result (H13). Then, the estimation result is accumulated on the heap memory (H4).

  Next, it is determined whether or not there is a process end instruction (H14). If there is no process end instruction, the process waits until the next fixed time elapses (H15), and then returns to step H1. If there is a process termination instruction, the process is terminated. The above is the movement state estimation flow by the majority decision method using the movement state schema.

  Next, an estimated data log acquisition process flow will be described with reference to FIG. First, according to the process of FIG. 3, the user's movement state in a certain movement state per unit time is estimated using the estimation acceleration data obtained from the acceleration sensor possessed by the user through a function consisting of HMM (I1 ). The movement state corresponding to the HMM having the maximum likelihood is specified by the HMM estimation. The movement state ID indicating the movement state is used as estimated data and stored in the heap memory (I2). Then, acceleration data for the next unit time is acquired (I3). Here, it is determined whether or not there is a process end instruction. If there is no process end instruction, the process returns to step I1, and if there is, the movement state estimation process ends. Steps I1 to I3 are repeated until a process end command is issued, and a log of estimated movement state data (estimated data log) is obtained by calculation by the CPU.

As described above, by using this majority decision estimation method, the time required for estimating the movement state is further shortened compared to the case of using the multistage estimation method. The majority estimation method first identifies a moving state group and then estimates which moving state in the moving state group belongs. In the majority estimation method, by using the movement state schema, when the movement state estimated using the previous estimation data is different from the movement state estimated in the first predetermined period, the first predetermined period is used. Ignore the estimation result of the estimated movement state. As a result, a more accurate estimation result can be obtained. In addition, the amount of calculation can be greatly reduced, and the processing time required for estimation can be shortened.
A moving state estimation method using a plurality of types of external environment information Further embodiments include a minimum error estimation method, and an HMM estimation method using a combination of three types of sensors: an acceleration sensor, a microphone sensor, and a positioning system receiver. About. FIG. 11 is a diagram illustrating a system configuration of a user movement state estimation method according to an aspect of the present invention. The mobile terminal device (A1) includes at least an acceleration sensor (A50), a memory, a central processing unit (CPU) (A10), and a battery (A60), and further includes a microphone sensor (A70) and a positioning system receiver (A80). be able to. Each of these components is preferably mounted on one mobile terminal device. By mounting on one mobile terminal device, it is possible to reduce the burden of the user holding the mobile terminal device.

  In this embodiment, a method suitable for estimation of each movement state is applied step by step to narrow down the estimated movement state candidates, thereby improving estimation accuracy and reducing the amount of calculation. More specifically, for the movement state that can be estimated without performing the HMM estimation, the movement state of the user is estimated by a minimum error estimation method with a small amount of calculation. This makes it possible to use a minimum error estimation method that is simpler and requires less calculation in a moving state in which the type of moving state can be estimated, for example, in a running state or a walking state, without performing HMM estimation via a function composed of HMMs. Thus, the moving state can be estimated. For a moving state that is difficult to estimate with high accuracy by the minimum error estimation method, HMM estimation is performed by using a combination of three types of sensors: an acceleration sensor, a microphone sensor, and a positioning system receiver that can be mounted on a portable terminal device. To estimate the user's movement state. By treating the temporal change of sensor data as a state transition model by HMM estimation, it is possible to avoid a decrease in estimation accuracy due to the existence of a time zone similar to other moving states. Also, by using these sensors in combination, if the form of the power spectrum obtained from the acceleration sensor that the user has in a certain time zone is similar to the form of the power spectrum in other moving states, it can be obtained from the microphone sensor. The movement state of the user can be estimated using the sound data obtained and the position data obtained from the position positioning system, and the estimation accuracy can be improved.

  FIG. 12 is a process flow estimation process flow using a plurality of types of external environment information. The plural types of external environment information are information representing the environment where the user is placed, and are, for example, speed change information, sound information, and position information obtained through the mobile terminal device possessed by the user. First, measurement is periodically performed by an acceleration sensor, a microphone sensor, and a positioning system receiver (step J1). Next, the moving state is estimated using the minimum error estimation method based on the power spectrum calculated using the acceleration value acquired from the acceleration sensor (step J2). If the most frequent moving state is the first predetermined moving state (here, walking state, running state), the moving state is acquired as an estimation result (step J3). Note that the minimum error estimation method is a method for estimating the first predetermined movement state with a small amount of calculation compared to the HMM estimation method. Without using this method, the minimum error estimation method is performed by HMM estimation using an acceleration sensor described in steps J4 and J5. It is also possible to estimate the first predetermined movement state.

  Next, if the estimated result is not the first predetermined movement state, the likelihood of the HMM via the function consisting of the HMM using the power spectrum calculated using the acceleration value acquired from the acceleration sensor. Is evaluated (step J4). At this time, the first predetermined movement state already estimated in step J2 is excluded from the estimation target. Thus, by narrowing down the movement state estimated stepwise, it is possible to improve estimation accuracy and shorten the time required for estimation. If the moving state having the maximum likelihood is the second predetermined moving state (here, the bicycle state or the stopped state), the moving state is acquired as an estimation result (step J5).

  Next, when the estimated result is not the second predetermined movement state, the likelihood of the HMM is obtained through a function composed of the HMM using the power spectrum calculated using the sound data acquired from the microphone sensor. Evaluate (step J6). At this time, the states that can be estimated in steps J2 and J4 are already excluded from the estimation targets. When the movement state having the maximum likelihood is the third predetermined movement state (in this case, the automobile state), the movement state is acquired as an estimation result (step J7).

Finally, if the estimated result is not the third predetermined movement state, the HMM's through a function consisting of the HMM using the average velocity data calculated using the position data obtained from the positioning system receiver. The likelihood is evaluated (step J8). At this time, the states that can be estimated in steps J2, J4, and J6 are already excluded from the estimation targets. And the movement state which has the maximum likelihood is acquired as an estimation result (step J9).
Minimum error estimation method In this embodiment, a predetermined number of IDs of representative power spectra having the smallest error from the estimation power spectrum are acquired from the representative power spectrum list group, and the majority of the acquired IDs is taken to obtain the highest frequency. A movement state is acquired as an estimation result, and a user's movement state is estimated. Unlike the moving state estimation method using the HMM, it is not necessary to calculate the likelihood for each moving state using the estimation symbol sequence generated from the estimation power spectrum, so that the calculation amount can be reduced. it can.

  Hereinafter, with reference to FIG. 13, a description will be given of an estimation processing flow of a user's movement state using the minimum error estimation method. First, the estimation power spectrum list is selected by the same method as the learning power spectrum list selection method for acceleration described above (FIG. 4). That is, acceleration data for estimation is acquired from an acceleration sensor of the user over a predetermined period (here, 2 seconds), and time window data of the acceleration for estimation is generated. The time window data of the generated acceleration for estimation is FF transformed to generate an estimation power spectrum. The generation of the acceleration value is repeated with a certain shift width (for example, 0.5 seconds) to generate a power spectrum list for estimation (step K1).

  Next, pattern matching is performed between each representative power spectrum in the representative power spectrum group obtained in step B3 and the estimation power spectrum (step K2). For example, pattern matching is performed by calculating the sum of errors for each frequency between one representative power spectrum of each representative power spectrum group and the estimation power spectrum.

  Next, the ID of the representative power spectrum nearest to the estimation power spectrum (here, the ID of the representative power spectrum having the smallest sum of errors for each frequency) is selected (K3) and stored in the heap memory (step K4). Steps K2 to K4 are repeated for each estimation power spectrum in the estimation power spectrum list, a predetermined number of selected representative power spectrum IDs are acquired, and a minimum error estimation data log is generated. Here, the minimum error estimation data is the ID of the representative power spectrum nearest to the estimation power spectrum obtained in step K3, and the minimum error estimation data log obtained by obtaining a predetermined number thereof. . The representative power spectrum ID is an ID of a power spectrum representing each movement state, and the movement state in a predetermined period can be specified from the selected representative power spectrum ID.

  Here, if the movement state is estimated using only one estimation power spectrum, the estimation accuracy decreases when the acceleration includes a sudden change. Therefore, the majority of the ID of the representative power spectrum is obtained from a certain number of minimum error estimation data logs in the past, and the movement state corresponding to the ID of the representative power spectrum with the highest frequency is acquired as an estimation result (step K4). Here, the past fixed number is obtained based on the time required for estimating the first predetermined movement state (here, the walking state and the running state) that can be estimated using the minimum error estimation method. In this embodiment, the time required for estimating the first predetermined movement state is 30 seconds, and a minimum error estimation data log composed of 60 minimum error estimation data is acquired. For example, even if the movement state corresponding to the latest estimation symbol is a bicycle, if the majority result is determined to be “walking state”, the estimated movement state is set to “walking state”.

  Although this method does not treat temporal changes in acceleration values as state transitions as in the case of estimation using HMM estimation, the mobile terminal device can be made physically by taking a majority decision using estimation results for a certain period in the past. It is possible to prevent a decrease in the estimation accuracy of the moving state due to a sudden change in acceleration caused by contact with the head. Therefore, in the case of a moving state in which high-accuracy estimation can be performed with a power spectrum of 1 for acceleration, the moving state can be estimated by the minimum error estimation method. Here, in the case of a moving state in which high-precision estimation can be performed with one power spectrum, the feature of the form of the power spectrum of acceleration obtained during a predetermined period is remarkable, and high-precision estimation can be performed with one power spectrum. Such as a running state and a walking state. In addition, in the case of sudden change in acceleration, the original power spectrum is restored in a short time, so that it is possible to estimate the moving state with high accuracy using a simple minimum error estimation method.

However, when the form of the estimation power spectrum is similar to the form of the power spectrum indicating another movement state for a certain period or longer, it is difficult to accurately estimate the movement state using the power spectrum for one acceleration. In this case, the estimation accuracy can be further improved by estimating the moving state by HMM estimation using the following acceleration sensor.
HMM Estimation Method Using Acceleration Sensor When the movement state estimated by the minimum error estimation method is not the first predetermined movement state, HMM estimation using an acceleration sensor is performed. The HMM estimation method using the acceleration sensor is performed according to the processing flow shown in FIG. Thereby, the movement state corresponding to the HMM having the highest likelihood is specified every predetermined time (here, 2 seconds), and estimated data for acceleration is obtained. Here, the estimated data is an ID that evaluates the likelihood of the HMM through a function composed of the HMM, identifies the HMM having the maximum likelihood, and indicates the movement state corresponding to the HMM. A log is a log of estimated data acquired per unit time (for example, 3 seconds). In this embodiment, the shift width is set to 0.5 seconds, and an estimated data log consisting of 6 estimated data is obtained in 3 seconds.

  When the acceleration includes a sudden change, if the movement state is estimated from one estimation data, the estimation accuracy decreases due to the sudden state change. Therefore, the majority of the estimated data is determined from the estimated data log for a certain number of accelerations in the past, and the movement state corresponding to the ID of the most representative power spectrum is specified. When the identified movement state is the second predetermined movement state (for example, a bicycle state, a stopped state), the identification result is set as the estimated movement state.

Since the HMM estimation method estimates the user's movement state from the state transition of the movement state via a function composed of the HMM, the form of the estimation power spectrum based on the acceleration value and the representative power spectrum in the other movement state Even if there is a certain period similar to the form, it is possible to improve the estimation accuracy of the moving state.
HMM estimation method using microphone sensor When the movement state estimated by the HMM estimation method using the acceleration sensor is not the second predetermined movement state, HMM estimation is performed using the microphone sensor. For example, in a walking state, a running state, a stopped state, and a bicycle state, various time-series changes appear in the form of a power spectrum for sound according to the environment where the user is placed. When the form of the power spectrum for the sound has various time series changes depending on the environment, the time series change of the power spectrum corresponding to the environment is more than the time series change of the power spectrum for the sound obtained for each moving state. If it appears prominently, it may be difficult to estimate the movement state of the user via the HMM. For example, in the walking state and the running state, various changes appear in the form of the power spectrum with respect to the sound depending on the environment where the user is placed, but the time-series changes in the form of the power spectrum are similar in any moving state In this case, it is difficult to estimate the movement state. Therefore, there are movement states in which it is difficult to estimate the movement state using only the microphone sensor. On the other hand, for example, in a train state, a bus state, and an automobile state, a time-series change in the form of a power spectrum with respect to sound according to the environment in which the user is placed appears significantly for each movement state. Therefore, for the train state, bus state, and car state, it is possible to estimate the moving state by calculating the power spectrum from the sound data and paying attention to the time series change.

  In this embodiment, the microphone sensor is used for a second predetermined movement state (for example, a train state, a bus state, and an automobile state) in which the movement state cannot be estimated with sufficiently high accuracy by HMM estimation using an acceleration sensor. HMM estimation was performed. In the HMM estimation using the microphone sensor, the HMM estimation processing (FIG. 3) and the HMM learning processing (FIG. 6) are performed according to the processing flow described above, but the processing for symbolizing sound data obtained from the microphone sensor is different. Hereinafter, a processing flow for symbolizing the estimation sound data will be described with reference to FIG. Note that the learning sound data is also symbolized by the same processing.

  First, sound data is continuously acquired from a microphone sensor of a user every predetermined period (for example, 9 seconds) for a certain period (for example, 300 msec), time window data of sound data is generated, and the result is stored in a heap memory. Accumulate (step L1). Here, for example, if the sampling rate of the microphone sensor is 24 KHz, the time window data of the sound data is composed of 7200 pieces of sound data in 300 milliseconds. The fixed period is the shortest period in which the characteristics of each moving state appear in the power spectrum that has been FF-converted when the sound data is FF-converted. A shorter period is desirable because it reduces the calculation load on the CPU, but if it is too short, the characteristics of each moving state do not appear in the power spectrum, so a period having a certain time width is required. The predetermined period is the shortest period in which a certain change appears in the acquired sound data. If the predetermined period is too short, there will be no constant change in the sound data. If it is too long, not only will the processing load increase, but the temporal change information of the sound data may be lost by FF conversion. It takes a period of

  Next, the time window is shifted by a predetermined period, and the step of generating time window data of the acquired sound data is repeated to generate a bundle of time window data of the sound data. The time window data of the sound data is FF-transformed for each of the generated time window data bundles to generate a power spectrum list for estimating sound data (step L2). Next, paying attention to the band (here, 100 to 1500 Hz) in which the characteristics according to the moving state appear in the power spectrum of the sound data, the average of the amplitude of the estimation power spectrum for every fixed frequency (here, 100 Hz) A value is calculated (step L3). Note that, in the case of a power spectrum obtained from sound data, a feature does not appear in the power spectrum with a narrow frequency width (for example, 1 Hz), so an average value of the amplitude of the power spectrum is calculated for each fixed frequency.

  Then, a vector composed of an average value of the amplitudes of a plurality of calculated (14 in this case) estimation power spectra is used as an estimation vector, and the generation of the estimation vector is performed as an estimation power spectrum in the estimation power spectrum list. Every time, the estimation vector list is generated (step L4).

  Next, the estimation vector generated in step L2 is compared with one representative vector in the representative vector list group, and the ID of the nearest representative vector is selected for the estimation vector and stored in the heap memory. (Step L5). Then, the accumulation process is performed for each estimation vector in the estimation vector list, and the ID of the selected representative vector is sequentially accumulated in the estimation symbol string to generate an estimation symbol string (step L6). Here, the representative vector / list group is the learning sound data generated by acquiring the sound data in the reference state in a certain specific moving state for a certain period, and for each moving state according to the processing flow of FIG. A group of representative vector lists to be selected. The symbolization of the estimation sound data refers to a series of processes from steps L1 to L5 described above.

Next, an evaluation value for each HMM is calculated using the symbol string for estimation of sound data obtained by symbolization (step L7), the movement state corresponding to the HMM having the highest likelihood is specified, and the estimation result Is obtained (step L8). When the estimated result is the third movement state, the result is set as the estimated movement state. With this method, the microphone sensor can detect the power spectrum for estimation with respect to the acceleration and the power spectrum in the other moving state in a moving state with a long time zone (for example, an automobile state, a bus state, a train state). By using the obtained sound data, it is possible to obtain a characteristic difference in the time series change of the power spectrum with respect to the sound of the third predetermined movement state (in this case, the automobile state), and to improve the estimation accuracy Can do.
By using HMM estimation using an HMM estimation method microphone sensor using a positioning system, it is possible to expect an improvement in estimation accuracy even in a bus state and a train state. However, for example, when the volume of environmental noise obtained from the microphone sensor is reduced as a whole, such as when the train is congested, the form of the power spectrum of the sound in the bus state and the train state may be similar. In addition, it is difficult to estimate the movement state with high accuracy using the microphone sensor. Therefore, the estimation accuracy is improved by performing HMM estimation on the position data using the positioning system. The position positioning system includes a system such as GPS (Global Positioning System). GPS is a system that determines the position of a user on the earth using a satellite in low earth orbit. The DGPS (Differential Global Positioning System) is a system that improves positioning accuracy using GPS by correcting the error from the position received by the GPS receiver and the true position. AGPS (Assisted Global Positioning System) is a system in which a server on a network calculates a position, reduces processing capacity built into the GPS receiver, and enables high-speed processing.

  In the present embodiment, HMM estimation is performed using position data obtained by a GPS positioning system receiver. Here, when GPS is used, it is difficult to estimate the moving state in an environment where GPS positioning is impossible, such as indoors, and a positioning error of several meters to around 100 m occurs. It is difficult to estimate a moving state in which the moving distance in a predetermined period, such as a traveling state, is within the range of positioning error. However, for example, in the case of high-speed movement such as the car state, bus state, train state, etc., there is little influence by positioning error, and for the moving state that performs characteristic acceleration / deceleration, the average speed is calculated from the periodic positioning result, The movement state can be estimated by paying attention to the time series change.

  In the HMM estimation using the positioning system, the HMM estimation processing (FIG. 3) and the HMM learning processing (FIG. 6) are performed according to the above-described processing flow, and the velocity data obtained from the receiver of the positioning system is symbolized. The series of processing is different. Hereinafter, a processing flow for symbolizing the estimation speed data will be described with reference to FIG. Note that the learning speed data is also symbolized by the same process.

  First, position data including longitude and latitude is acquired from a position positioning system receiver of the user every predetermined period (for example, 20 seconds) (step M1), and the result is stored in a heap memory. Here, since the GPS positioning error is about several meters to 100 meters during the predetermined period, if it is too short, the accuracy of measuring the average speed will not be improved. Therefore, a certain period is required.

  Next, the nth position data among the position data stored in the memory is sequentially specified as the nth position data. Here, n is an integer from 1 to N. Then, average speed data required for the movement between the two points is generated from the distance between the nth position data and the (n + 1) th position data and the time required for the movement between the two points (step M2). . Then, the average speed data is generated at least twice in the order in which the position data is specified, and an average speed data list is generated.

  Next, the average speed data calculated in step M2 is quantized for each fixed unit (for example, 10 km / h) to generate speed data for estimation (step M3). Here, the quantization of the average speed data is performed in order to reduce the calculation amount by the HMM estimation. The estimation speed data is generated for each average speed data in the average speed data list, and an estimation speed data list including the estimation speed data is generated (step M4). The generation of the estimation speed data list may be obtained by moving average the average speed data obtained in step M2. The moving average is a technique for smoothing time-series data. Specifically, first, an average speed data list is generated from the average speed data. Next, an average value of m pieces of average speed data is calculated using the n-th to n + m-th second average speed data in the generated average speed data list. Then, the step of calculating the average value is repeated at least twice in the specific order in which the position data is specified, and the average speed data list is generated. Here, m is an integer from 1 to M, and M is a value smaller than N.

  Next, the generated estimation speed data is compared with each representative speed data in the representative speed data list group, the ID of the nearest representative speed data is selected for the estimation speed data, and the heap memory (Step M5). Then, the above accumulation process is performed for each estimation speed data in the estimation speed data list, and the ID of the selected representative speed data is sequentially stored in the estimation symbol string to generate an estimation symbol string (step M6). . Here, the representative speed data list group refers to the moving state according to the processing flow of FIG. 5 using the learning speed data generated by acquiring the position data of the reference state in a specific moving state for a certain period. A group consisting of a list of representative speed data selected for each. The symbolization of the estimation speed data refers to a series of processes from the above-described steps M1 to M5.

  Next, an evaluation value for each HMM is calculated using the symbol string for estimation of velocity data obtained by symbolization (step M7), the movement state corresponding to the HMM having the highest likelihood is specified, and the estimation result (Step M8), and the result is set as the estimated movement state. By this method, the position measurement system reception is performed in a long-time moving state (in this case, a train state and a bus state) in which the form of the power spectrum for sound is similar to the form of the power spectrum for sounds in other moving states. By using the average speed data obtained from the machine, it is possible to obtain a characteristic difference in the time series change of the speed data with respect to each moving speed, and to improve the estimation accuracy.

It is a system configuration figure in one embodiment of the present invention. It is a flowchart of the model creation process for learning. It is a flowchart of the estimation process of a movement state. It is a flowchart of the production | generation process of the power spectrum list | wrist for learning. It is a flowchart of the selection process of a representative power spectrum list. It is a flowchart of the learning process of HMM. It is a processing flow figure by the multistage estimation system using a variable-length symbol sequence. It is a movement state schema diagram of a movement state. It is a flowchart of the majority vote estimation process using an estimation data log. It is a flowchart of an acquisition process of an estimation data log. It is a system configuration figure in one embodiment of the present invention. It is a processing flowchart by the movement state estimation method using multiple types of external environment information. It is an estimation processing flowchart of a user's movement state using the minimum error estimation method It is an estimation processing flow diagram of the movement state using the microphone sensor It is an estimation process flow figure of a movement state using a position positioning system.

Claims (36)

A method for estimating a movement state of a mobile terminal device including at least a central processing unit (CPU), a memory, and an environment information acquisition device, wherein the memory includes at least two or more representing a predetermined movement state Storing a first group of symbol sequences composed of symbols;
The CPU is
Symbolizing environmental information values collected for a predetermined period by the environmental information acquisition device;
Repeating the step of shifting the time window by a predetermined period to make the symbol at least twice, and generating a second symbol string composed of the plurality of symbolized environment information values generated repeatedly;
Determining a likelihood of the second symbol sequence for the first group of symbol sequences;
And estimating the movement state by specifying the first symbol sequence having the maximum likelihood.   The environmental information acquisition apparatus according to claim 1, comprising at least one of an acceleration sensor, a microphone sensor, and a positioning system receiver. A method for estimating a movement state of a mobile terminal device including at least a central processing unit, a memory, and an acceleration sensor having a predetermined sampling rate, wherein the memory represents at least two representative of the predetermined movement state Storing a group of first spectrum lists composed of the above spectra;
The CPU is
Continuously collecting acceleration values by the acceleration sensor;
In the collected continuous acceleration values, a time window data generation step of generating a spectrum composed of data of the continuous acceleration values for a predetermined period as time window data;
Repeating the step of generating the time window data by shifting the time window for a certain period at least twice, and generating a data list for estimation from the plurality of time window data repeatedly generated;
Fast Fourier transforming the generated estimation data list to generate a second spectrum list;
Evaluating the likelihood of the second spectrum list for the group of first spectrum lists;
The movement state is estimated by specifying the first spectrum list having the maximum likelihood. A method for estimating a moving state of a mobile terminal device including at least a central processing unit, a memory, and a microphone sensor having a predetermined sampling rate, wherein the memory represents at least two representing a predetermined moving state Store a first vector list group consisting of the above vectors,
The CPU is
A time window data generating step of collecting sound data continuously for a predetermined period by the microphone sensor and generating a spectrum composed of the collected continuous sound data as time window data;
Repeating the step of generating the time window data by shifting the time window by the predetermined period at least twice, and generating a data list for estimation from the plurality of time window data repeatedly generated;
Fast Fourier transforming the generated estimation data list to generate a second spectrum list;
Calculating an average value of the amplitude of the second spectrum for each predetermined frequency from the generated second spectrum list, and generating a second vector list;
Determining a likelihood of the second vector list for the group of first vector lists;
And estimating the movement state by specifying the first vector list having the maximum likelihood. A method for estimating a moving state of a mobile terminal device including at least a central processing unit, a memory, and a positioning system receiver, wherein the memory is at least two or more speed data representing a predetermined moving state. Storing a group of first velocity data lists consisting of:
The CPU is
Collecting position data and storing it in a memory every predetermined period by the positioning system receiver;
The nth position data among the position data stored in the memory is used as the nth position data, and the Nth position data is sequentially specified, where n is an integer from 1 to N; ,
An average speed data generation step for generating an average speed between two points using the nth position data and the (n + 1) th position data;
Generating the average speed data at least twice or more in the specific order, and generating a second speed data list from the plurality of average speed data generated repeatedly;
Determining a likelihood of the second speed data list for the group of first speed data lists;
The moving state is estimated by specifying the first speed data list having the maximum likelihood. The step of generating the second velocity data list according to claim 5 comprises:
The step of generating the average speed data is repeated at least twice or more in the specific order, and the average speed data list is generated from the plurality of average speed data generated repeatedly.
A step of calculating an average value of m average speed data using the nth to n + mth average speed data from the generated average speed data list, wherein m is 1 to M A step in which M is an integer less than N;
Repeating the step of calculating the average value at least twice or more in the specific order, and generating a second speed data list from the plurality of average values repeatedly generated;
The method of claim 5 comprising: When the movement state estimated by the method according to claim 3 is the first or second predetermined movement state, the first step of specifying the estimated movement state as the movement state;
When the movement state estimated by the method according to claim 3 is not the first or second predetermined movement state, the movement state is further estimated by the method according to claim 4, and the estimated movement state is A second step of identifying the estimated movement state as the movement state when the predetermined movement state is 3,
When the movement state estimated by the method according to claim 4 is not the third predetermined movement state, the movement state is further estimated by the method according to claim 5, and the estimated movement state is set as the movement state. How to identify. A method for estimating a movement state of a mobile terminal device including at least a central processing unit, a memory, and an acceleration sensor having a predetermined sampling rate, wherein the memory represents at least two representative of the predetermined movement state Storing a group of first spectrum lists composed of the above spectra;
The CPU is
Continuously collecting acceleration values by the acceleration sensor;
In the collected continuous acceleration values, a time window data generation step of generating a spectrum composed of data of the continuous acceleration values for a predetermined period as time window data;
Repeating the step of generating the time window data by shifting the time window for a certain period at least twice, and generating a data list for estimation from the plurality of time window data repeatedly generated;
Fast Fourier transforming the generated estimation data list to generate a second spectrum list;
By comparing one spectrum of each group of the first spectrum list with one second spectrum of each of the second spectrum list, the second spectrum in the predetermined period and the latest spectrum Selecting a first side spectrum;
And the step of selecting the first spectrum is repeated a predetermined number of times, and the first spectrum list having the highest frequency is specified, thereby estimating the movement state. In the first step of claim 7,
The movement estimated by the method according to claim 8 is specified by identifying the estimated movement state as a movement state when the movement state estimated by the method according to claim 8 is a first predetermined movement state. When the state is not the first movement state, the movement state is estimated by the method according to claim 3, and when the estimated movement state is the second predetermined movement state, the estimated movement state is The method to identify as a moving state. A system for estimating a movement state of a mobile terminal device including at least a central processing unit, a memory, and an environment information acquisition device, wherein the memory is composed of at least two symbols representing a predetermined movement state A first group of symbol sequences to be stored,
The CPU is
Symbolize environmental information values collected for a predetermined period by the environmental information acquisition device,
Repeating the step of symbolizing the time window by shifting a certain period at least twice, to generate a second symbol string comprising a plurality of symbolized environment information values generated repeatedly,
A system for obtaining a likelihood of the second symbol sequence with respect to the group of the first symbol sequences, and estimating a movement state by specifying the first symbol sequence with the maximum likelihood.   The environmental information acquisition apparatus according to claim 10, comprising at least one of an acceleration sensor, a microphone sensor, and a positioning system receiver. A system for estimating a movement state of a mobile terminal device including at least a central processing unit, a memory, and an acceleration sensor having a predetermined sampling rate, wherein the memory represents at least two representative of the predetermined movement state Storing a group of first spectrum lists composed of the above spectra;
The CPU is
The acceleration sensor continuously collects acceleration values,
In the collected continuous acceleration value, a spectrum composed of data of the continuous acceleration value for a predetermined period is generated as time window data,
The generation of the time window data is repeated at least twice by shifting the time window for a certain period, and an estimation data list is generated from the plurality of time window data generated repeatedly.
Fast Fourier transform the generated estimation data list to generate a second spectrum list;
A system for evaluating a likelihood of the second spectrum list with respect to the group of the first spectrum list, and estimating a movement state by specifying the first spectrum list having the maximum likelihood . A system for estimating a movement state of a mobile terminal device including at least a central processing unit, a memory, and a microphone sensor having a predetermined sampling rate, wherein the memory is at least two representative of the predetermined movement state Store a first vector list group consisting of the above vectors,
The CPU is
The microphone sensor continuously collects sound data for a predetermined period every predetermined period, generates a spectrum composed of the collected continuous sound data as time window data,
The time window is shifted by the predetermined time period and the generation of the time window data is repeated at least twice, and a data list for estimation is generated from the plurality of time window data repeatedly generated,
Fast Fourier transform the generated estimation data list to generate a second spectrum list;
Calculating an average value of the amplitude of the second spectrum for each fixed frequency from the generated second spectrum list, and generating a second vector list;
A system for estimating a moving state by determining a likelihood of the second vector list with respect to the group of the first vector list and specifying the first vector list having the maximum likelihood. A system for estimating a moving state of a mobile terminal device including at least a central processing unit, a memory, and a position positioning system receiver, wherein the memory has at least two or more speed data representing a predetermined moving state. Storing a group of first velocity data lists consisting of:
The CPU is
Collecting position data every predetermined period by the positioning system receiver and storing it in a memory;
Of the position data stored in the memory, the n-th position data is set as the n-th position data, and the N-th position data is sequentially specified, where n is an integer from 1 to N,
Using the nth position data and the (n + 1) th position data, an average speed between the two points is generated,
The generation of the average speed data is repeated at least twice or more in the specified order, and a second speed data list is generated from the plurality of average speed data generated repeatedly,
The likelihood of the second speed data list is obtained with respect to the group of the first speed data list, and the movement state is estimated by specifying the first speed data list having the maximum likelihood. System. The generation of the second speed data list according to claim 14 comprises:
The generation of the average speed data is repeated at least twice or more in the specific order, and an average speed data list is generated from the plurality of average speed data generated repeatedly.
The average value of m average speed data is calculated using the nth to n + mth average speed data from the generated average speed data list, where m is an integer from 1 to M. M is a value smaller than N,
The system according to claim 14, wherein the calculation of the average value is repeated at least twice or more in the specific order, and a second speed data list is generated from the plurality of average values repeatedly generated. When the movement state estimated by the system according to claim 12 is the first or second predetermined movement state, the estimated movement state is specified as a movement state,
If the movement state estimated by the system according to claim 12 is not the first or second predetermined movement state, the movement state is further estimated by the system according to claim 13, and the estimated movement state is If the predetermined movement state is 3, the estimated movement state is specified as the movement state,
When the movement state estimated by the system according to claim 13 is not the third predetermined movement state, the movement state is further estimated by the system according to claim 14, and the estimated movement state is set as the movement state. The system to identify. A system for estimating a movement state of a mobile terminal device including at least a central processing unit, a memory, and an acceleration sensor having a predetermined sampling rate, wherein the memory represents at least two representative of the predetermined movement state Storing a group of first spectrum lists composed of the above spectra;
The CPU is
The acceleration sensor continuously collects acceleration values,
In the collected continuous acceleration value, a spectrum composed of data of the continuous acceleration value for a predetermined period is generated as time window data,
The generation of the time window data is repeated at least twice by shifting the time window for a certain period, and an estimation data list is generated from the plurality of time window data generated repeatedly.
Fast Fourier transform the generated estimation data list to generate a second spectrum list;
By comparing one spectrum of each group of the first spectrum list with one second spectrum of each of the second spectrum list, the second spectrum in the predetermined period and the latest spectrum A system for estimating a moving state by selecting a first spectrum in the vicinity, repeating the selection of the first spectrum a predetermined number of times, and specifying a first spectrum list having the highest frequency. When the estimated movement state according to claim 16 is the first or second predetermined movement state, the identification of the movement state is:
The movement estimated by the system according to claim 17 is specified as the movement state when the movement state estimated by the system according to claim 17 is the first predetermined movement state. When the state is not the first movement state, the movement state is estimated by the system according to claim 12, and when the estimated movement state is the second predetermined movement state, the estimated movement state is A system that is performed by specifying a moving state. A method for estimating a movement state of a mobile terminal device including at least a central processing unit, a memory, and an acceleration sensor having a predetermined sampling rate, wherein the memory represents at least two representative of the predetermined movement state Storing a group of first spectrum lists composed of the above spectra;
The CPU is
Continuously collecting acceleration values by the acceleration sensor;
In the collected continuous acceleration values, a time window data generation step of generating a spectrum composed of data of the continuous acceleration values for a predetermined period as time window data;
Repeating the step of generating the time window data by shifting the time window for a certain period at least twice, and generating a data list for estimation from the plurality of time window data repeatedly generated;
Fast Fourier transforming the generated estimation data list to generate a second spectrum list;
Evaluating the likelihood of the second spectrum list for the group of first spectrum lists;
The movement state is estimated by specifying the first spectrum list having the maximum likelihood. Evaluating the likelihood comprises:
By the CPU
The second spectrum list is configured by comparing and fitting the second spectrum list with respect to the first spectrum list group, among the spectra constituting the first spectrum list. Select the nearest spectrum for each spectrum,
The method according to claim 19, wherein the likelihood of the HMM is evaluated for each moving state through a function composed of the HMM (Hidden Markov Model) for each selected first spectrum set. The first spectrum list is:
The CPU is
Collecting the acceleration values for learning continuously by the acceleration sensor;
Generation of learning time window data for generating a learning spectrum composed of data of acceleration values for continuous learning for a predetermined period as the time window data for learning in the collected continuous learning acceleration values Steps,
Repeating the step of generating the learning time window data by shifting the time window for a fixed period at least twice, and generating a learning data list from the plurality of learning time window data repeatedly generated;
Fast Fourier transforming the generated learning data list to generate a learning power spectrum list; and
Applying a genetic algorithm to the learning power spectrum list and calculating an evaluation value;
21. Method according to claim 19 or 20, characterized in that it is determined by making the list with the highest calculated evaluation value the first spectrum list. The step of evaluating the likelihood of the HMM according to claim 20, wherein the CPU further comprises:
Evaluating the likelihood of each HMM for each selected first set of spectra via a function comprising the HMM over a predetermined time span. In addition to the steps recited in claim 19, the CPU further comprises:
A first step of evaluating the likelihood of the HMM according to claim 22 for each movement state;
A second step of identifying a movement state in which the likelihood is maximum;
And when the HMM corresponding to the maximum likelihood obtained in the predetermined time width is not in a predetermined moving state, the first step and the first step with a predetermined time width longer than the predetermined time width 21. The method according to claim 20, wherein the moving state is estimated by repeatedly specifying the moving state by repeating the second step. The step of collecting acceleration values according to claim 19, wherein the CPU further collects acceleration values per unit time continuously by the acceleration sensor;
The movement state estimated by specifying the first spectrum list having the maximum likelihood is stored in a memory as estimated data per unit time. A method for estimating the movement state described. 25. The method according to claim 24, wherein the data structure in which the estimated movement state and the movement state having a high frequency of being mistaken for the estimated movement state are grouped together includes one or a plurality of movement states, respectively. Or a plurality of movement state groups, wherein the movement state group is a method of estimating a movement state using a data structure including one basic movement state, wherein the CPU
Reading the estimated data per unit time from the memory;
Identifying the type of movement state group of the data structure using the estimated data per unit time;
Obtaining estimated data per unit time for a predetermined period and generating an estimated data log;
Estimating one movement state included in the movement state group by majority estimation using the estimation data log; and
A method for estimating a moving state.   26. The method according to claim 25, wherein the unit time is a time required to acquire a predetermined number of estimated data required to identify the type of movement state group of the data structure. The step of estimating the movement state by the majority voting method using the estimation data log according to claim 25,
If the highest frequency movement state included in the estimated data log belongs to the identified movement state group type, identify the highest frequency movement state as the estimated movement state;
If the most frequent movement state included in the estimated data log does not belong to the type of the identified movement state group, the basic movement state of the identified movement state group is specified as the estimated movement state 27. The method according to claim 25 or 26, wherein the movement state is estimated. A system for estimating a moving state of a mobile terminal device including at least a CPU, a memory, and an acceleration sensor having a predetermined sampling rate, wherein the memory has at least two or more spectra representing the predetermined moving state. Store a first group of spectral lists consisting of:
The CPU is
The acceleration sensor continuously collects acceleration values,
In the collected continuous acceleration value, a spectrum composed of data of the continuous acceleration value for a predetermined period is generated as time window data,
The step of generating the time window data by shifting the time window for a certain period is repeated at least twice, and a data list for estimation is generated from the plurality of time window data repeatedly generated,
Fast Fourier transform the generated estimation data list to generate a second spectrum list;
Evaluating the likelihood of the second spectrum list for the first group of spectrum lists;
A system for estimating a moving state by specifying the first spectrum list having the maximum likelihood. The likelihood evaluation is:
By the CPU
The second spectrum list is configured by comparing and fitting the second spectrum list with respect to the first spectrum list group, among the spectra constituting the first spectrum list. Select the nearest spectrum for each spectrum,
29. The system according to claim 28, wherein the likelihood of the HMM is evaluated for each movement state through the function of the HMM for each set of the first spectra selected. The first spectrum list is:
The CPU is
The acceleration sensor continuously collects acceleration values for learning,
In the collected continuous learning acceleration values, a learning spectrum composed of data of the continuous learning acceleration values for a predetermined period is generated as time window data for learning,
The step of generating the learning time window data by shifting the time window for a certain period is repeated at least twice, and a learning data list is generated from the plurality of repeatedly generated learning time window data,
The generated learning data list is fast Fourier transformed to generate a learning power spectrum list,
Applying a genetic algorithm to the learning power spectrum list, calculating an evaluation value,
30. System according to claim 28 or 29, characterized in that it is determined by making the list with the highest calculated evaluation value the first spectrum list. The likelihood evaluation of the HMM according to claim 29, wherein the CPU further
Evaluating the likelihood of each HMM for each selected first set of spectra through a function comprising the HMM over a predetermined time span. The CPU according to claim 28 further comprises:
A first step of evaluating the likelihood of the HMM according to claim 31 for each movement state;
A second step of identifying a movement state in which the likelihood is maximum;
And when the HMM corresponding to the maximum likelihood obtained in the predetermined time width is not in a predetermined moving state, the first step and the first step with a predetermined time width longer than the predetermined time width 30. The system according to claim 29, wherein the moving state is estimated by repeatedly specifying the moving state by repeating the second step. The acceleration value collection according to claim 28, wherein the CPU further causes the acceleration sensor to continuously collect acceleration values per unit time,
29. The moving state according to claim 28, wherein the moving state estimated by specifying the first spectrum list having the maximum likelihood is stored in a memory as estimated data per unit time. Estimating system. 34. The system according to claim 33, wherein the data structure in which the estimated movement state and a movement state having a high frequency of being mistaken for the estimated movement state are grouped together includes one or a plurality of movement states, respectively. Or a plurality of movement state groups, wherein the movement state group is a system for estimating a movement state using a data structure including one basic movement state, and the CPU includes:
Read the estimated data per unit time from the memory,
Identifying the type of movement state group of the data structure using the estimated data per unit time,
The estimated data per unit time is acquired for a predetermined period to generate an estimated data log,
Estimating one movement state included in the movement state group by a majority voting estimation method using the estimation data log;
A system for estimating a moving state including:   35. The system according to claim 34, wherein the unit time is a time required to acquire a predetermined number of estimated data required to identify the type of movement state group of the data structure. The estimation of the movement state of the majority estimator using the estimation data log according to claim 33 is:
If the highest frequency movement state included in the estimated data log belongs to the identified movement state group type, identify the highest frequency movement state as the estimated movement state;
If the most frequent movement state included in the estimated data log does not belong to the type of the identified movement state group, the basic movement state of the identified movement state group is specified as the estimated movement state 36. The system according to claim 34 or 35, wherein the movement state is estimated.

Images