JP5356923B2 - Method and system for estimating movement state of portable terminal device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and a system by which a model is prepared to estimate the moving condition of users and the model is used to automatically estimate the moving condition of the users by using a portable terminal device. <P>SOLUTION: The method is used to estimate the moving condition of a portable terminal device provided with at least one CPU and a memory. The method includes: the step wherein, according to a difference of each parameter between a feature parameter wherein the CPU is comprised of one or a plurality of parameters stored in the memory in time series and each of representative feature parameters comprised of one or a plurality of parameters, one first cluster wherein the difference satisfies a first condition is selected from among the plurality of first clusters; and the step wherein a moving condition satisfying a second condition is predetermined from the segment probability table for estimation that is computed on the basis of a first probability table corresponding to the selected first cluster, so as to estimate the moving condition of users. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

TECHNICAL FIELD OF THE INVENTION

  The present invention relates to a method and system for estimating a user's movement state, and more particularly, to a method and system for automatically estimating a user's movement state in a ubiquitous environment using a mobile terminal device.

  In recent years, there has been a need to detect a user's movement state, or to automatically recognize and identify a user's movement state accurately, and various techniques relating to estimation of the user's movement state have been proposed. However, conventionally, identifying a certain state of the user, for example, a state where the user is stopped, a state where he / she is on a car, a bus, a train, etc. is an estimation of a moving state even if information from the user is measured over a long period of time. The accuracy was low. Conventionally, when estimating a user's movement state, one power spectrum is selected at regular intervals using data obtained from an acceleration sensor of the user, and the selected power spectrum and each movement state are represented. The movement state was estimated by comparing with each representative power spectrum. Accordingly, one selected power spectrum does not include a time-series change, and if the acceleration sensor suddenly changes, the user state may not be estimated correctly. For this reason, it is difficult for conventional pattern matching of the power spectrum to cope with sudden noise and vibration pattern fluctuations, resulting in a decrease in the estimation accuracy of the moving state.

  Further, in Non-Patent Document 1, acceleration sensors are attached to a plurality of locations on the body, and “sit”, “stand”, “walk”, “up stairs”, “down stairs”, “shake hands”, “write on blackboard”, “keyboard” Motion estimation such as “typing”. In Non-Patent Documents 2 and 3, acceleration sensors are attached to five locations of both wrists, both ankles and thighs, and 20 types of motion estimation are performed.

  However, in these non-patent documents, it is possible to estimate various human operating states using various sensors. However, a plurality of sensor mounting locations are necessary or the mounting method is fixed. . Therefore, when the user's movement state is estimated using these methods, since the user is heavily equipped, the convenience is lowered, which is not desirable.

Nicky Kern, "A Model for Human Interruptability: Experimental Evaluation and Automatic Estimation from Wearable Sensors," ISWC'04, pp.158-165 (2004). Stephen S. Intille, "Acquiring In Situ Training Data for Context-Aware Ubiquitous Computing Applications," CHI 2004, ACM Press, pp. 1-9 (2004). L. Bao and SS Intille, "Activity recognition from user-annotated acceleration data," in Proceedings of PERVASIVE 2004, vol. LNCS 3001, A. Ferscha and F. Mattern, Eds. Berlin Heidelberg: Springer-Verlag, 2004, pp. 1-17.

  The invention of the present application aims to solve the above-described problems, and shortens the movement state of a user by using an external environment information acquisition device such as an acceleration sensor mounted on a portable terminal device that fits in the palm of a person. The present invention provides a method and system for automatically estimating with high accuracy in time. Here, the moving state of the user indicates, for example, a human activity state such as running, walking, and stopping and a riding state of transportation such as a bicycle, a car, a bus, and a train. Hereinafter, the above-described states are referred to as a stopped state, a walking state, a running state, a bicycle state, an automobile state, a bus state, and a train state, respectively. In addition, the automobile state, the bus state, and the train state are collectively referred to as an electric vehicle state. Note that the mobile terminal device may be a size that fits in the palm of a person or a size smaller than that.

  Conventionally, when estimating a user's movement state, one power spectrum is selected at regular intervals using data obtained from an acceleration sensor of the user, and the selected power spectrum and each movement state are represented. The movement state was estimated by comparing with each representative power spectrum. However, the selected one power spectrum generally includes representative power spectrum characteristics corresponding to a plurality of movement states. Therefore, when one power spectrum is similar to a power spectrum in another moving state, it is difficult to estimate the moving state with high estimation accuracy.

  In addition, one selected power spectrum does not include a time-series change, and if the acceleration sensor suddenly changes, the user state may not be estimated correctly. For this reason, it is difficult for conventional pattern matching of the power spectrum to cope with sudden noise and vibration pattern fluctuations, resulting in a decrease in the estimation accuracy of the moving state.

  Conventionally, various human operating states have been estimated using various sensors. However, a plurality of sensor mounting locations are required or the mounting method is fixed. In addition, multiple devices consume a lot of power. Therefore, when the user's movement state is estimated using these methods, since the user is heavily equipped, the convenience is lowered, which is not desirable.

  The present invention generates a model for estimating a user's movement state, and estimates the user's movement state by specifying the movement state from the feature parameters of the user in the movement state using the generated model. For the purpose.

  The invention of the present application is characterized by estimating a moving state by paying attention to the fact that feature parameter information obtained from a user's acceleration sensor generally has a feature corresponding to one or a plurality of moving states. To do. The present invention is based on one estimation feature parameter obtained from an acceleration sensor possessed by the user, and determines the actual user's movement state as a probability for each movement state, and by selecting the movement state having the highest probability, This improves the estimation accuracy of the moving state. Further, according to the present invention, it is also possible to estimate the possessed state of the portable end device, such as a state in a pocket or a bag, a state possessed by a hand, or a state worn on an arm. Furthermore, according to the present invention, it is possible to specify a person who owns the mobile terminal device.

  Further, the present invention generates a model composed of clusters generated from a learning acceleration data list acquired in a specific reference state, and estimates a user's movement state using the generated model. Is. The power spectrum obtained by simply Fourier-transforming the user's acceleration data value obtained from the acceleration sensor does not have information on the time series change of the acceleration data. In the present invention, a model is created using time series data of characteristic parameters obtained from an acceleration sensor possessed by the user, and the movement state of the user is estimated using the model. Therefore, in the present invention, since the user's movement state is estimated using time-series data indicating the user's movement state, there are cases where the user movement state includes a sudden change or a time zone similar to other states. Even if it occurs, it is possible to correctly estimate the movement state of the user.

  The present invention estimates a user's movement state using a model composed of clusters generated from a learning acceleration data list acquired in a specific reference state. The feature parameter obtained from one acceleration data may be similar to the feature parameter in a plurality of moving states. In the present invention, even when the feature parameter obtained from one acceleration data is similar to the feature parameter in a plurality of movement states, the movement state is determined from the frequency at which the feature parameter appears in each movement state by using a model. It is possible to estimate. Therefore, even when the feature parameters are similar to those in a plurality of movement states, the movement state of the user can be estimated with high accuracy.

  Further, according to the present invention, a plurality of estimation models, that is, a first model and a second model are generated at the time of estimating a user's movement state, and the estimation result obtained from the first model and the second model are obtained. When the estimation results are equal, the movement state estimated by the first model is determined to be highly reliable, and the estimated movement state is specified as the movement state of the user. As described above, the present invention evaluates the reliability of the obtained estimation result and extracts only the time zone with high reliability, thereby enabling estimation with higher reliability.

  Further, the present invention mainly estimates a user's movement state using an acceleration sensor included in the mobile terminal device. Moreover, in this invention, it is also possible to estimate a user's movement state based on the information obtained from the other external information acquisition apparatus with which a portable terminal device is equipped other than the information obtained from an acceleration sensor. Therefore, the user can estimate the movement state with light equipment without installing the device in addition to the portable terminal device. In addition, these external information acquisition devices consume less power than an acceleration sensor, and the processing load when acquiring external information is also small. In the present invention, usually, whether or not there is a change in the movement state of the user is determined by these external information acquisition devices that consume less power and have a smaller processing load than the acceleration sensor. When there is a change, it is possible to reduce the processing load for the entire estimation process and save power by estimating the movement state using the acceleration sensor.

  In addition, the present invention estimates the moving state by using other external information acquisition devices in combination with the acceleration obtained from the acceleration sensor. By acquiring external information of a plurality of types of users as described above, it is possible to further improve the estimation accuracy of the user's movement state.

  As described above, the present invention is characterized in that the movement state is automatically estimated without depending on the possession state of the mobile terminal device, without depending on the environment where the user is placed, and without requiring any operation by the user. In addition, the sensor data obtained in a relatively short period of time is used to reduce the load required for one estimation process and to detect a change in the movement state of the user in a short time.

  The present invention is a method for estimating a movement state of a mobile terminal device including at least one CPU (central processing unit) and a memory, and the memory is a plurality of first clusters, each of which is the first cluster. A plurality of first clusters corresponding to one representative feature parameter (first average power spectrum) representing one and a first probability table (probability table) having a probability for each moving state, For each parameter of the estimation feature parameter (estimation power spectrum) composed of one or a plurality of parameters and each representative feature parameter composed of one or a plurality of parameters stored in the memory in time series A step of selecting one first cluster that satisfies the first condition from the plurality of first clusters, and corresponding to the selected first cluster. A method comprising a step of estimating a moving state by specifying a moving state satisfying the second condition from an estimation segment probability table calculated based on the first probability table, and (first estimation). .

  Here, the first probability table is characterized by having probability data for each moving state corresponding to the learning feature parameter included in each first cluster for each moving state.

  The present invention is a method for estimating a movement state of a mobile terminal device including at least one CPU and a memory, and the memory is a plurality of first clusters, each representing the first cluster. A plurality of first clusters corresponding to one representative feature parameter (first average power spectrum) and one first probability table (probability table) having a probability for each moving state are stored. Based on a difference for each parameter between an estimation feature parameter (estimation power spectrum) composed of one or a plurality of parameters and each representative feature parameter composed of one or a plurality of parameters stored in the memory Selecting a first cluster whose difference satisfies the first condition from a plurality of first clusters, and a first probability table corresponding to the selected first cluster. A step of calculating an estimation segment probability table in time series, and a step of generating a plurality of second clusters using the estimation segment probability table, wherein each of the second representatives represents the second cluster. Generating a plurality of second clusters corresponding to a probability table (second average probability table) and one second probability table (moving state rank table) having a probability for each moving state; Based on the difference for each parameter between the estimation segment probability table composed of parameters and each second representative probability table (second average probability table) composed of one or a plurality of parameters, Selecting one second cluster whose difference satisfies the third condition, and selecting a movement state satisfying the fourth condition from the second probability table corresponding to the selected second cluster. In Estimating a moving status Ri features a method consisting of (second estimation).

  Here, the second probability table (moving state rank table) has probability data for each moving state corresponding to the learning segment probability table included in each second cluster for each moving state.

  The present invention is a method for estimating a movement state of a mobile terminal device including at least one CPU and a memory, and the memory is a plurality of first clusters, each representing the first cluster. A plurality of first clusters corresponding to one representative feature parameter and one first probability table having a probability for each moving state are stored, and the CPU stores one or more Based on the difference for each parameter between the estimation feature parameter composed of parameters and each representative feature parameter composed of one or a plurality of parameters, the difference satisfies the first condition from the plurality of first clusters. A step of selecting one first cluster, and a segment probability table for estimation stored in a memory in time series calculated based on the first probability table corresponding to the selected first cluster A first estimation step of estimating a movement state by specifying a movement state satisfying the second condition (first estimation), and a step of generating a plurality of second clusters using the estimation segment probability table A plurality of second clusters each corresponding to one second representative probability table representing the second cluster and one second probability table having a probability for each moving state; Alternatively, based on the difference for each parameter between the estimation segment probability table composed of a plurality of parameters and each of the second representative probability tables composed of one or a plurality of parameters, the difference is calculated from the plurality of clusters. Selecting a second cluster that satisfies the above condition and selecting a movement state that satisfies the fourth condition from the second probability table corresponding to the selected second cluster. A second estimating step of, characterized the steps of estimating the movement by using the results estimated first estimation step, and a second estimation step, the method comprising.

Furthermore, the present invention is characterized by movement state estimation using key frames. More specifically, the present invention is a method for estimating a movement state of a mobile terminal device including at least one CPU and a memory, the memory being a plurality of first clusters, each of the first clusters. A plurality of first clusters corresponding to one representative feature parameter that represents and one first probability table having a probability for each movement state, and the CPU is stored in the memory in time series, Based on the difference for each parameter between the feature parameter for estimation composed of one or a plurality of parameters and each representative feature parameter composed of one or a plurality of parameters, the difference from the plurality of first clusters is the first From the step of selecting one first cluster satisfying the condition and the segment probability table for estimation calculated based on the first probability table corresponding to the selected first cluster, the second condition By specifying the moving condition is satisfied: a first estimating step of estimating the movement (first estimation),
A step of generating a plurality of second clusters using the estimation segment probability table, wherein each of the second representative probability table representing the second cluster and one second probability having a probability for each moving state A plurality of second clusters corresponding to the table, an estimation segment probability table composed of one or more parameters, and each second representative probability table composed of one or more parameters (first A step of selecting one second cluster in which the difference satisfies the third condition from a plurality of second clusters based on the difference for each parameter from the two average probability table), and corresponding to the selected second cluster A second estimation step for estimating a movement state by selecting a movement state satisfying the fourth condition from the second probability table (second estimation), a first estimation step, and a second estimation step In When the estimated result satisfies the fifth condition, a frame list is generated by sequentially storing the frame corresponding to the result obtained by the fifth condition as a key frame in the frame list. In the step and the generated frame list, when one key frame and the immediately preceding key frame correspond to the same movement state, the movement state corresponding to the one key frame is determined to be a predetermined value. In the moving state, a step of determining whether a predetermined condition is satisfied between the one key frame and the immediately preceding key frame, and when the predetermined condition is satisfied, The movement state is estimated by determining the movement state corresponding to the one key frame as the movement state corresponding to the key frame from the immediately preceding one key frame. A method, a method comprising the features.

  Further, the present invention is characterized in that the first model is generated by clustering. More specifically, the present invention is a method for generating a model used for estimating a movement state of a mobile terminal device including at least one CPU and a memory, and the CPU is stored in the memory in time series. Clustering a plurality of learning feature parameters corresponding to all movement states into a plurality of first clusters, each first cluster generating a first cluster group comprising one or more of the learning feature parameters; In the first cluster group, a first cluster corresponding to one representative feature parameter representing each first cluster and a first probability table having a probability that each moving state appears for each first cluster. And (first model).

  Further, the present invention is characterized in that the first model and the second model are generated by clustering. More specifically, the present invention relates to a method for generating first and second models used for estimating a movement state of a mobile terminal device including at least one CPU and a memory, wherein the CPU stores the memory in time series. A plurality of learning feature parameters (learning power spectra) corresponding to all the movement states stored in are clustered into a plurality of first clusters, and each first cluster includes one or more learning feature parameters. Generating a first cluster group; in the first cluster group, for each first cluster, one representative feature parameter representing each of the first clusters, and a first probability of each moving state appearing A step of generating a first cluster corresponding to the probability table (first model), a learning feature parameter, and a representative feature parameter for each learning feature parameter; And sequentially selecting one first cluster for which the difference satisfies a predetermined condition based on the difference for each parameter, and for each predetermined number of the first clusters selected in sequence, A step of calculating a learning segment probability table based on each first probability table corresponding to, and clustering each of the calculated learning segment probability tables into a plurality of second clusters, wherein each second cluster is 1 Or a step of generating a second cluster group comprising a plurality of learning segment probability tables, and one second representative probability table representing each second cluster in the second cluster group for each second cluster; And a second probability table representing the probability that each moving state appears, and a step of generating a corresponding second cluster (second model).

  Furthermore, the invention of the present application is characterized by estimation of a moving state using the number of steps and base station information. More specifically, the present invention is a method for detecting a change in a moving state of a mobile terminal device including at least one CPU, a pedometer, an antenna, and an acceleration sensor, and the mobile terminal device includes the step count. The first information (number of steps) can be obtained by the meter, the second information (base station information) can be obtained by the antenna, and the third information (acceleration) can be obtained by the acceleration sensor. When the movement state is the first movement state (running, walking, bicycle state), only the first information is acquired, and when the first information corresponds to the first movement state, Acquiring the estimated first movement state as an estimation result, monitoring whether or not the third information has changed, and the estimated movement state is a second movement state (stop state). In the case, the first information and the second information are obtained, and the previous information When the first information and the second information correspond to the second movement state, the estimated second movement state is acquired as an estimation result, and the first information (step number) and The step of monitoring whether or not there is a change in the second information, and when the estimated movement state is the third movement state (train, car, bus state), the first information is acquired. If the first information corresponds to the third movement state, the estimated third movement state is acquired as an estimation result, and whether or not there is a change in the first information. And a step of monitoring, and a method of detecting a change in a moving state.

  Furthermore, this invention relates to the portable terminal device which estimates the movement state of a portable terminal device. More specifically, it is a mobile terminal device that estimates the movement state of the mobile terminal device, and is a memory that stores a plurality of first clusters, each of the first clusters representing one first cluster. Corresponding to the representative feature parameter and the first probability table having one probability for each moving state, and one or a plurality of CPUs stored in the memory in time series Based on the difference for each parameter between the estimation feature parameter composed of the parameters and each of the representative feature parameters composed of one or a plurality of parameters, the difference from the plurality of first clusters satisfies the first condition. By selecting one satisfying first cluster and specifying a movement state satisfying the second condition from the estimation segment probability table calculated based on the probability table corresponding to the selected first cluster. For estimating the movement (first estimation) and said mobile terminal device.

Furthermore, the present invention is a mobile terminal device that estimates a movement state of the mobile terminal device,
A memory for storing a plurality of first clusters, wherein each first cluster includes one representative feature parameter representing each first cluster and one first probability table having a probability for each moving state. Corresponding memory and CPU, each of which is composed of one or a plurality of parameters for estimation and one or a plurality of parameters stored in the memory in time series Based on the difference between each representative feature parameter and each of the representative feature parameters, one first cluster in which the difference satisfies the first condition is selected from the plurality of first clusters and corresponds to the selected first cluster. Calculating a segment probability table for estimation in time series based on the first probability table, and generating a plurality of second clusters using the segment probability table for estimation, each representing the second cluster A plurality of second clusters corresponding to one second representative probability table and one second probability table having a probability for each moving state, and an estimation segment probability composed of one or a plurality of parameters Based on the difference for each parameter between the table and each second representative probability table composed of one or a plurality of parameters, one second cluster whose difference satisfies the third condition is selected from the plurality of second clusters. The mobile terminal device is characterized by selecting and estimating a moving state by selecting a moving state that satisfies the fourth condition from the second probability table corresponding to the selected second cluster (second estimation). .

  Furthermore, the present invention is a portable terminal device, a memory storing a plurality of first clusters, each first cluster including one representative feature parameter representing each first cluster, and a moving state A memory and a CPU corresponding to one first probability table having a probability for each, and the CPU is composed of one or a plurality of parameters stored in the memory in time series One first cluster in which the difference satisfies a first condition from a plurality of first clusters based on a difference for each parameter between a feature parameter for use and each of the representative feature parameters composed of one or a plurality of parameters And the movement state satisfying the second condition is identified from the estimation segment probability table stored in the memory in time series calculated based on the first probability table corresponding to the selected first cluster. You 1st estimation which estimates a movement state by this (1st estimation), It is the step which produces | generates several 2nd clusters using the estimation segment probability table | surface, Comprising: A plurality of second clusters corresponding to the second representative probability table and one second probability table having a probability for each moving state, and an estimation segment probability table composed of one or a plurality of parameters; Based on the difference for each parameter from each second representative probability table composed of one or a plurality of parameters, one second cluster in which the difference satisfies the third condition is selected and selected. The second estimation for estimating the movement state by selecting the movement state satisfying the fourth condition from the second probability table corresponding to the second cluster (second estimation), a first estimation step, And the second estimate For estimating the movement by using the results estimated by step, and wherein the portable terminal device.

  Furthermore, the present invention is a mobile terminal device that estimates a movement state of a mobile terminal device, and is a memory that stores a plurality of first clusters, and each first cluster represents each first cluster. A memory and a CPU corresponding to one representative feature parameter and one first probability table having a probability for each movement state, wherein the CPU is stored in the memory in time series. Alternatively, based on the difference for each parameter between the estimation feature parameter composed of a plurality of parameters and each of the representative feature parameters composed of one or a plurality of parameters, the difference from the plurality of first clusters is the first A first cluster that satisfies the condition is selected, and a movement state that satisfies the second condition is determined from the estimation segment probability table calculated based on the first probability table corresponding to the selected first cluster. Performing a first estimation for estimating a movement state (first estimation) and generating a plurality of second clusters using an estimation segment probability table, each representing the second cluster A plurality of second clusters corresponding to one second representative probability table and one second probability table having a probability for each moving state are generated, and the estimation segment probability composed of one or a plurality of parameters Based on the difference for each parameter between the table and each second representative probability table composed of one or a plurality of parameters, one second cluster whose difference satisfies the third condition is selected from the plurality of second clusters. Selecting, from the second probability table corresponding to the selected second cluster, performing a second estimation for estimating the movement state by selecting a movement state satisfying the fourth condition (second estimation), The first estimation step, When the result estimated by the second estimation step satisfies the fifth condition, the frame corresponding to the result obtained by the fifth condition is sequentially stored in the frame list as a key frame. A frame list is generated according to the above, and in the generated frame list, when one key frame and the immediately preceding key frame correspond to the same movement state, the one corresponds to the one key frame. When the movement state is a predetermined movement state, it is determined whether a predetermined condition is satisfied between the one key frame and the one key frame immediately before the one key frame, and the predetermined condition is satisfied The moving state corresponding to the one key frame from the immediately preceding one key frame is determined as the moving state corresponding to the key frame. Thus, the mobile terminal device that estimates the movement state is characterized.

  Furthermore, the present invention is a system for generating a model used for estimating the movement state of a mobile terminal device, a memory storing a plurality of learning feature parameters corresponding to all movement states in time series, and a CPU Wherein the CPU clusters a plurality of learning feature parameters corresponding to all movement states stored in the memory in time series into a plurality of first clusters, and each of the first clusters is one or more. A first cluster group consisting of learning feature parameters is generated, and in the first cluster group, one representative feature parameter representing each first cluster and each moving state appear for each first cluster. Generating a first cluster corresponding to a first probability table having a probability of (first model).

  Furthermore, the present invention is a system for generating first and second models used for estimating a movement state of a mobile terminal device, and storing a plurality of learning feature parameters corresponding to all movement states in time series. A memory and a CPU, wherein the CPU clusters a plurality of learning feature parameters corresponding to all movement states stored in the memory in time series into a plurality of first clusters. One cluster generates a group of first clusters including one or a plurality of learning feature parameters, and one representative feature parameter that represents each first cluster in the first cluster group for each first cluster. And a first probability table corresponding to the first probability table having the probability that each moving state appears (first model), and for each of the learning feature parameters, the learning feature parameter, Based on the difference for each parameter from the table feature parameter, one first cluster in which the difference satisfies a predetermined condition is sequentially selected, and each first number is selected for each predetermined number of the sequentially selected first clusters. Based on each first probability table corresponding to the cluster, a learning segment probability table is calculated, the calculated learning segment probability tables are clustered into a plurality of second clusters, and each second cluster is 1 or Generating a second cluster group composed of a plurality of learning segment probability tables, and for each second cluster in the second cluster group, one second representative probability table representing each second cluster; The system is characterized by generating a second cluster corresponding to the second probability table representing the probability that each moving state appears (second model).

  Furthermore, the present invention is a system for detecting a change in a moving state of a mobile terminal device including at least one CPU, a pedometer, an antenna, and an acceleration sensor, the mobile terminal device being 1 information, 2nd information can be acquired by the antenna, and 3rd information can be acquired by the acceleration sensor, and when the estimated moving state is the first moving state, the CPU If the first information corresponds to the first movement state, the estimated first movement state is acquired as an estimation result, and the third information is included in the third information. Whether or not there is a change is monitored, and when the estimated movement state is the second movement state, the first information and the second information are acquired, and the first information and the second information If the information corresponds to the second movement state, the estimated second movement is performed. A state is acquired as an estimation result, and whether or not there is a change in the first information and the second information is monitored. When the estimated movement state is the third movement state, the first information When information is acquired and the first information corresponds to the third movement state, the estimated third movement state is acquired as an estimation result, and the first information has a change. It is characterized by a system that detects whether or not the moving state is detected.

It is a system configuration diagram in one embodiment of the present invention. It is a flowchart of the model creation process for learning. FIG. 10 is a flowchart of learning power spectrum (PS) list generation processing. It is a flowchart of the process which clusters the group of the power spectrum list | wrist for a learning. It is a flowchart of the clustering process of the segment probability table list for learning It is a flowchart of the estimation process of the movement state using a 1st model and a 2nd model. It is a system configuration | structure figure in the other embodiment of this invention. It is a flowchart of the process which estimates a movement state using the information from an external information acquisition apparatus in addition to the estimation using a 1st model and a 2nd model. It is a flowchart of the process in the other embodiment which estimates a movement state using the information from an external information acquisition apparatus in addition to the estimation using a 1st model and a 2nd model. It is a flowchart of the cancellation process of a key frame area. It is the figure which showed the data table produced | generated in this invention. It is the figure which showed the data structure of the group of the power spectrum list for learning. It is the figure which showed the data table produced | generated in this invention. It is the figure which showed the data structure of the group of the 1st cluster. It is the figure which showed the data table produced | generated in this invention. It is the figure which showed the data structure of the symbol column for learning. It is the figure which showed the data table produced | generated in this invention. It is the figure which showed the data structure of the segment probability table | list for a learning. It is the figure which showed the data table produced | generated in this invention. It is the figure which showed the data structure of the group of a 2nd cluster. It is the figure which showed the data table produced | generated in this invention. It is the figure which showed the data structure of the power spectrum list for estimation. It is the figure which showed the data table produced | generated in this invention. It is the figure which showed the data structure of the symbol sequence for estimation. It is the figure which showed the data table produced | generated in this invention. It is the figure which showed the data structure which stored the estimation result by a 1st model and a 2nd model, and the frame list | wrist in memory in time series.

FIG. 1 is a diagram showing a system configuration of a user movement state estimation method according to an aspect of the present invention. The user movement state estimation method is mounted on, for example, a mobile terminal device, and the mobile terminal device (A1) includes an acceleration sensor (A50), a memory, and a central processing unit (CPU) (A10). The memory may include a heap memory (A20) and a flash memory (A30). Here, the acceleration sensor acquires acceleration data and stores the acceleration data in a memory space in the sensor. For example, the CPU instructs to load the program from the flash memory and store it in the heap memory. For example, the flash memory stores a program such as a data acquisition function. For example, the heap memory temporarily stores a program stored in the flash memory.
Model Generation FIG. 2 is a flowchart for generating a model. First, learning acceleration data in all reference states is acquired for each reference state, and a group of learning feature parameters is generated. In this embodiment, a learning power spectrum list group is generated as a group of learning feature parameters (B1). The group of the learning power spectrum list is generated by decomposing a function composed of acceleration values into a continuous spectrum of frequency components (for example, Fourier transform) from the learning acceleration data by calculation by the CPU and is stored in the flash memory. Stored. Note that the learning feature parameter is a maximum amplitude frequency of the learning power spectrum (learning PS), a triaxial acceleration vector of the learning acceleration, a square sum of acceleration values, or a variance value of the square sum of acceleration values. You can also. The group of learning power spectrum lists is composed of learning power spectrum lists in all moving states, and the learning power spectrum list is composed of all learning power spectra in a certain reference state. Each learning power spectrum corresponds to a moving state in one reference state. The reference state includes not only the human activity state but also the state where the person is on board, for example, walking state, running state, stop state, bicycle state, car state, bus state, train state, A state assumed to be possible for a user having a mobile terminal device is defined in advance as a reference state. In creating a model, learning acceleration data is acquired in advance from an acceleration sensor of a reference user for each reference state under various conditions. The user data in the reference state is based on various states such as the assumed state of possession of the mobile terminal device, for example, the state in a pocket or a bag, the state in the palm of the hand, or the arm. Can be acquired. In addition, user data in the reference state can be acquired based on user characteristics that can be assumed, such as sex, age, height, weight, and the like. The model is generated based on sufficient variation data acquired in the reference state.

  More specifically, the CPU reads a model creation function (consisting of a Fourier transform function, a clustering function, a pattern matching function, and a symbol string generation function) from the flash memory in advance and loads it onto the heap memory. The CPU calculates the power spectrum list for learning by the Fourier transform function in the model creation function, and stores the calculation result in the heap memory.

Next, the group of the generated power spectrum list for learning is clustered (B2). More specifically, the CPU reads all the learning power spectra (N) included in the group of learning power spectra corresponding to all reference states from the flash memory by the clustering function in the model creation function. Then, after expanding in the heap memory, clustering is performed by causing the clustering function to read it. Here, the group of learning power spectrum lists is a group of learning power spectrum lists corresponding to all reference states. An individual ID is set for each movement state, and each learning power spectrum is associated with an ID of the movement state in the reference state. The group of the learning power spectrum list is composed of one corresponding movement state ID and the amplitude of the learning power spectrum for each frequency (i) for each learning power spectrum (FIG. 11A). . In this embodiment, as shown in FIG. 11A, the seven movement states are all the reference states. N is an arbitrary integer, and i is an arbitrary positive number. Clustering refers to dividing a set to be classified into subsets that achieve internal coupling and external separation. Note that clustering of the power spectrum for learning is performed according to the processing flow shown in FIG.
Generation of first model Next, generation of a model (generation of the first model) is performed (B3). The first model is generated by clustering each of the learning power spectra included in the learning power spectrum list group. Therefore, the first model is composed of a cluster (group of first clusters) having one or a plurality of learning power spectra. Here, a cluster generally refers to each subset after a set is divided, and a cluster group refers to a collection of such clusters.

  Each first cluster included in the first model has one or more feature parameters for learning, and each first cluster corresponds to one first representative feature parameter and one probability table. In the present embodiment, a first average power spectrum (PS) is obtained as a first representative feature parameter representing each first cluster by calculation by the CPU. The first average power spectrum is obtained by obtaining an average value of amplitudes for each frequency of each learning power spectrum included in the first cluster, and setting the obtained average value for each frequency. In the present embodiment, the average power spectrum is used as the first representative feature parameter. However, the first representative feature parameter only needs to be composed of feature parameters representing the corresponding cluster, and various conditions can be satisfied. Can be determined based on The first average power spectrum is stored in the heap memory during the clustering process, that is, during the execution of the step of FIG. 4, and is stored in the flash memory after the completion of the clustering. In addition, the probability table corresponding to each first cluster shows the probability that the ID of the moving state corresponding to each learning power spectrum is included in every learning power spectrum included in each first cluster for each moving state. Generated by seeking. Therefore, the probability table is a data table that associates the value obtained for each movement state with the probability that each movement state ID corresponding to each learning power spectrum is included in the cluster and the corresponding movement state ID. . Then, the CPU writes the probability table generated for each cluster into the flash memory. As described above, the CPU configures the first cluster group by associating the cluster ID, the first average power spectrum, and the probability table with each of the clustered first clusters (M) (FIG. 11B). .

  Next, the first cluster corresponding to the learning power spectrum and the nearest first average power spectrum is selected (B4). More specifically, the first average power spectrum and the learning power spectrum in each cluster in the cluster group consisting of the learning power spectra are pattern-matched by the pattern matching function in the model creation function. That is, the CPU loads one learning power spectrum list stored in time series into the learning power spectrum list, and each frequency of the first average power spectrum corresponding to each cluster in the cluster group The sum of the error of the amplitude of (in the case of the power spectrum) is calculated. The CPU obtains the first average power spectrum having the smallest calculated sum, selects the obtained first average power spectrum as the learning power spectrum and the nearest first average power spectrum, and the first average power spectrum Cluster symbols corresponding to the spectrum are stored in memory (symbolization). Here, one individual symbol is allocated to each cluster.

  Next, the CPU determines whether or not all the learning power spectra included in the learning power spectrum list group have been symbolized (B5). If all the learning power spectra are not symbolized in step B5, the learning power spectrum stored next to the previously loaded learning power spectrum is loaded, and the loaded learning power spectrum is loaded. On the other hand, Step B4 is repeated, and the symbol of the cluster corresponding to the first average power spectrum is stored in the symbol string at the next address of the previously stored symbol. On the other hand, if all the learning power spectra included in the group of the learning power spectrum list are symbolized in step B5, the symbolized symbol sequence stored in the memory is used as the learning symbol sequence. (FIG. 11C) is stored in the heap memory (B6).

  Next, a learning segment probability table is generated (B7). More specifically, the CPU first loads a probability table corresponding to each symbol stored in a certain section (segment) of the learning symbol string from the flash memory. Here, one probability table corresponds to each symbol of the first cluster included in the learning symbol string. The fixed section is a predetermined section that is determined in advance and includes a plurality of (a) symbols. If this fixed section is too short, the estimation error may be increased due to a large influence of sudden noise. If this section is too long, it may not be possible to grasp changes in the movement state that may occur within the period in a timely manner. And the average of the probability in 1 movement state of each probability table loaded by the calculation by CPU is calculated for every movement state. The learning segment probability table is a data table in which the average probability calculated for each moving state for a certain section is associated with the moving state. The generated learning segment probability table is stored in the heap memory in time series for each predetermined interval.

  Next, it is determined whether or not the learning segment probability table has been generated in all sections (N / a section) of the learning symbol string (B8). Here, the entire interval of the learning symbol string is all the intervals when the entire learning symbol string is divided into predetermined intervals. In step B8, when the learning segment probability table corresponding to all sections of the learning symbol string has not been generated, the process of step B7 is repeated. Here, the learning segment probability table stored in time series is referred to as a learning segment probability table list. If the learning segment probability table corresponding to all the sections of the learning symbol string has been generated in step B8, the generation of the learning segment probability table list is completed (B9), and the generation result is stored in the heap memory. The

Next, all the learning segment probability tables included in the generated learning segment probability table list are clustered (B10). More specifically, the clustering is performed on all learning segment probability tables included in the learning segment probability table list by the CPU using a clustering function in the model creation function. The clustering of the learning segment probability table is performed according to the processing flow shown in FIG.
Generation of second model Next, a second model is generated (B11). The second model is generated by clustering each of the learning segment probability tables included in the learning segment probability table list. Therefore, the second model is composed of a collection of clusters (second cluster group) each having one or a plurality of learning segment probability tables. First, the learning segment probability table is clustered by calculation by the CPU. The clustering result is stored in the flash memory.

  By clustering, each second cluster included in the second cluster group constituting the second model has one or a plurality of learning segment probability tables. Each second cluster corresponds to one second representative probability table and one movement state rank table. In the present embodiment, a second average probability table is obtained as a second representative probability table corresponding to each second cluster by calculation by the CPU. The second average probability table is obtained by calculating an average value of probabilities for each moving state of each learning segment probability table included in the second cluster and setting the obtained average value as a probability for each moving state. In the present embodiment, the average of each learning segment probability table is used as the second representative probability table. However, the second representative probability table may be a feature parameter that represents the corresponding cluster. It can be obtained based on various conditions.

Further, the movement state rank table corresponding to each second cluster has a probability that the movement state ID corresponding to each learning segment probability table is included in all learning segment probability tables included in each second cluster, It is generated by obtaining for each moving state. Therefore, the moving state rank table includes a value obtained by determining for each moving state the probability that each moving state ID corresponding to each learning segment probability table included in the second cluster is included in the cluster, and the corresponding moving state. It is a data table which associates with ID. Then, the CPU writes the movement state rank table generated for each cluster in the flash memory. As described above, the CPU associates the cluster ID, the second representative probability table, and the movement state rank table with each of the clustered second clusters (P), and configures the second cluster group ( FIG. 11E).
Generation of Learning Feature Parameter List FIG. 3 is a flow diagram of learning feature parameter list generation processing. This is processing corresponding to step B1 in FIG. In the present embodiment, a learning power spectrum list is generated as the learning feature parameter list. The learning power spectrum list is a data list used for generating a learning model, and is generated by acquiring reference state data in a specific movement state for a certain period. Here, the learning feature parameter includes the maximum amplitude frequency of the learning power spectrum, the triaxial acceleration value vector of the learning acceleration, the square sum of the acceleration values of the learning acceleration, and the variance value of the square sum of the acceleration values. It is also possible to do.

  First, learning acceleration data (x, y, z) in a reference state is acquired from the acceleration sensor (C1). Here, x, y, and z are accelerations on three axes, but they may be accelerations on one or two axes. More specifically, the CPU flashes in advance a learning power spectrum list generation function (acceleration value calculation, time interval determination, acceleration value time window data calculation function, FF conversion function), and an acceleration sensor device driver. Read from memory and load onto heap memory. Then, the CPU activates the acceleration sensor by the device driver and starts measurement. The device driver reads the acceleration sensor measurement values (learning acceleration data) and accumulates them in the heap memory.

Next, the CPU reads the learning acceleration data from the heap memory by the acceleration value calculation function, and generates the acceleration value α = (x ² + y ² + z ² ) ^1/2 from the learning acceleration data by calculation by the CPU. (C2), storing in the heap memory.

  Then, the CPU generates the acceleration value α over a predetermined period (for example, 2 seconds) by using the acceleration value time window data calculation function, generates time window data for learning the acceleration (C3), and sends the result to the heap memory. accumulate. Here, for example, when the sampling rate is 128 Hz, the time window data for learning the acceleration consists of 256 acceleration values α in 2 seconds. If the acceleration sensor has a sampling rate of 256 Hz, the predetermined period may be 1 second.

  Next, the CPU reads out the previous learning power spectrum generation time from the heap memory by the time interval determination function, measures the elapsed time, determines whether the threshold is exceeded, and if it does not exceed the threshold, it is for learning the acceleration. The step of generating the time window data is repeated (C4). If it exceeds the threshold, the process moves to step D5.

  Next, the CPU reads a bundle of acceleration time window data (here, the obtained bundle of time window data is referred to as a time-series data list) from the heap memory by the FF conversion function, and converts each time window data into the time window data. The power spectrum for learning is calculated by Fourier transform (C5), and stored in the heap memory. Of the learning power spectrum obtained by Fourier transform of the time window data, a spectrum in a predetermined frequency band, for example, a spectrum of 1 to 10 Hz can be used. The interval at which the step of acquiring acceleration time window data is repeated may be the predetermined period, or may be longer or shorter. In this embodiment, for example, the shift width of the time window for 2 seconds is set to 0.5 seconds.

Thus, the stored learning power spectrum list is set as a learning power spectrum list.
Clustering of Learning Feature Parameter List Group FIG. 4 is a flowchart of clustering processing of the learning feature parameter list group. This process corresponds to step B2 in FIG. In this embodiment, a learning power spectrum list is used as the learning feature parameter list. Here, the learning power spectrum list is a list composed of learning power spectra in the reference state in one movement state, and the group of learning power spectrum lists is in the reference state in all movement states. It is a list group composed of a learning power spectrum list.

  By clustering all the learning power spectra included in the learning power spectrum list group, a plurality of learning power spectra to be classified are classified so that similar learning power spectra are in the same subset. . A representative clustering method is a k-means method. The k-means method is a non-hierarchical method that performs clustering by selecting K random individuals from the input data as the center of the initial cluster, and thereafter repeating the step of moving the center of gravity of the cluster. Here, for example, clustering is performed using the k-means method. Note that SVM or the like can be used instead of the K-means method. SVM (Support Vector Machine) is a kind of learning machine that classifies two classes. Among the given training points, SVM (Support Vector Machine) maximizes the margin that is the distance between the training point located near the class boundary called the support vector and the identification surface. Classify and classify the hyperplane so that

  First, the CPU reads the clustering function in the model creation function from the flash memory and expands it into the heap memory. Then, all the learning power spectra included in the learning power spectrum list group are randomly classified into two clusters (D1). The classification result is stored in a heap memory. The “all learning power spectra” are all the learning power spectra included in the learning power spectrum list for all reference states at the beginning. A set of all learning power spectra included in the learning power spectrum list for all the initial reference states is set as an “initial population” of the learning power spectrum. In step D1, a set of all learning power spectra that will be classified into each cluster from now on is defined as a “population”. In this embodiment, the data is classified into two clusters. This is because if the classification is made into three or more clusters, the power spectrum for learning having the most characteristic in one group is concentrated, and the characteristics of the power spectrum for learning in the other group are not taken into consideration. This is because it may be difficult to estimate. In addition, when only the most characteristic moving state is estimated, it is also effective to classify into three or more clusters.

  Next, an average power spectrum is obtained from all learning power spectra in the cluster classified in step D1, and the obtained average power spectrum is set as a representative power spectrum (D2). More specifically, an average value of amplitudes for each frequency of each learning power spectrum is obtained, and an average power spectrum having the obtained average value for each frequency is set as a representative power spectrum. In the present embodiment, the average power spectrum is used as the representative power spectrum, but the representative power spectrum may be any characteristic parameter that represents the corresponding cluster, and is obtained based on various conditions. be able to.

  Next, all learning power spectra in the population are classified into representative power spectra having a short distance (D3). More specifically, for example, the sum of amplitude errors for each frequency between one learning power spectrum and the representative power spectrum of each cluster is calculated. Then, one learning power spectrum is classified into the cluster corresponding to the representative power spectrum having the smaller calculated sum. Therefore, each power spectrum for learning is classified into one cluster having a representative power spectrum having a smaller difference in the sum of errors for each frequency, that is, a closer distance.

  Next, based on the classification result in step D3, the average power spectrum is calculated from all the learning power spectra included in each cluster, and the calculated result is updated as the representative power spectrum of each cluster (D4). . More specifically, the average value of the magnitudes of the learning power spectra is calculated for each frequency based on all the learning power spectra included in the classified cluster. Then, a new power spectrum having the calculated average value for each frequency is updated as a new representative power spectrum for each cluster and stored in the heap memory.

  Next, it is determined whether or not the result of classifying the learning power spectrum into each cluster in step D3 is equivalent to the previous classification result (D5). In the present embodiment, for example, it is determined whether or not the average value for each frequency of the classified learning power spectrum included in each cluster is the same as the average value for each frequency of the learning power spectrum classified immediately before. . Here, the error between the average value of the classified learning power spectrum included in each cluster and the average value of the learning power spectrum classified immediately before for each frequency is calculated, and the sum of the errors is a threshold value. You may determine whether it is below. When the determination is made based on the threshold, the classification result converges earlier, so that the processing speed is improved. If the classified result is not equivalent to the previous classification result, the processes of steps D3 and D4 are repeated. If the classified result is equivalent to the previous classification result, it is compared to which cluster the number of learning power spectra included in each cluster is larger or smaller (D6).

  When the number of classified clusters is small, a probability table is generated based on the learning power spectrum belonging to the clusters (D7). Then, the generation result is stored in the heap memory. More specifically, the probability table obtains, for each moving state, a probability that the ID of the moving state corresponding to each learning power spectrum is included in all the learning power spectra included in the cluster, and the obtained movement. It is generated by associating the probability that the ID for each state is included with the corresponding movement state. That is, the probability table is a data list in which a value obtained for each movement state is obtained for each movement state ID corresponding to each learning power spectrum, and a corresponding movement state ID. It is. Then, the CPU removes all learning power spectra belonging to the small number of clusters from the population (D8) and stores them in the heap memory. A population excluding all learning power spectra belonging to the small number of clusters becomes the next population. Next, it is determined whether the number of learning power spectra included in the small number of clusters is equal to or less than a certain percentage of the initial population (population size determination), or whether further clustering is impossible. (D8). Here, the fixed ratio is a ratio between the number of learning power spectra included in the initial population and the number of learning power spectra included in the classified cluster, and is defined in advance. Further, the case where further clustering is impossible refers to a state where all learning power spectra are mapped to one cluster. Here, in addition to the determination of the size of the population and whether or not further clustering is possible, the determination can also be made based on, for example, the deviation of the distribution of the movement state. The determination based on the deviation of the distribution of the moving state is to calculate the probability that each moving state appears using all the learning power spectra belonging to the small number of clusters, and the probability of the moving state corresponding to the highest probability is calculated. It means that it is determined whether or not the probability of the moving state corresponding to the second-order probability is twice or more. Here, when the number is more than double, the clustering process of the cluster is completed, and one first model is generated. If it is less than or equal to double, the process returns to step D1 and the clustering process is continued.

On the other hand, when the number of classified clusters is large, it is determined whether the number of learning power spectra included in the clusters is equal to or less than a certain percentage of the initial population, or further clustering is impossible ( D9). If the result of determination in step D9 is negative, the processing from steps D1 to D6 is further repeated. This means that since the number of learning power spectra included in the cluster is large, classification of the learning power spectrum is necessary by further clustering processing. On the other hand, if the result of the determination in step D9 is affirmative, it means that the classification of the learning power spectrum has converged by clustering, and the cluster has already been subdivided into an appropriate size by clustering. Then, a probability table corresponding to the cluster is generated, and the clustering process of the learning power spectrum list is completed. By clustering, all learning power spectra are classified so as to be included in one cluster (here, the first cluster), and one cluster is configured to correspond to one probability table.
Clustering of Learning Segment Probability Table List FIG. 5 is a flowchart of clustering processing of the learning segment probability table list. This process corresponds to step B10 in FIG. By calculating a learning segment probability table in all sections of the learning symbol string in step B9, a learning segment probability table list including a plurality of learning segment probability tables is generated. By clustering the learning segment probability tables included in the learning segment probability table list, the plurality of learning segment probability tables to be classified are classified so that similar learning segment probability tables become the same subset. The

  First, the CPU reads the clustering function in the model creation function from the flash memory and expands it into the heap memory. Then, all learning segment probability tables included in the learning segment probability table list are randomly classified into two clusters (E1). The classification result is stored in a heap memory. The “all learning segment probability tables” are all learning segment probability tables generated in step B9 initially. The initial set of all learning segment probability tables, that is, the set of all learning segment probability tables generated in step B9 is defined as the “initial population” of the learning segment probability table. In step E1, a set of all learning segment probability tables that will be classified into clusters from now on is defined as a “population”.

  Next, an average probability table is obtained from the learning segment probability table in the cluster classified in step E1, and the obtained average probability table is used as a representative probability table (E2). More specifically, an average probability is obtained for each moving state of each learning segment probability table, and a representative probability table is obtained in which the obtained average probability is a representative probability for each moving state.

  Next, all the learning segment probability tables in the population are classified into representative probability tables having a short distance (E3). More specifically, for example, the sum of the error of probability for each moving state between each learning segment probability table and the representative probability table of each cluster is calculated. Then, one learning segment probability table is classified into the cluster corresponding to the representative probability table having the smaller calculated sum. Therefore, each learning segment probability table is classified into one cluster having a representative probability table having a smaller difference in the sum of the errors of the probabilities for each movement state, that is, a closer probability table.

  Next, based on the classification result in step E3, an average probability table obtained by averaging them from all the learning segment probability tables included in each cluster is calculated, and the calculated result is updated as a representative probability table of each cluster. (E4). More specifically, an average value of probabilities is calculated for each movement state based on all learning segment probability tables included in the classified cluster. Then, a new probability table having the calculated average value for each movement state is updated as a new representative probability table for each cluster and stored in the heap memory.

  Next, it is determined whether the result of classifying the learning segment probability table into each cluster in step E3 is equivalent to the previous classification result (E5). In this embodiment, for example, whether the average probability for each moving state of the classified learning segment probability table included in each cluster is the same as the average probability for each moving state of the learning segment probability table classified immediately before Judge whether or not. Here, an error between the average probability for each moving state of the classified learning segment probability table included in each cluster and the average probability for each moving state of the learning segment probability table classified immediately before is calculated, It may be determined whether the sum of errors is equal to or less than a threshold value. When the determination is made based on the threshold value, the classification result converges earlier, so that the processing speed is improved. If the classified result is not equivalent to the previous classification result, the processes of steps E3 and E4 are repeated. If the classified result is equivalent to the previous classification result, it is compared to which cluster the number of learning segment probability tables included in each cluster is larger or smaller (E6).

  When the number of the classified clusters is small, a moving state rank table of groups consisting of learning segment probability tables belonging to the clusters is generated (E7). Then, the generation result is stored in the heap memory. More specifically, the moving state rank table is based on all learning segment probability tables included in each second cluster, and the probability that the moving state ID corresponding to each learning segment probability table is included for each moving state. It is generated by asking for. That is, the moving state rank table includes a value obtained by determining for each moving state the probability that each moving state ID corresponding to all learning segment probability tables included in the cluster is included in the cluster, and the corresponding moving state. This is a data list in which the IDs are associated with each other. Then, the CPU removes all learning segment probability tables belonging to the small number of clusters from the population (E7) and stores them in the heap memory. The population excluding all learning segment probability tables belonging to the small number of clusters becomes the next population. Next, it is determined whether the number of learning segment probability tables included in the small number of clusters is equal to or less than a certain percentage of the initial population (population size determination), or whether further clustering is impossible. (E8). Here, the fixed ratio is a ratio between the number of learning segment probability tables included in the initial population and the number of learning segment probability tables included in the classified cluster, and is defined in advance. Further, the case where further clustering is impossible means a state where all learning segment probability tables are mapped to one cluster. Here, in addition to the determination of the size of the population and whether or not further clustering is possible, the determination can also be made based on, for example, the deviation of the distribution of the movement state. The determination based on the deviation of the distribution of the moving state is to calculate the probability that each moving state appears using all the learning segment probability tables belonging to the small number of clusters, and the probability of the moving state corresponding to the highest probability. Is to determine whether or not is equal to or more than twice the probability of the movement state corresponding to the second probability. Here, when the number is more than double, the clustering process of the cluster is completed, and one second model is generated. If it is less than or equal to double, the process returns to step E1 to continue the clustering process.

On the other hand, when the number of classified clusters is large, it is determined whether the number of learning segment probability tables included in the clusters is equal to or less than a certain percentage of the initial population, or further clustering is impossible. (E9). If the result of determination in step E9 is negative, the processing from steps E1 to E6 is further repeated. This means that since the number of learning segment probability tables included in the cluster is large, it is necessary to classify the learning segment probability table by further clustering processing. On the other hand, if the determination result in step E9 is affirmative, the classification of the learning segment probability table has converged by clustering, and the learning segment probability table has already been subdivided into an appropriate size by clustering. Means that. Then, a moving state rank table corresponding to the cluster is generated, and the clustering process of the learning segment probability table is completed. By clustering, all learning segment probability tables are classified so as to be included in one cluster (here, the second cluster), and one moving state rank table is configured to correspond to one cluster.
Estimation of User Movement State FIG. 6 is a flowchart for estimating a user movement state. The user's movement state is estimated using a model generated using the learning power spectrum, that is, the first model and / or the second model.

  In one embodiment, it is possible to estimate the user's movement state by using only the first model. By using only the first model, the processing load for estimating the movement state of the user is reduced.

  In yet another embodiment, the movement state can be estimated by using only the second model. By using only the second model, the processing load for estimating the movement state of the user is reduced.

In yet another embodiment, it is possible to estimate the movement state of the user by combining the first model and the second model, thereby further reducing the movement state estimation error. Specifically, the movement state estimation result (first estimation result) obtained by using the first model is the same as the movement state estimation result (second estimation result) obtained by using the second model. If this is the case, the moving state is selected and this is used as the estimation result. As a result, the reliability of the result estimated by the first model can be evaluated using the second model, and a highly reliable estimation result can be obtained.
Acquisition of Estimation Result Using First Model According to the flowchart of FIG. 6, the movement state estimation method will be described in detail below. First, the acceleration data is acquired from the acceleration sensor provided in the mobile terminal device of the user that estimates the moving state, and the function consisting of the acceleration is estimated by decomposing it into a continuous spectrum of its frequency components (for example, Fourier transform). A power spectrum list is generated (F1). In the estimation power spectrum list, the ID of each estimation power spectrum and the amplitude for each frequency corresponding to each estimation power spectrum (Q) are stored in time series (FIG. 11F).

  Next, using the first model, a first cluster corresponding to one estimation power spectrum and the nearest first average power spectrum in the estimation power spectrum list is selected (F2). More specifically, for example, the CPU calculates a sum of amplitude errors for each frequency between each of the first average power spectrum and the estimation power spectrum in each first cluster of the cluster group. Then, the first cluster corresponding to the first average power spectrum having the smallest calculated sum is selected as the estimation average power spectrum and the nearest first average power spectrum. Note that the nearest first average power spectrum may be obtained by selecting the first average power spectrum that minimizes the sum of the squares of the probabilities of each moving state.

  Next, the symbol corresponding to the selected first nearest cluster is stored (symbolized) in the estimation symbol string (F3). Each first cluster corresponds to one first average power spectrum and one probability table, and one different symbol is assigned to each first cluster.

  Next, it is determined whether or not all the estimation power spectra in the fixed section in the estimation power spectrum list have been symbolized (F4). Here, the predetermined section is a predetermined section that is determined in advance and corresponds to the minimum number (b) of estimation power spectra necessary for estimating the movement state. If this fixed section is too short, the estimation error may be increased due to a large influence of sudden noise. If this section is too long, it may not be possible to grasp changes in the movement state that may occur within the period in a timely manner. If the estimation power spectrum in a certain section is not all symbolized, steps F2 and F3 are repeated until all symbolization is completed. When all the estimation power spectra in the fixed section are symbolized, the process proceeds to step F5.

  Next, an estimation segment probability table is generated based on the generated estimation symbol string (FIG. 11G) (F5). The estimation segment probability table is a probability table obtained based on each probability table corresponding to consecutive symbols in a certain section in the estimation symbol string. The estimation segment probability table is obtained by calculating an average value of probabilities for each movement state using the probability table of the first cluster corresponding to each of a plurality of symbols stored in a predetermined section of the estimation symbol string. It is done. The estimation segment probability table is a table having an average probability for each fixed section for each movement state calculated for each movement state.

  Next, the CPU refers to the estimation segment probability table, and selects the movement state corresponding to the highest probability among the average probabilities for each movement state (acquisition of movement state estimation result by the first model) (F6). . In the present embodiment, the movement state corresponding to the highest probability is selected, but other predetermined conditions can be set for selection. For example, the portable terminal device may be held in a pocket or bag, held in a hand, or worn on an arm or the like. When estimating these possession states, the possession states can be estimated by selecting states corresponding to a plurality of probabilities from the top of the estimation segment probability table, for example. Furthermore, in the present embodiment, the movement state is selected, but a person holding a mobile terminal device can also be specified using a similar method, for example. Further, for example, by selecting a state corresponding to a plurality of probabilities from the top, it is possible to identify a plurality of highly likely persons.

Then, the CPU stores the ID corresponding to the selected movement state in the memory as an estimation result. The CPU refers to the estimation segment probability table using the estimation power spectrum stored in the memory in time series, and sequentially selects the movement state corresponding to the highest probability. Then, the estimation result of the movement state of the user by the first model is stored in the memory in time series (first model estimation result in FIG. 11H). As described above, the movement state of the user is estimated by the first model using the time series data. Even when a sudden change is included in the user movement state or when a time zone similar to another state occurs, the movement state of the user can be correctly estimated. Moreover, the information of one power spectrum obtained from a user's acceleration sensor may have the characteristic corresponding to a 1 or several movement state. In the present invention, even if the feature parameter obtained from one acceleration data is similar to the feature parameter in a plurality of moving states, the first model is used to move from the frequency at which the feature parameter appears in each moving state. The state can be estimated. Therefore, even when the feature parameters are similar to those in a plurality of movement states, the movement state of the user can be estimated with high accuracy.
Generation of Estimation Power Spectrum List Generation of estimation power spectrum (step F1) will be described in detail below. The estimation power spectrum list used for estimating the movement state of the user in a certain movement state is performed by the same process as the selection of the learning power spectrum list. More specifically, acceleration data for estimation is acquired over a predetermined period from an acceleration sensor possessed by a user in a certain moving state, and acceleration time window data is generated from the obtained acceleration data for estimation.

Then, the generation of the estimation acceleration time window data is repeated a predetermined number of times to obtain a time series data list (estimation data list) of the estimation acceleration. Note that the generation of the estimation acceleration time window data is repeated a predetermined number of times until the number of data necessary for estimating the user's movement state is obtained. Then, the time window data of the plurality of estimation accelerations are each subjected to Fourier transform by calculation by the CPU, and an estimation power spectrum list including a plurality of estimation power spectra is calculated.
Acquisition of Estimation Result Using Second Model Next, the CPU selects a second cluster having an estimation segment probability table and the nearest second representative probability table using the second model (F7). In the present embodiment, the second average probability table is used as the second representative probability table. More specifically, the second average probability table is moved using, for example, one or a plurality of learning segment probability tables corresponding to each second cluster in each cluster included in the cluster group constituting the second model. It is generated for each cluster by obtaining an average probability for each state. The second average probability table is a data table in which the average probability for each moving state is associated with the ID of the corresponding moving state. Here, the estimation segment probability table and the nearest second average probability table are, for example, one second average probability table corresponding to each cluster constituting the second model, and an estimation segment probability table. The difference between the probabilities for each movement state is obtained, and the second average probability table having the smallest sum of the differences is selected. Note that the nearest second average probability table may be obtained by selecting a probability table that minimizes the square of the sum of differences for each movement state.

  Next, the CPU selects the movement state of the user based on the second model by selecting the movement state ID corresponding to the highest probability based on the movement state rank table corresponding to the selected second cluster ( (Acquisition result of movement state by second model) (F8). Then, the CPU stores the ID of the selected moving state in the memory as an estimation result. Here, one second average probability table and one moving state rank table correspond to the second cluster constituting the second model (FIG. 11E). The moving state rank table is a value obtained by determining, for each moving state, the probability that the ID of each moving state corresponding to each learning segment probability table is included in the cluster in all learning segment probability tables included in the second cluster. And a data table that associates the ID of the movement state corresponding to the ID.

The CPU selects a cluster corresponding to the estimation segment probability table and the nearest second average probability table using the generated estimation segment probability table, refers to the movement state rank table corresponding to the cluster, The movement state corresponding to the highest probability is selected sequentially. Then, the estimation result of the movement state of the user by the second model is stored in the memory in time series (FIG. 11H). As described above, the movement state of the user is estimated by the second model using the time series data. Therefore, in this embodiment, since the user's movement state is estimated using time-series data indicating the user's movement state, the user movement state includes a sudden change or a time zone similar to other states. Even if this occurs, it is possible to correctly estimate the user's movement state. Further, even when the characteristics of the obtained probability table are similar to the characteristics of the probability table in other movement states, the movement state can be estimated with high accuracy.
Determination of Key Frame Next, it is determined whether or not the first model estimation result and the second model estimation result stored in the memory are in the same moving state (F9). More specifically, in step F9, it is determined whether or not the ID of the movement state selected by the first model in the certain section is the same as the ID of the movement state selected by the second model. If the determination results in step F9 are not the same, it is determined that the movement state of the user selected by the first model and the second model is not reliable, and one frame corresponding to this certain section is determined as an unconfirmed frame ( (unknown) is stored in one frame of the frame list (F11). Further, the processing from step F1 to step F9 is repeated.

  On the other hand, if the result determined in step F9 is the same movement state, the estimation result obtained by the first model and the second model is considered to be highly reliable, and one frame corresponding to this certain section is selected as a key. A frame (known) is stored in one frame of the frame list (F10). The frame list is a list composed of a plurality of frames in which one frame corresponds to one fixed section. Each frame stores in time series whether the frame is a key frame or an indeterminate frame.

From the above, as shown in FIG. 6, when estimation of the movement state using the first model and estimation of the movement state using the second model are performed, when the estimation results of both models match, By estimating the movement state assuming that the estimation accuracy is high, the estimation accuracy can be increased.
Number of steps, moving state estimating method diagram using base station information 7 is a diagram showing the system configuration of the moving state estimating method using the external information acquisition device. FIG. 7 is a configuration including an antenna (G60) and a pedometer (G70) as an external information acquisition device in addition to the configuration of FIG. The mobile terminal device acquires the latitude and longitude information of the base station closest to the mobile terminal device by communicating with the base station through the antenna (G60). The mobile terminal device can also acquire base station ID information as base station information. The base station information can also be acquired from a CDMA base station, a base station such as a wireless tag, a wireless LAN, or the like. In this embodiment, by further using the information obtained by using the external information acquisition device, it is possible to further improve the estimation accuracy of the moving state and reduce the processing load.

  The latitude and longitude information of the base station is directly stored in the heap memory. Also, the pedometer (G70) operates when vibration (acceleration) of a certain level or more is applied, and counts the number of steps of the user. A pedometer mounted on a portable terminal device is generally implemented in hardware, and can operate with less power than an acceleration sensor. In this embodiment, when the estimated movement state obtained is highly reliable, it is determined whether or not the movement state has changed using only a pedometer with low power consumption. Further, when the estimated movement state obtained is a stop state or an electric vehicle state, that is, an automobile state, a bus state, or a train state, it is detected only that the number of steps does not change. By detecting only the rhythm, it is possible to determine the change in the movement state. Here, the detection of the rhythm of the number of steps is performed by detecting the number of steps counted in a certain time. Thus, when there is a change in the moving state, the processing load for estimating the moving state can be reduced by appropriately selecting a device that acquires user information.

  Here, the number of steps acquired from the pedometer and the base station information acquired from the antenna can be acquired by accessing the memory without any operation by the CPU (by low-load processing). Therefore, it is possible to estimate the moving state with lower power consumption than to estimate the moving state by always operating the acceleration sensor. In addition, the acceleration sensor can be operated at all times to reduce the load on the CPU, acquire external information at high speed, and operate with low power consumption. State estimation is possible. Information from a global positioning system (GPS) base station can also be used to acquire the position of the mobile terminal device.

  FIG. 8 is a process flow diagram for estimating the movement state using information acquired from the external information acquisition device in addition to the estimation using the first and second models. According to this embodiment, it is possible to evaluate the validity of the estimation result of the key frame interval selected based on the estimation result obtained by using the first model and the second model. Thereby, the precision of movement state estimation can be improved further.

  The estimation method of the moving state according to FIG. 8 uses the step information obtained from the pedometer G70 and the base station information obtained from the antenna G60, which the user has, after estimating using the first and second models. 1. The validity of the movement state estimated by the second model is evaluated. The base station information obtained from the antenna includes, for example, latitude and longitude information of the base station closest to the mobile terminal device possessed by the user. In this estimation method, after the estimation process shown in FIG. 6, the movement state is estimated according to the estimation process flow shown in FIG.

  First, it is determined whether or not the movement state corresponding to the current key frame is the same as the movement state corresponding to the key frame registered immediately before (H1). The movement state ID corresponding to the current key frame is compared with the movement state ID corresponding to the key frame registered immediately before. If they are not the same, the obtained current key frame and In the registered key frame, since the reliability of the selected moving state is considered to be low, the process returns to Step F1 again to start the estimation process.

  If the movement state corresponding to the current key frame is the same as the movement state corresponding to the key frame registered immediately before, the process proceeds to step H2. In step H2, it is determined whether the movement state corresponding to the key frame is the first, second, or third movement state (H2). Here, the first movement state refers to, for example, a running state, a walking state, and a bicycle state. Further, the second moving state refers to, for example, a stopped state. The third movement state refers to, for example, a train state, a car state, and a bus state. All movement states are classified in advance into first, second and third movement states in relation to the number of steps and base station information. In this embodiment, the user's movement state estimated from information acquired by an external information acquisition device, for example, a pedometer and / or an antenna, corresponds to the estimated movement state using the first and second models. Determine whether or not. Here, the CPU acquires the number of steps and the base station information from the flash memory, stores it in the heap memory, and determines (H2). Then, the determination result is stored in the heap memory.

  If the movement state corresponding to the key frame is the first movement state, is the number of steps obtained from the pedometer increasing in all frames from the key frame registered immediately before to the current key frame? It is determined whether or not (H3). The user's step count information is always measured by the pedometer, and the step count acquired from the pedometer is stored in the heap memory in time series for each frame configured in step F10. The CPU acquires the step count information stored in the memory for each frame of a certain section, and the CPU determines whether or not the step count is increased in all frames in the obtained step count information. In the bicycle state, the number of steps may not be increased in all frames, and it can be determined by whether or not the number of steps is increased in one or more frames. This is because the number of steps may not increase in all frames, such as when the estimation target user is in a bicycle state, or when the user is going downhill without stroking the pedal.

  When the number of steps is increased in all frames, it is determined that the movement state corresponding to the key frame, that is, the first movement state is correct. That is, since the increase in the number of steps corresponds to a state in which the user is moving, it corresponds to the first movement state including running, walking, and a bicycle state. Therefore, it is recognized that the results estimated by the first model and the second model have high reliability. Then, the section from the key frame registered immediately before to the current key frame is determined as the key frame section (H6). On the other hand, if the number of steps has not increased as a result of the determination, the estimation process is performed again (F1), assuming that the moving state obtained as an estimation result by the model has low reliability.

  When the movement state corresponding to the key frame is the second movement state, it is determined whether there is no change in the number of steps and the base station in all frames from the key frame registered immediately before to the current key frame. (H4). More specifically, the step count information and the base station information corresponding to all the frames up to the current key frame are stored in the memory, respectively. Every time, the step count information and the base station information stored in the memory are acquired. Based on the obtained step count information and base station information, the CPU determines whether or not the number of steps has increased in all frames or whether the base station has changed. The state in which the number of steps is not increased and the base station is not changed corresponds to the state in which the user is almost stopped, and therefore corresponds to the second movement state. Therefore, it is recognized that the results estimated by the first model and the second model have high reliability. Then, the section from the key frame registered immediately before to the current key frame is determined as the key frame section (H6). As a result of the determination, if the number of steps is increased or if there is a change in the base station, the movement state obtained as an estimation result by the model is assumed to be low in reliability and the estimation process is performed again (F1).

  If the movement state corresponding to the key frame is the third movement state, it is determined whether there is no change in the number of steps in all frames between the key frame registered immediately before and the current key frame (H5). . More specifically, the step count information corresponding to all the frames up to the current key frame is stored in the memory, and the CPU stores it in the memory for each frame in a certain section configured in step F10. Get step count information. Then, the CPU determines whether or not the number of steps has increased in all frames based on the obtained number of steps information. If the number of steps has not increased, but the base station has changed or has not changed, the user himself / herself is not moving, but the vehicle on which the user is on is in a state where the vehicle is moving / stopped. , Corresponding to a third movement state that is an automobile state and a bus state. Then, the section from the key frame registered immediately before to the current key frame is determined as the key frame section (H6). If the number of steps is increased as a result of the determination, the moving state obtained as an estimation result by the model is assumed to be low in reliability and the estimation process is performed again (F1).

  Next, all frames in the key frame section are associated with the movement state corresponding to the key frame (FIG. 11H). Thereby, CPU confirms the estimation result of a movement state (H7). If the estimation result is the second movement state, that is, the stop state here, the acceleration data acquisition is stopped, and only the step count and base station information are acquired (transition to the step count / base station information monitoring mode). ). If the estimation result is the third movement state, that is, the train state, the car state, or the bus state here, the acquisition of acceleration data is stopped and only the number of steps is acquired (transition to the step number monitoring mode). In this way, it is possible to save power by stopping the acquisition of acceleration data (transition to the low power consumption mode). On the other hand, when the estimation result is the first movement state, that is, here, in the running state, the walking state, and the bicycle state, the acquisition of acceleration data is continued. In the case of the second movement state and the third movement state, the second movement state and the third movement state are maintained by acquiring only the number of steps and / or base station information without using the acceleration data. In the case of the first movement state, it is difficult to grasp whether the first movement state is maintained only with the number of steps and the base station information. Because there are cases.

  FIG. 9 shows a process flow for a further embodiment. The processing flow shown in FIG. 9 is similar to the processing flow shown in FIG. 8, but in a further embodiment, the movement state corresponding to the current key frame is the movement state corresponding to the previous key frame. If the determination results are not the same, it is further determined whether the movement state corresponding to the current key frame is the third movement state. In the present embodiment, when the movement state corresponding to the current key frame is the third movement state, even if there is a key frame corresponding to the second movement state in the key frame section, the third movement is performed. The key frame section is determined as the state. For example, when the vehicle frequently stops due to traffic jams or the like in the third moving state such as an automobile state or a bus state, the moving state corresponding to the previous key frame and the moving state corresponding to the current key frame This is to avoid that the key frame section cannot be determined due to the difference between and.

More specifically, it is first determined whether or not the movement state corresponding to the current key frame is the same as the movement state corresponding to the immediately preceding key frame (I1). If the determination results are the same, it is determined whether this movement state is the first, second, or third movement state (I2). The processes from I3 to I7 are the same as the corresponding processes (from H3 to H7) in FIG. On the other hand, if the determination results are not the same, it is determined whether the movement state corresponding to the immediately preceding key frame is the second movement state and the movement state corresponding to the current key frame is the third movement state. (H8). Here, it can be similarly determined whether the movement state corresponding to the immediately preceding key frame is the third movement state and the movement state corresponding to the current key frame is the second movement state. If the result of determination by H8 is affirmative, the section from the previous key frame to the current key frame is determined as the key frame section corresponding to the third movement state (H6). Here, also when the previous movement state is the third and the current movement state is the second, the section up to the current key frame is determined as the key frame section corresponding to the third movement state. As described above, even when the vehicle is frequently stopped due to traffic jams or the like in the automobile state or the bus state which is the third movement state, the key frame section is determined as being in the third movement state, and more The key frame section can be determined quickly.
Key Frame Section Canceling Process FIG. 10 shows a flow of a key frame section canceling process. Once the key frame interval is set, it is not canceled until the next key frame is detected or the number of steps and / or base station information has changed.

  After the movement state estimation processing (F1 to F10 and H1 to H7) is performed, the key frame section cancellation processing according to FIG. 10 is performed. After the movement state is estimated in step H7 or I7, if the estimation result is the second movement state, that is, the stop state here, the acceleration data acquisition is stopped, and only the step count and base station information are obtained. get. When the estimation result is the third movement state, that is, here, the train state, the car state, and the bus state, the acquisition of acceleration data is stopped and only the number of steps is acquired. When the estimation result is the first movement state, that is, here, in the running state, the walking state, and the bicycle state, acceleration data is acquired (F1).

  Next, the CPU determines whether the movement state corresponding to the key frame in the key frame section is the first, second, or third movement state (J1). When the movement state corresponding to the key frame is the first movement state, a new key frame is detected by calculation by the CPU (J2). That is, the processes from Step F1 to F10 are repeated until a new key frame is detected. Next, it is determined whether or not the movement state corresponding to the detected key frame is the same as the movement state corresponding to the immediately preceding key frame (J3). If the result of determination is that they are not in the same movement state, the key frame section is canceled (J4). That is, all the frames corresponding to the frame section from the frame next to the immediately preceding key frame to the detected key frame are determined as unconfirmed frames. Here, the release of the key frame section refers to a process of returning the key frame (known) to the unconfirmed frame (unknown). Then, the next frame section estimation process (F1 to F10) is performed (J5). On the other hand, if the result of determination is that the movement states are the same, estimation processing (F1 to F10) for the next frame section is performed (J5).

  Next, when the movement state corresponding to the key frame is the second movement state, the CPU determines whether or not there is a change in the number of steps or the base station information (J6). If there is a change in the number of steps or the base station information, the key frame section is canceled (J4). That is, all the frames corresponding to the frame section from the frame next to the immediately preceding key frame to the detected key frame are determined as unconfirmed frames. Then, an estimation process for the next frame section is performed (J5). On the other hand, if there is no change in the number of steps or the base station information as a result of the determination, an estimation process (F1 to F10) for the next frame section is performed (J5).

  Next, when the movement state corresponding to the key frame is the third movement state, the CPU determines whether or not there is a change in the number of steps (J7). If there is a change in the number of steps, the key frame section is canceled (J4). That is, all the frames corresponding to the frame section from the frame next to the immediately preceding key frame to the detected key frame are determined as unconfirmed frames. Then, an estimation process for the next frame section is performed (J5). On the other hand, as a result of the determination, if there is no change in the number of steps, estimation processing (F1 to F10) for the next frame section is performed (J5).

  As described above, the key frame section is released by a different method depending on the type of movement state corresponding to the key frame. As described above, by using the key frame section, it is possible to estimate the moving state in the low power consumption mode and detect the change of the moving state until the key frame section is canceled.

Claims (18)

A method for estimating a movement state of a mobile terminal device including at least one CPU (central processing unit) and a memory, the memory being a plurality of first clusters, each representing a first cluster A plurality of first clusters corresponding to a representative feature parameter of and a first probability table of one having a probability for each moving state;
Based on the difference for each parameter between the feature parameter for estimation composed of one or more parameters stored in the memory in time series and each representative feature parameter composed of one or more parameters, Selecting a first cluster of which the difference satisfies a first condition from a plurality of first clusters;
Estimating a moving state by specifying a moving state satisfying a second condition from an estimation segment probability table calculated based on the first probability table corresponding to the selected first cluster;
A method consisting of:   2. The method according to claim 1, wherein the first probability table includes probability data for each movement state corresponding to the learning feature parameter included in each first cluster for each movement state. A method for estimating a movement state of a mobile terminal device including at least one CPU and a memory, wherein the memory is a plurality of first clusters, each of which represents one representative feature parameter representing the first cluster, , Storing a plurality of first clusters corresponding to one first probability table having a probability for each moving state,
Based on the difference for each parameter between the feature parameter for estimation composed of one or more parameters stored in the memory in time series and each representative feature parameter composed of one or more parameters, Selecting a first cluster of which the difference satisfies a first condition from a plurality of first clusters;
Calculating an estimation segment probability table in time series based on the first probability table corresponding to the selected first cluster;
A step of generating a plurality of second clusters using the estimation segment probability table, wherein each of the second representative probability table representing the second cluster and one second having a probability for each moving state Generating a plurality of second clusters corresponding to the probability table;
From the plurality of second clusters, based on the difference for each parameter between the estimation segment probability table composed of one or a plurality of parameters and each of the second representative probability tables composed of one or a plurality of parameters Selecting one second cluster for which the difference satisfies a third condition;
Estimating a moving state by selecting a moving state satisfying a fourth condition from the second probability table corresponding to the selected second cluster;
A method consisting of:   4. The first probability table according to claim 3, wherein the first probability table includes probability data for each moving state corresponding to the learning feature parameter included in each first cluster, and the second probability table includes The method which has the probability data for every movement state corresponding to the segment probability table for learning contained in a 2nd cluster for every movement state. A method for estimating a movement state of a mobile terminal device including at least one CPU and a memory, wherein the memory is a plurality of first clusters, each of which represents one representative feature parameter representing the first cluster, , Storing a plurality of first clusters corresponding to one first probability table having a probability for each moving state,
Based on the difference for each parameter between the estimation feature parameter composed of one or a plurality of parameters and each representative feature parameter composed of one or a plurality of parameters stored in the memory in time series Selecting a first cluster of which the difference satisfies a first condition from the first cluster of:
By identifying a movement state satisfying the second condition from the estimation segment probability table stored in the memory in time series calculated based on the first probability table corresponding to the selected first cluster A first estimation step for estimating a moving state;
A step of generating a plurality of second clusters using the estimation segment probability table, wherein each of the second representative probability table representing the second cluster and one second having a probability for each moving state Generating a plurality of second clusters corresponding to the probability table;
Based on the difference for each parameter between the estimation segment probability table composed of one or a plurality of parameters and each of the second representative probability tables composed of one or a plurality of parameters, the difference is calculated from a plurality of clusters. Selecting one second cluster that satisfies condition 3;
A second estimation step of estimating a movement state by selecting a movement state satisfying a fourth condition from the second probability table corresponding to the selected second cluster;
Estimating a moving state using the result estimated by the first estimation step and the second estimation step;
A method consisting of: A method for estimating a movement state of a mobile terminal device including at least one CPU and a memory, wherein the memory is a plurality of first clusters, each of which represents one representative feature parameter representing the first cluster, , Storing a plurality of first clusters corresponding to one first probability table having a probability for each moving state,
Based on the difference for each parameter between the feature parameter for estimation composed of one or more parameters stored in the memory in time series and each representative feature parameter composed of one or more parameters, Selecting a first cluster of which the difference satisfies a first condition from a plurality of first clusters;
A first estimation of a movement state by specifying a movement state satisfying a second condition from an estimation segment probability table calculated based on the first probability table corresponding to the selected first cluster. An estimation step;
A step of generating a plurality of second clusters using the estimation segment probability table, wherein each of the second representative probability table representing the second cluster and one second having a probability for each moving state Generating a plurality of second clusters corresponding to the probability table;
From the plurality of second clusters, based on the difference for each parameter between the estimation segment probability table composed of one or a plurality of parameters and each of the second representative probability tables composed of one or a plurality of parameters Selecting one second cluster for which the difference satisfies a third condition;
A second estimation step of estimating a movement state by selecting a movement state satisfying a fourth condition from the second probability table corresponding to the selected second cluster;
When the result estimated by the first estimation step and the second estimation step satisfies a fifth condition, a frame corresponding to the result obtained by the fifth condition is used as a key frame. Generating a frame list by sequentially storing in the list;
In the generated frame list, when one key frame and the immediately preceding key frame correspond to the same movement state,
A step of determining whether a predetermined condition is satisfied between the one key frame and the immediately preceding one key frame when the movement state corresponding to the one key frame is a predetermined movement state; When,
A step of estimating a moving state by determining a moving state corresponding to the one key frame from the immediately preceding one key frame as a moving state corresponding to the key frame when the predetermined condition is satisfied; When,
A method consisting of: A method for generating a model used for estimating a movement state of a mobile terminal device including at least one CPU and a memory, the CPU comprising:
A plurality of learning feature parameters corresponding to all the movement states stored in the memory in time series are clustered into a plurality of first clusters, and each of the first clusters is composed of one or more learning feature parameters. Generating a group of clusters;
In the first cluster group, a first cluster corresponding to one representative feature parameter representing each first cluster for each first cluster and a first probability table having a probability that each moving state will appear. Generating step;
Including methods. A method for generating first and second models used for estimating a movement state of a mobile terminal device including at least one CPU and a memory, the CPU comprising:
A plurality of learning feature parameters corresponding to all the movement states stored in the memory in time series are clustered into a plurality of first clusters, and each of the first clusters is composed of one or more learning feature parameters. Generating a group of clusters;
In the first cluster group, for each first cluster, one representative feature parameter representing each first cluster and a first probability table having a probability that each moving state appears corresponds to the first cluster. Generating step;
Sequentially selecting one first cluster for which the difference satisfies a predetermined condition based on the difference between the learning feature parameter and the representative feature parameter for each learning feature parameter;
Calculating a learning segment probability table based on each first probability table corresponding to each first cluster for each predetermined number of first clusters selected in sequence;
Clustering each of the calculated learning segment probability tables into a plurality of second clusters, each of the second clusters generating a group of second clusters comprising one or more of the learning segment probability tables;
In the second cluster group, for each second cluster, one second representative probability table representing each second cluster and a second probability table representing the probability that each moving state appears correspond to the second cluster. Generating two clusters;
Including methods. A method for detecting a change in a moving state of a mobile terminal device including at least one CPU, a pedometer, an antenna, and an acceleration sensor, wherein the mobile terminal device receives first information by the pedometer, The second information can be acquired by the antenna, and the third information can be acquired by the acceleration sensor.
When the estimated movement state is the first movement state, only the first information is acquired, and when the first information corresponds to the first movement state, the estimated first state Obtaining a movement state of 1 as an estimation result, and monitoring whether there is a change in the third information;
When the estimated movement state is the second movement state, the first information and the second information are acquired, and the first information and the second information correspond to the second movement state. If there is, the step of acquiring the estimated second movement state as an estimation result and monitoring whether there is a change in the first information and the second information;
When the estimated movement state is the third movement state, the first information is acquired, and when the first information corresponds to the third movement state, the estimated first state is obtained. Obtaining a movement state of 3 as an estimation result, and monitoring whether there is a change in the first information;
A method for detecting a change in a moving state. A mobile terminal device that estimates the movement state of the mobile terminal device,
A memory for storing a plurality of first clusters, wherein each first cluster includes one representative feature parameter representing each first cluster and one first probability table having a probability for each moving state. Corresponding memory,
A CPU, wherein the CPU
Based on the difference for each parameter between the feature parameter for estimation composed of one or more parameters stored in the memory in time series and each representative feature parameter composed of one or more parameters, Selecting a first cluster of which the difference satisfies the first condition from a plurality of first clusters;
From the estimation segment probability table calculated based on the probability table corresponding to the selected first cluster, the movement state is estimated by specifying the movement state satisfying the second condition;
Mobile terminal device.   11. The mobile terminal device according to claim 10, wherein the first probability table includes probability data for each moving state corresponding to the learning feature parameter included in each first cluster for each moving state. A mobile terminal device that estimates the movement state of the mobile terminal device,
A memory for storing a plurality of first clusters, wherein each first cluster includes one representative feature parameter representing each first cluster and one first probability table having a probability for each moving state. Corresponding memory,
A CPU, wherein the CPU
Based on the difference for each parameter between the feature parameter for estimation composed of one or more parameters stored in the memory in time series and each representative feature parameter composed of one or more parameters, Selecting a first cluster of which the difference satisfies the first condition from a plurality of first clusters;
Calculating an estimation segment probability table in time series based on the first probability table corresponding to the selected first cluster;
A step of generating a plurality of second clusters using the estimation segment probability table, wherein each of the second representative probability table representing the second cluster and one second having a probability for each moving state Generating a plurality of second clusters corresponding to the probability table;
From the plurality of second clusters, based on the difference for each parameter between the estimation segment probability table composed of one or a plurality of parameters and each of the second representative probability tables composed of one or a plurality of parameters Selecting one second cluster whose difference satisfies the third condition;
A movement state is estimated by selecting a movement state satisfying a fourth condition from the second probability table corresponding to the selected second cluster.
Mobile terminal device.   13. The first probability table according to claim 12, wherein the first probability table has probability data for each moving state corresponding to the learning feature parameter included in each first cluster for each moving state, and the second probability table includes The portable terminal device which has the probability data for every movement state corresponding to the segment probability table for learning contained in a 2nd cluster for every movement state. A mobile terminal device that estimates the movement state of the mobile terminal device,
A memory for storing a plurality of first clusters, wherein each first cluster includes one representative feature parameter representing each first cluster and one first probability table having a probability for each moving state. Corresponding memory,
A estimation feature parameter composed of one or a plurality of parameters and each representative feature parameter composed of one or a plurality of parameters stored in the memory in time series by the CPU. Based on the difference for each parameter, a first cluster in which the difference satisfies the first condition is selected from the plurality of first clusters,
By identifying a movement state satisfying the second condition from the estimation segment probability table stored in the memory in time series calculated based on the first probability table corresponding to the selected first cluster Make a first estimate to estimate the movement state;
A step of generating a plurality of second clusters using the estimation segment probability table, wherein each of the second representative probability table representing the second cluster and one second having a probability for each moving state Generating a plurality of second clusters corresponding to the probability table;
Based on the difference for each parameter between the estimation segment probability table composed of one or a plurality of parameters and each of the second representative probability tables composed of one or a plurality of parameters, the difference is calculated from a plurality of clusters. 1 second cluster satisfying the condition 3 is selected,
From the second probability table corresponding to the selected second cluster, a second estimation that estimates the movement state by selecting a movement state that satisfies the fourth condition,
Estimating a moving state using the result estimated by the first estimation step and the second estimation step;
Mobile terminal device. A mobile terminal device that estimates the movement state of the mobile terminal device,
A memory for storing a plurality of first clusters, wherein each first cluster includes one representative feature parameter representing each first cluster and one first probability table having a probability for each moving state. Corresponding memory,
A CPU, wherein the CPU
Based on the difference for each parameter between the feature parameter for estimation composed of one or more parameters stored in the memory in time series and each representative feature parameter composed of one or more parameters, Selecting a first cluster of which the difference satisfies the first condition from a plurality of first clusters;
A first estimation of a movement state by specifying a movement state satisfying a second condition from an estimation segment probability table calculated based on the first probability table corresponding to the selected first cluster. Make an estimate,
A step of generating a plurality of second clusters using the estimation segment probability table, wherein each of the second representative probability table representing the second cluster and one second having a probability for each moving state Generating a plurality of second clusters corresponding to the probability table;
From the plurality of second clusters, based on the difference for each parameter between the estimation segment probability table composed of one or a plurality of parameters and each of the second representative probability tables composed of one or a plurality of parameters Selecting one second cluster whose difference satisfies the third condition;
From the second probability table corresponding to the selected second cluster, a second estimation that estimates the movement state by selecting a movement state that satisfies the fourth condition,
When the result estimated by the first estimation step and the second estimation step satisfies a fifth condition, a frame corresponding to the result obtained by the fifth condition is used as a key frame. Generate a frame list by sequentially storing in the list,
In the generated frame list, when one key frame and the immediately preceding key frame correspond to the same movement state,
When the movement state corresponding to the one key frame is a predetermined movement state, it is determined whether a predetermined condition is satisfied between the one key frame and the one key frame immediately before the one key frame,
When the predetermined condition is satisfied, the moving state is estimated by determining the moving state corresponding to the one key frame from the immediately preceding one key frame as the moving state corresponding to the key frame.
Mobile terminal device. A system for generating a model used for estimating a moving state of a mobile terminal device,
A memory storing a plurality of learning feature parameters corresponding to all moving states in time series;
A CPU, the CPU clustering the plurality of learning feature parameters corresponding to all the movement states stored in the memory in time series into a plurality of first clusters, Generating a first cluster group comprising one or more of the learning feature parameters;
In the first cluster group, a first cluster corresponding to one representative feature parameter representing each first cluster for each first cluster and a first probability table having a probability that each moving state will appear. Generate,
system. A system for generating first and second models used for estimating a movement state of a mobile terminal device,
A memory storing a plurality of learning feature parameters corresponding to all moving states in time series;
A CPU, wherein the CPU
A plurality of learning feature parameters corresponding to all the movement states stored in the memory in time series are clustered into a plurality of first clusters, and each of the first clusters is composed of one or more learning feature parameters. Create a group of clusters,
In the first cluster group, for each first cluster, one representative feature parameter representing each first cluster and a first probability table having a probability that each moving state appears corresponds to the first cluster. Generate and
For each of the learning feature parameters, one first cluster in which the difference satisfies a predetermined condition is sequentially selected based on the difference between the learning feature parameter and the representative feature parameter.
A learning segment probability table is calculated based on each first probability table corresponding to each first cluster for each predetermined number of the first clusters selected in sequence.
Clustering each of the calculated learning segment probability tables into a plurality of second clusters, each of the second clusters generating a group of second clusters including one or more of the learning segment probability tables;
In the second cluster group, for each second cluster, one second representative probability table representing each second cluster and a second probability table representing the probability that each moving state appears correspond to the second cluster. Create two clusters,
system. A mobile terminal device that detects a change in a moving state of a mobile terminal device including at least one CPU, a pedometer, an antenna, and an acceleration sensor, wherein the mobile terminal device receives first information by the pedometer. The second information can be acquired by the antenna, and the third information can be acquired by the acceleration sensor.
When the estimated movement state is the first movement state, only the first information is acquired, and when the first information corresponds to the first movement state, the estimated first state 1 movement state is obtained as an estimation result, monitoring whether there is a change in the third information,
When the estimated movement state is the second movement state, the first information and the second information are acquired, and the first information and the second information correspond to the second movement state. If there is, obtain the estimated second movement state as an estimation result, and monitor whether there is a change in the first information and the second information,
When the estimated movement state is the third movement state, the first information is acquired, and when the first information corresponds to the third movement state, the estimated first state is obtained. 3 is obtained as an estimation result, and monitoring whether or not there is a change in the first information,
By this, the portable terminal device which detects the change of a movement state.

Images