JP2014003536A - Recognition device, recognition method and recognition system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow for high accuracy input operation to a terminal, without forcing special operations on the user.SOLUTION: Since a recognition device 1 provides input operation to a terminal 3 by recognizing whether or not the holding data, indicating the holding state of the terminal by a user, corresponds to a prestored holding state, any special operation is not forced on the user. Furthermore, since time fluctuation data of the holding data is calculated, and the posterior probability distribution calculated therefrom by a statistical technique is stored as the holding state information for recognizing the holding state, the recognition device is robust to the noise caused by unique circumstances of a detector for detecting the holding data, and other sporadic noises. Since the holding state is recognized based on the holding state information robust to various noises, input to a terminal based on high accuracy recognition processing is achieved.

Description

  The present invention relates to a recognition device, a recognition method, and a recognition system.

  Conventionally, there has been a technique for giving a predetermined command input to a terminal by means other than key input or touch panel operation. For example, a technique is known in which a sensor such as an acceleration sensor or a gyro sensor is provided in a terminal, and a predetermined command input is given to the terminal when a predetermined operation state is detected by the sensor while the user moves while holding the terminal. It has been. Further, a technique is known in which a user's fingerprint is detected and authentication is performed when the fingerprint corresponds to a pre-registered fingerprint. This technique uses a predetermined fingerprint pattern detection as an input command. Also, an authentication method for detecting payment by a sensor provided on a side surface of a mobile phone casing and executing settlement when the detected holding pattern matches a registered pattern registered in advance. Is known (see, for example, Patent Document 1).

JP 2009-77221 A

  In the prior art, in order to give a desired input to the terminal, the user is forced to perform a predetermined action, and thus cannot be used in a situation where the user cannot perform such an action. Also, in the technique of inputting to a terminal by inputting a fingerprint pattern, the user is forced to perform a specific operation for causing the sensor to read the fingerprint, and the operation is a burden on the user. Moreover, since the sensor in the technique described in Patent Document 1 only detects whether or not the sensor has been operated, this technique cannot make a highly accurate determination.

  Therefore, the present invention has been made in view of the above problems, and provides a recognition device, a recognition method, and a recognition system that can perform a highly accurate input operation to a terminal without forcing a user to perform a special operation. For the purpose.

  In order to solve the above-described problem, a recognition apparatus according to an aspect of the present invention is based on a gripping data group that indicates a gripping state of a terminal by a user and includes a plurality of gripping data detected in time series at the terminal. A recognition device for recognizing a gripping state of a terminal, the first gripping data acquiring unit acquiring a gripping data group acquired based on the first gripping state as a first gripping data group, and each of the gripping data groups Time-varying grip data consisting of a group of time-varying data indicating time fluctuations of grip data at time, wherein the time-varying grip data based on the first grip data group acquired by the first grip data acquisition means Based on the first time-variable grip data calculating means that is calculated as grip data, and the first time-variable grip data calculated by the first time-variable grip data calculating means Prior probability distribution calculating means for calculating a prior probability distribution for estimating a past time fluctuation state from past observed time fluctuation states, and first time fluctuation grasping data calculated by the first time fluctuation grasping data calculating means. State transition probability distribution calculating means for calculating a state transition probability distribution for estimating a current time fluctuation state from a past time fluctuation state, and a prior probability distribution and a state transition probability calculated by the prior probability distribution calculating means A posteriori probability distribution calculating means for calculating a posterior probability distribution for estimating a current temporal fluctuation state from a currently observed temporal fluctuation state based on the state transition probability distribution calculated by the distribution calculating means; The first grip state information including the posterior probability distribution calculated by the means is stored in the grip state information storage means as information for recognizing the grip state. Gripping state information storage means, second gripping data acquisition means for acquiring a gripping data group acquired based on the second gripping state as a second gripping data group, and second information acquired by the second gripping data acquisition means Based on the second time-variable grip data calculating means for calculating the time-variable grip data based on the grip data group as the second time-variable grip data, and the second time-variable grip data calculating means calculated by the second time-variable grip data calculating means, Using the first gripping state information stored in the gripping state information storage means, a corresponding probability indicating the probability that the time variation indicated by the second time-variable gripping data corresponds to the time variation in the first gripping state is calculated. And when the corresponding probability calculated by the corresponding probability calculation means is equal to or greater than a predetermined threshold, it is determined that the second gripping state corresponds to the first gripping state Determining means.

  In order to solve the above-described problem, a recognition method according to an aspect of the present invention is based on a gripping data group that includes a plurality of gripping data that indicates a gripping state of a terminal by a user and is detected in time series on the terminal. A recognition method in a recognition device for recognizing a gripping state of a terminal by a user, a gripping data acquisition step for acquiring a gripping data group acquired based on the first gripping state as a first gripping data group, and gripping data Time-variable gripping data composed of a group of time-variable data indicating temporal variation of gripping data at each time included in the group, and time-variable gripping data based on the first gripping data group acquired in the first gripping data acquisition step In the first time-variable gripping data calculation step and the first time-variable gripping data calculation step Based on the calculated first time fluctuation grasping data, a prior probability distribution calculating step for calculating a prior probability distribution for estimating a past time fluctuation state from a previously observed time fluctuation state; and first time fluctuation grasping data A state transition probability distribution calculating step for calculating a state transition probability distribution for estimating a current time fluctuation state from a past time fluctuation state based on the first time fluctuation grasping data calculated in the calculating step, and a prior probability distribution calculation Based on the prior probability distribution calculated in the step and the state transition probability distribution calculated in the step, the posterior probability distribution that estimates the current time fluctuation state from the currently observed time fluctuation state A posterior probability distribution calculating step and a posterior probability distribution calculating step including a posterior probability distribution calculated in the posterior probability distribution calculating step. A gripping state information storage step for storing the state information in the gripping state information storage unit as information for recognizing the gripping state, and a gripping data group acquired based on the second gripping state is acquired as a second gripping data group. A second gripping data acquisition step, a second time-varying gripping data calculation step for calculating time-varying gripping data based on the second gripping data group acquired in the second gripping data acquisition step as second time-varying gripping data, Based on the second time variation grip data calculated in the two time variation grip data calculation step, using the first grip state information stored in the grip state information storage means, the time variation indicated in the second time variation grip data Is a corresponding probability calculating step for calculating a corresponding probability indicating a probability corresponding to time variation in the first gripping state, and a corresponding probability calculating A determination step of determining that the second gripping state corresponds to the first gripping state when the corresponding probability calculated in the step is equal to or greater than a predetermined threshold value.

  In order to solve the above problem, a recognition system according to an aspect of the present invention includes a terminal and a grip data group that includes a plurality of grip data that indicates a grip state of the terminal by a user and is detected in time series at the terminal A recognition device for recognizing a gripping state of a terminal by a user, the terminal including a detection unit that detects gripping data and a plurality of gripping data detected in time series by the detection unit. Transmission means for transmitting a grip data group including the recognition data to the recognition device, and the recognition device transmits the grip data group transmitted from the terminal and acquired based on the first grip state to the first grip data group Time-variable gripping data comprising a first gripping data acquisition unit that acquires the time-variable data indicating the temporal variation of gripping data at each time included in the gripping data group. First time-variable grip data calculating means for calculating time-variable grip data based on the first grip data group acquired by the first grip data acquisition means as first time-variable grip data, and first time-variable grip data calculation A prior probability distribution calculating means for calculating a prior probability distribution for estimating a past time fluctuation state from a time fluctuation state observed in the past based on the first time fluctuation grasping data calculated by the means; and a first time fluctuation A state transition probability distribution calculating means for calculating a state transition probability distribution for estimating a current time fluctuation state from a past time fluctuation state based on the first time fluctuation gripping data calculated by the grip data calculating means; Based on the prior probability distribution calculated by the distribution calculating means and the state transition probability distribution calculated by the state transition probability distribution calculating means. A posterior probability distribution calculating unit for calculating a posterior probability distribution for estimating a current temporal variation state from an inter-variation state, and grasping first grasping state information including the posterior probability distribution calculated by the posterior probability distribution calculating unit. As information for recognizing the state, a gripping state information storage unit to be stored in the gripping state information storage unit and a gripping data group transmitted from the terminal and acquired based on the second gripping state A second grip data acquisition unit that acquires two grip data groups, and a second time variation that calculates time-varying grip data based on the second grip data group acquired by the second grip data acquisition unit as second time-varying grip data Based on the grip data calculation means and the second time fluctuation grip data calculated by the second time fluctuation grip data calculation means, the first stored in the grip state information storage means A corresponding probability calculating means for calculating a corresponding probability indicating a probability that the time variation indicated in the second time-variable gripping data corresponds to the time variation in the first gripping state using the gripping state information; Determining means for determining that the second gripping state corresponds to the first gripping state when the corresponding probability calculated by the above is equal to or greater than a predetermined threshold value.

  According to the said form, since input operation to a terminal is implement | achieved by recognition of the holding data which shows the holding state of the terminal by a user, a special operation is not forced to a user. In addition, the time variation data of the grip data is calculated, and the posterior probability distribution calculated by the statistical method from the calculated time variation data is stored as the grip state information for recognizing the grip state. It is robust against noise caused by circumstances specific to the detection device for detection and other sudden noises. Since the gripping state is recognized based on the gripping state information that is robust against various noises, input to the terminal based on highly accurate recognition processing is realized.

  In the recognition apparatus according to another aspect, the posterior probability distribution calculating unit may calculate the posterior probability distribution using sequential Bayesian estimation.

  According to the above aspect, since the posterior probability distribution is calculated using sequential Bayesian estimation, an appropriate posterior probability distribution is obtained as information for recognizing the grip state.

  In a recognition apparatus according to another aspect, the posterior probability distribution calculating means may calculate the posterior probability distribution using a Kalman filter.

  According to the above aspect, when the grip data indicating the grip state and the noise included in the grip data can be regarded as following a Gaussian distribution, a suitable posterior probability distribution is calculated.

  In a recognition apparatus according to another embodiment, the posterior probability distribution calculating means may calculate the posterior probability distribution using a particle filter.

  According to the above aspect, even if the grip data indicating the grip state and the noise included in the grip data do not follow the Gaussian distribution, an appropriate posterior probability distribution is calculated.

  In the recognition apparatus according to another aspect, the grip data group acquired based on the third grip state is acquired as an update grip data group, and the grip data acquisition unit for update is acquired by the update grip data acquisition unit. Update time fluctuation grip data calculating means for calculating time change grip data based on the update grip data group as update time fluctuation grip data, and update time fluctuation grip data calculated by the update time fluctuation grip data calculation means A prior probability distribution calculating means for calculating the prior probability distribution based on the update prior probability distribution, and a state transition probability distribution based on the updating time fluctuation grasping data calculated by the updating time fluctuation grasping data calculating means, Calculated by the update state transition probability distribution calculation means for calculating the update state transition probability distribution and the update prior probability distribution calculation means Update posterior probability distribution calculating means for calculating the posterior probability distribution based on the updated prior probability distribution and the update state transition probability distribution calculated by the update state transition probability distribution as update posterior probability distribution; An update verification grip data acquisition unit that acquires a grip data group acquired based on the fourth grip state as an update verification grip data group, and an update verification grip data group acquired by the update verification grip data acquisition unit. Based on the time-variable grasping data for update verification calculated by the time-variable grasping data for update verification calculated by the time-variable grasping data for update verification calculated by the time-variable grasping data for update verification, Using the posterior probability distribution in the first gripping state information stored in the gripping state information storage means, the update verification time variation gripping data is stored. The first corresponding probability indicating the probability that the time variation indicated by the update verification time variation grasping data calculated by the data calculating means corresponds to the time variation in the first grasping state is calculated, and the update posterior probability distribution is calculated. Using the update posterior probability distribution calculated by the means, the second corresponding probability indicating the probability that the time variation indicated in the update verification time variation grasping data corresponds to the time variation in the third grasping state is calculated. A second corresponding probability calculating means; a second determining means for determining whether or not the second corresponding probability is greater than the first corresponding probability calculated by the second corresponding probability calculating means; and a second determining means When it is determined that the second corresponding probability is greater than the first corresponding probability, the first gripping state information stored in the gripping state information storage unit is changed to the third gripping that includes the update posterior probability distribution. Update to status information Update means.

  In the recognition method according to another embodiment, the grip data group acquired based on the third grip state is acquired in the update grip data acquisition step for acquiring the update grip data group and the update grip data acquisition step. Time-varying grip data calculation step for calculating time-varying grip data based on the update grip data group as time-varying grip data for updating, and time-varying grip data for updating calculated in the time-varying grip data calculating step for updating A prior probability distribution calculating step for calculating the prior probability distribution based on the update prior probability distribution, and a state transition probability distribution based on the updating time varying gripping data calculated in the updating time varying gripping data calculating step, A state transition probability distribution calculating step for updating as a state transition probability distribution for updating; Update that calculates the update prior probability distribution calculated in the prior probability distribution calculation step and the posterior probability distribution based on the update state transition probability distribution calculated in the update state probability distribution as the update posterior probability distribution Acquired in an posterior probability distribution calculation step, an update verification grip data acquisition step for acquiring a grip data group acquired based on the fourth grip state as an update verification grip data group, and an update verification grip data acquisition step Update verification time-varying grip data calculation step for calculating time-variable grip data based on the updated verification grip data group as update verification time-variable grip data, and update verification calculated in the update verification time-variable grip data calculation step Based on the time-variable gripping data for use and stored in the gripping state information storage means Using the posterior probability distribution in the gripping state information for the update verification, the time variation indicated in the update verification time variation gripping data calculated in the update verification time variation gripping data calculation step corresponds to the time variation in the first gripping state. The first corresponding probability indicating the probability is calculated, and the time variation indicated in the update verification time variation grasping data is calculated using the update posterior probability distribution calculated in the update posterior probability distribution calculating step. A second corresponding probability calculating step for calculating a second corresponding probability indicating a probability corresponding to a time variation in the state, and the second corresponding probability is greater than the first corresponding probability calculated in the second corresponding probability calculating step. A second determination step for determining whether or not the second corresponding probability is greater than the first corresponding probability in the second determination step; An update step of updating the first gripping state information stored in the holding state information storage unit to the third gripping state information including the update posterior probability distribution may be further included.

  According to the above embodiment, the time variation calculated from the update verification grip data based on the fourth gripping state is more likely to be the time variation in the third gripping state than the probability corresponding to the time variation in the first gripping state. When the corresponding probability is higher, the first grip state information stored in the grip state information storage unit as information for recognizing the grip state is obtained from the update grip data group based on the third grip state. It is updated to the third gripping state information including the calculated update posterior probability distribution. Accordingly, appropriate gripping state information is stored in the gripping state information storage unit as information for recognizing the gripping state. Therefore, for example, by using the second grip data based on the second grip state determined to correspond to the first grip state as the third grip data, the grip referred to for recognition of the grip state The information can be kept as appropriate information according to, for example, changes over time.

  In the recognition device according to another embodiment, before using the grip data group for calculation of time-variable grip data, noise caused by a transient state related to detection of the grip data in the grip data included in the grip data group, It may further comprise noise removing means for removing at least one of pulse noise, noise that can be removed by wavelet shrinkage on the grip data, noise caused by a DC component related to detection of the grip data, and Gaussian noise. Good.

  According to the above aspect, noise is removed from the grip data before calculating the time-varying grip data, so the accuracy of the time-varying grip data and various probability distributions calculated based on the time-varying grip data are improved. Is done.

  In the recognition device according to yet another aspect, the corresponding probability calculation means is configured to determine whether each of the plurality of time variation data included in the second time variation grip data is based on the posterior probability distribution included in the first grip state information. The probability corresponding to the gripping state is calculated for each time variation data, and the determination means determines the probability calculated by the probability calculation means for the number of time variation data included in the second time variation grip data is a predetermined threshold value. A similarity measure may be calculated based on the ratio of the number of time variation data as described above, and when the similarity measure is equal to or greater than a predetermined threshold, it may be determined that the second gripping state corresponds to the first gripping state. .

  According to the said form, determination is performed by the similarity scale calculated based on the ratio of the number of the time variation data applicable to the 1st holding state with respect to all the time variation data contained in the 2nd time variation grasping data. Thereby, it is appropriately determined whether or not the second gripping state corresponds to the first gripping state.

  In a recognition system according to another aspect, the recognition apparatus further includes recognition information transmission means for transmitting recognition information related to a determination result by the determination means to the terminal, and the terminal receives the recognition information transmitted from the authentication apparatus. It is good also as a recognition information receiving means and an execution means to perform the predetermined process according to the recognition information received by the recognition information receiving means.

  According to the above aspect, for example, a predetermined process such as user authentication at the terminal can be realized without forcing the user to perform a special operation.

  Provided are a recognition device, a recognition method, and a recognition system that enable a highly accurate input operation to a terminal without forcing a user to perform a special operation.

It is a figure which shows the apparatus structure of a recognition system. It is a schematic diagram which shows arrangement | positioning of the pressure sensor in a terminal. It is a block diagram which shows the functional structure of a terminal. It is a block diagram which shows the functional structure of a recognition apparatus. It is a hard block diagram of a recognition device. It is a timing chart which shows the processing content of the recognition method in a recognition system. It is a timing chart which shows the processing content of the recognition method in a recognition system. It is a timing chart which shows the processing content of the recognition method in a recognition system.

  Embodiments of a recognition apparatus, a recognition method, and a recognition system according to the present invention will be described with reference to the drawings. If possible, the same parts are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 1 is a diagram illustrating a device configuration of a recognition system. The recognition system 100 includes a recognition device 1 and a terminal 3. The recognition apparatus 1 is communicable with the terminal 3 via the network N and is data indicating the gripping state of the terminal 3 by the user, and includes a plurality of gripping data detected in time series at the terminal 3 Is a device that recognizes the gripping state of the terminal 3 by the user. The recognition device 1 can be a device that performs user authentication by determining whether or not the user holding the terminal corresponds to a user registered in advance based on the holding state of the terminal 3, for example. .

  The terminal 3 can be a mobile terminal as exemplified by a mobile phone, for example. The terminal 3 is held by the user in the use state. And the terminal 3 is provided with the sensor for detecting the holding state by a user. The sensor provided in the terminal 3 is a pressure sensor, for example.

FIG. 2 is a schematic diagram showing the arrangement of the pressure sensor d in the terminal 3. As illustrated in FIG. 2, the terminal 3 includes a total of N pressure sensors on the bottom surface p _B and the side surfaces p _S1 and p _S2 of the housing. More specifically, provided near an end portion of the bottom _{p B,} a plurality of sensors _{d n1 + 1, d n1 +} 2, d n1 + 3, ··, d n2, d n2 + 1, d n2 + 2, d n2 + 3, ·· d n3 is an array It has been. Further, a plurality of sensors d ₁ , d ₂ , d ₃ ,..., Dn ₁ , dn _{3 + 1} , dn _{3 + 2} , dn _{3 + 3} ,... D _N are provided in an array on each of the side surfaces p _S1 and p _S2. Yes. In the present embodiment, the sensor included in the terminal 3 is a plurality of pressure sensors, but may be an acceleration sensor, a gyro sensor, or the like.

  Subsequently, prior to the description of the recognition device 1, the function of the terminal 3 will be described. FIG. 3 is a block diagram showing a functional configuration of the terminal 3. The terminal 3 includes a detection unit 31 (detection unit), a transmission unit 32 (transmission unit), a recognition information reception unit 33 (recognition information reception unit), and an execution unit 34 (execution unit).

検出部３１は、取得した把持データ群ｆ_ｔを送信部３２に送出する。 The detection unit 31 is a part that detects gripping data indicating a gripping state of the terminal 3 by the user. Specifically, the grip data is detected in time series by each of the _N sensors d _{1 to} d N shown in FIG. The detection unit 31 acquires the grip data fi (t) from the i-th sensor at time t. Then, gripping the data group f _t detected in a time series is expressed as in equation (1).

The detection unit 31 sends the acquired gripping data group _ft to the transmission unit 32.

The transmission unit 32 is a part that transmits the grip data group f _t to the recognition device.

The recognition information receiving unit 33 is a part that receives the recognition information transmitted from the authentication device 1. Recognition information, for example, the gripping position shown by the transmitted gripped data group f _t is, information about advance whether corresponds to a predetermined holding state which is stored in the recognition device 1 the determination result, based on the determination result user Information related to the result of authentication can be included.

The execution unit 34 is a part that executes predetermined processing according to the recognition information received by the recognition information receiving unit 33. For example, when the recognition system 100, the gripping position shown by the transmitted gripped data group f _t is with that corresponding to a predetermined holding state, a system in which the predetermined command input to the terminal 3, The execution unit 34 executes predetermined processing according to the command.

Further, the recognition system 100, when the gripping position shown by the transmitted gripped data group f _t is a system that performs user authentication with that corresponding to a predetermined holding state, the execution unit 34, the terminal 3 Control the login state by the user.

  Next, the function of the recognition apparatus 1 will be described with reference to FIG. FIG. 4 is a block diagram showing a functional configuration of the recognition apparatus 1. The recognition apparatus 1 includes a grip data acquisition unit 11 (first grip data acquisition unit, second grip data acquisition unit, update grip data acquisition unit, update verification grip data acquisition unit), and a noise removal unit 12 (noise removal unit). , Time fluctuation grip data calculation unit 13 (first time fluctuation grip data calculation means, second time fluctuation grip data calculation means, update time fluctuation grip data calculation means, update verification time fluctuation grip data calculation means), prior probability distribution Calculation unit 14 (prior probability distribution calculation unit, update prior probability distribution calculation unit), state transition probability distribution calculation unit 15 (state transition probability distribution calculation unit, update state transition probability distribution calculation unit), posterior probability distribution calculation unit 16 (Posterior probability distribution calculation means, update posterior probability distribution calculation means), gripping state information storage unit 17 (grip state information storage unit), gripping state information storage unit 18 (grip state) Information storage unit), corresponding probability calculation unit 19 (corresponding probability calculation unit), determination unit 20 (determination unit), recognition information transmission unit 21 (recognition information transmission unit), second corresponding probability calculation unit 22 (second corresponding probability calculation) Means), a second determination unit 23 (second determination unit), and an update unit 24 (update unit).

  FIG. 5 is a hardware configuration diagram of the recognition device 1. As shown in FIG. 5, the recognition apparatus 1 physically includes a CPU 101, a RAM 102 and a ROM 103 that are main storage devices, a communication module 104 that is a data transmission / reception device, an auxiliary storage device 105 such as a hard disk and a flash memory, and an input device. The computer system includes an input device 106 such as a keyboard and an output device 107 such as a display. Each function shown in FIG. 4 has a communication module 104, an input device 106, and an output device 107 under the control of the CPU 101 by loading predetermined computer software on the hardware such as the CPU 101 and the RAM 102 shown in FIG. This is realized by reading and writing data in the RAM 102 and the auxiliary storage device 105. The terminal 3 shown in FIG. 3 is similarly configured as a computer system. Again, with reference to FIG. 4, each function part of the recognition apparatus 1 is demonstrated.

  Again, with reference to FIG. 4, each function part of the recognition apparatus 1 is demonstrated. First, in the system for recognizing whether or not the gripping state of the terminal 3 by the user corresponds to the predetermined gripping state, the function of each function unit in the process for obtaining gripping state information indicating the predetermined gripping state Will be described. In the following description, the predetermined gripping state is referred to as a “first gripping state”.

  The grip data acquisition unit 11 is a part that acquires a grip data group including time-series grip data transmitted from the terminal 3. Here, the grip data acquisition unit 11 acquires the grip data group acquired based on the first grip state as the first grip data group. The first grip data group acquired based on the first grip state is handled as so-called learning grip data, and is represented by Expression (1).

  The noise removing unit 12 is a part that removes noise of grip data included in the grip data group. The noise removing unit 12 includes noise caused by a transient state related to detection of the gripping data, pulse noise, noise that can be removed by wavelet shrinkage on the gripping data, noise caused by a direct current component related to detection of the gripping data, and At least one of the Gaussian noise can be removed. In the present embodiment, the noise removing unit 12 removes all of these noises from the grip data included in the first grip data group. The noise removal will be specifically described below.

The noise removing unit 12 removes noise caused by a transient state related to detection of the grip data as follows. That is, assuming that the time at which the grip data f _i (t) is acquired is T _S ≦ t ≦ T _e , the noise removing unit 12 acquires data in a transient state in data detection as shown in Expression (2). Data acquired at time δt immediately after the start and immediately before the end of data acquisition is removed.

Next, the noise removing unit 12 removes the pulse noise as follows. That is, when the grip data f _i (t) satisfies the condition shown in Expression (3), it can be determined that the grip data includes a pulse, so the noise removing unit 12 satisfies the condition of Expression (3). All grip data acquired at the time t when the grip data f _i (t) is acquired are excluded from the processing target.

そして、このウェーブレット係数に基づき推定されるノイズの標準偏差δは、式（５）により表される。

また、ノイズの標準偏差δに基づき閾値λを式（６）のように算出する。

ただし、式（６）におけるＮはウェーブレット係数の総数を表す。そして、この閾値λを用いて式（７）に示すＳｏｆｔ Ｔｈｒｅｓｈｏｌｄ処理を施す。

さらに、ノイズ除去部１２は、式（７）により処理されたウェーブレット係数を逆ウェーブレット変換し、式（８）に示されるような、ノイズ除去された把持データを算出する。

なお、式（８）における左辺を、以下の説明において、ｆ＾_ｉ（ｔ）と記す。 Subsequently, the noise removal unit 12 performs noise removal by wavelet shrinkage on the grip data. The noise removal by wavelet shrinkage will be described below. The wavelet coefficient d ^j _k when the gripping data f _i (t) is expanded with the wavelet basis function φ ^j _k (t) is expressed as in Expression (4).

The standard deviation δ of noise estimated based on this wavelet coefficient is expressed by equation (5).

Further, the threshold λ is calculated as shown in Expression (6) based on the standard deviation δ of noise.

However, N in Formula (6) represents the total number of wavelet coefficients. Then, using this threshold λ, the Soft Threshold process shown in Expression (7) is performed.

Further, the noise removing unit 12 performs inverse wavelet transform on the wavelet coefficients processed by Expression (7), and calculates the gripped data from which noise is removed as shown in Expression (8).

In addition, the left side in Formula (8) is described as f ^ _i (t) in the following description.

なお、式（９）におけるＴは、時系列の把持データからなる把持データ群が取得された時間を表す。 Next, the noise removing unit 12 removes noise caused by a DC component related to detection of the grip data. That is, as shown in Expression (9), the noise removing unit 12 subtracts the time average of the grip data and calculates the alternating current component of the grip data.

Note that T in Expression (9) represents a time when a gripping data group including time-series gripping data is acquired.

なお、σはガウス関数のハイパーパラメータを表し、ｔ_ｎｅは時間ｔの近傍を表す。 Further, the noise removing unit 12 removes Gaussian noise. That is, as shown in Expression (10), the noise removing unit 12 smoothes data by convolution with a Gaussian function Gauss (t _ne , σ), and removes Gaussian noise from the grip data.

Note that σ represents a hyperparameter of a Gaussian function, and t _ne represents the vicinity of time t.

そして、Ｎ個のセンサ全てにより取得された把持データの時間変動データは、式（１２）により表される。

The time variation grip data calculation unit 13 is a part that calculates time variation grip data composed of a group of time variation data indicating time variation of grip data at each time included in the grip data group. The time variation data is difference data between the grip data at one time and the grip data acquired immediately before in time series. Here, the time-variable gripping data calculation unit 13 calculates time-variable gripping data based on the first gripping data group from which noise has been removed by the noise removing unit 12 as first time-variable gripping data. For example, the time variation grip data calculation unit 13 calculates time variation data y _i (t) using the grip data obtained from Equation (10) as shown in Equation (11).

Then, the time variation data of the grip data acquired by all the N sensors is expressed by Expression (12).

  The prior probability distribution calculation unit 14 is a part that calculates a prior probability distribution for estimating the past time fluctuation state from the time fluctuation state observed in the past based on the time fluctuation grasping data. Here, the prior probability distribution calculation unit 14 calculates the prior probability distribution based on the first time variation gripping data calculated by the time variation gripping data calculation unit 13.

  The state transition probability distribution calculation unit 15 is a part that calculates a state transition probability distribution for estimating the current time variation state from the past time variation state based on the time variation grasping data. Here, the state transition probability distribution calculation unit 15 calculates the state transition probability distribution based on the first time variation gripping data calculated by the time variation gripping data calculation unit 13.

  The posterior probability distribution calculation unit 16 uses the prior probability distribution calculated by the prior probability distribution calculation unit 14 and the state transition probability distribution calculated by the state transition probability distribution calculation unit 15 based on the currently observed time variation state. This is the part that calculates the posterior probability distribution that estimates the current temporal fluctuation state.

  The calculation of the prior probability distribution by the prior probability distribution calculation unit 14, the calculation of the state transition probability distribution by the state transition probability distribution calculation unit 15, and the calculation of the posterior probability distribution by the posterior probability distribution calculation unit 16 will be specifically described below.

The first time-varying grip data calculated by the time-varying grip data calculating unit 13 is expressed by Expression (13).

In the present embodiment, the posterior probability distribution p (x _t | y _t ) is calculated based on the first time-variable gripping data and the internal state x _t in the current gripping state, and the calculated posterior probability distribution is gripped as gripping state information. It is stored in the state information storage unit 18. The internal state x _t at holding state is a true state showing a gripping state of the terminal by the user, is directly impossible measurement condition. Actually measurable state, noise and errors and the like are grasped data observed participating to the internal state x _t.

  In the present embodiment, the posterior probability distribution calculation unit 16 calculates the posterior probability distribution using sequential Bayesian estimation. Here, an example in which a Kalman filter is used as a method for estimating the posterior probability distribution will be described. In sequential Bayesian estimation using a Kalman filter, a suitable posterior probability distribution is calculated when the gripping data indicating the gripping state and the noise included in the gripping data can be regarded as following a Gaussian distribution. Note that a method for estimating the posterior probability distribution is not particularly limited, and various methods can be employed. For example, a method using a particle filter may be used. In sequential Bayesian estimation using a particle filter, an appropriate posterior probability distribution is calculated even when gripping data indicating a gripping state and noise included in the gripping data do not follow a Gaussian distribution.

  The Kalman filter predicts the system state based on the state observed immediately before and the state observed at that time. During learning, the "prediction step" that predicts the current state based on the previous state and the "correction step" that corrects the prediction based on the error between the predicted state and the actually observed state are repeated. The fluctuation prediction distribution is obtained. In this prediction, it is assumed that the central force always works when the terminal 3 is held by the user. That is, the “state in which the central force is applied” refers to a state having a restoring force such as a spring in which the user tries to loosen after grasping the terminal 3 and tries to grasp after the user loosens.

ここで、Ｇ_ｔは状態雑音行列を表し、Ｎ（０，Ｒ_ｔ）は、平均０、共分散Ｒ_ｔの正規分布を表す。 Here, it is assumed that the internal state of the gripping state at a certain time t is x _t and the observation state is y _t , and the internal state changes while satisfying the Markov property. If the state transition matrix F _t and the observation matrix H _t are expressed by a linear model and the system noise w _t and the observation noise v _t can be expressed by a Gaussian distribution, x _{t + 1} , y _t , w _t , v _t Is expressed as in equations (14) to (17).

Here, G _t represents a state noise matrix, and N (0, R _t ) represents a normal distribution with mean 0 and covariance R _t .

そして、推定されるべき内部状態は、ｘ_ｔの期待値として、式（２０）のように表される。

なお、式（２０）における左辺を、以下の説明において、ｘ＾_ｔと記す。 At this time, x _t and y _t are expressed in the form of conditional likelihood distributions p (x _t | x _t −1) and p (y _t | x _t ). That is, the state transition probability distribution calculation unit 15 calculates a state transition probability distribution as shown in Equation (18), and the posterior probability distribution calculation unit 16 calculates a posterior probability distribution as shown in Equation (19). To do.

Then, the internal state to be estimated, as the expected value of x _t, is expressed by the equation (20).

In addition, the left side in Formula (20) is described as x ^ _t in the following description.

なお、上記式中のｋ_ｔは、ｘ_ｔに無関係な正規化定数である。 Furthermore, the prior probability distribution calculated by the prior probability distribution calculating unit 14 is expressed as in equations (21) to (22).

Incidentally, k _t in the formula is independent normalization constant x _t.

  In the sequential Bayesian estimation using the Kalman filter, as shown below, the posterior probability distribution is calculated by repeating the prediction step and the correction step using the above equations. This will be specifically described below.

さらに、式（２６）、式（２７）に基づき得られる予測分布の共分散行列Ｐ_ｔ／ｔに基づき、式（２８）のように、カルマンゲインＫ_ｔを算出する（修正ステップ）。カルマンゲインは、観測誤差と推定誤差の大きさに応じて値を調整するものである。

First, as an initial value, the gripping state x ^ _0/0 estimated _(symbol ^ is to be described in the following text, positioned above the letter. Similarly in the following description), the covariance matrix P _0/0 give. Then, x ^ _{t / t-1} and _{Pt / t-1} are recursively calculated according to the equations (24) and (25) (prediction step). In each expression and the following description, the slash symbol in the notation t / t−1 means a value that estimates the state at time t based on the state at time t−1.

Furthermore, based on the covariance matrix P _{t / t} of the predicted distribution obtained based on the equations (26) and (27), the Kalman gain K _t is calculated as in the equation (28) (correction step). The Kalman gain is a value that is adjusted according to the magnitude of the observation error and the estimation error.

そして、予測ステップ及び修正ステップを、取得された第１時間変動把持データ全て（１≦ｔ≦Ｔ）について繰り返し実施する。 Note that the state transition matrix F _t and the observation matrix H _t can be formulated if a grasping state model can be defined basically. As described above, the gripping state is always center force (restoring force like a spring that loosens after being gripped and tries to grip after being loosened) works near the local area, and most of the observed noise is random. Since it is assumed that noise is superimposed, each matrix can be defined as in equations (29) to (32). Note that the optimum parameters of these matrices can be determined by the EM algorithm. Further, the state noise matrix G _t can be defined as in the equations (33) to (34).

Then, the prediction step and the correction step are repeatedly performed for all the acquired first time variation gripping data (1 ≦ t ≦ T).

When the above processing is completed, the gripping state information storage unit 17 uses the first gripping state information including the posterior probability distribution calculated by the posterior probability distribution calculating unit 16 as information for recognizing the gripping state. The information is stored in the information storage unit 18. The gripping state information storage unit 18 is a storage unit for storing gripping state information and the like. Specifically, as shown in the equation (35), the gripping state information storage unit 17 uses the data set group DS calculated by the sequential Bayes estimation described above as data for collation when identifying the gripping state (first It is stored in the gripping state information storage unit 18 as gripping state information).

  Next, processing contents of the recognition method in the recognition system 100 and the recognition apparatus 1 will be described with reference to FIG.

  First, the detection unit 31 detects first grip data (S1). The first grip data is data that is the basis of the verification data when the grip state is recognized, and is acquired based on the first grip state. Subsequently, the transmission unit 32 transmits a first grip data group including time-series grip data to the recognition device 1 (S2).

  When the first grip data group is transmitted from the terminal 3, the grip data acquisition unit 11 acquires the first grip data group acquired based on the first grip state (S3). Next, the noise removing unit 12 removes various noises included in the grip data included in the first grip data group (S4). Subsequently, the time-variable grip data calculation unit 13 calculates time-variable grip data based on the first grip data group from which noise has been removed by the noise removal unit 12 in step S4 as first time-variable grip data (S5).

  Next, the prior probability distribution calculation unit 14 calculates the prior probability distribution based on the first time variation gripping data calculated by the time variation gripping data calculation unit 13 (S6). Further, the state transition probability distribution calculation unit 15 calculates the state transition probability distribution based on the first time variation gripping data calculated by the time variation gripping data calculation unit 13 (S7). Further, the posterior probability distribution calculation unit 16 calculates the posterior probability distribution based on the prior probability distribution calculated by the prior probability distribution calculation unit 14 and the state transition probability distribution calculated by the state transition probability distribution calculation unit 15 (S8). ).

  The gripping state information storage unit 17 stores the first gripping state information including the posterior probability distribution calculated by the posterior probability distribution calculating unit 16 in the gripping state information storage unit 18 as information for recognizing the gripping state. Store (S9).

  Next, in the system for recognizing whether or not the gripping state of the terminal 3 by the user corresponds to the predetermined gripping state, the gripping state of the terminal 3 by the user is stored in the recognition device 1 The function of each functional unit in the process of determining whether or not the above applies will be described. In the following description, the gripping state to be determined and verified is referred to as a “second gripping state”.

  The grip data acquisition unit 11 acquires a grip data group acquired based on the second grip state as a second grip data group. That is, the second grip data group is verification grip data to be determined and verified. The second grip data group is data detected by the detection unit 31 and further transmitted by the transmission unit 32 in the terminal 3 gripped by the user in the second grip state.

  The noise removal unit 12 removes noise included in the grip data included in the second grip data group acquired by the grip data acquisition unit 11 in the same manner as the first grip data group.

The time variation grip data calculation unit 13 calculates time variation grip data based on the second grip data group from which noise has been removed by the noise removal unit 12 as second time variation grip data. The second time fluctuation grip data calculated here is expressed by Expression (36). In the expression (36) and other expressions, the character V means that it relates to verification data based on the second gripping state.

  The corresponding probability calculation unit 19 uses the first gripping state information stored in the gripping state information storage unit 18 based on the second time-varying gripping data calculated by the time-varying gripping data calculation unit 13 to generate the second time A corresponding probability indicating a probability that the time variation indicated by the variable gripping data corresponds to the time variation in the first gripping state is calculated. Specifically, the corresponding probability calculation unit 19 sets each of the plurality of time variation data included in the second time variation grip data to the first grip state based on the posterior probability distribution included in the first grip state information. The corresponding probability is calculated for each time variation data.

そして、該当確率算出部１９は、第２時間変動把持データの全て（１≦ｔ≦Ｔ_Ｖ）について、確率を算出する。 That is, the corresponding probability calculation unit 19 calculates the probability Pro (t) at time t as shown in Expression (37) based on the posterior probability distribution calculated by Expression (19) and Expression (20).

Then, the corresponding probability calculation unit 19 calculates probabilities for all the second time-variable gripping data (1 ≦ t ≦ T _V ).

  The determination unit 20 determines that the second gripping state corresponds to the first gripping state when the corresponding probability calculated by the corresponding probability calculating unit 19 is equal to or greater than a predetermined threshold. Specifically, the determination unit 20 has a ratio of the number of time variation data in which the probability calculated by the corresponding probability calculation unit 19 is equal to or greater than a predetermined threshold with respect to the number of time variation data included in the second time variation grasping data. A similarity scale is calculated based on the above, and when the similarity scale is equal to or greater than a predetermined threshold, it is determined that the second gripping state corresponds to the first gripping state.

That is, the determination unit 20 calculates the ratio of the number of time variation data with which the value of the probability Pro (t) is equal to or greater than the threshold Th _{predict to} the number of time variation data included in the second time variation grasping data, using the equation (38). And as shown in (39), it calculates as a similarity scale Sim (Y, YT _(V) ).

Then, as shown in the equation (40), the determination unit 20 determines that the second gripping state is the first gripping state when the similarity measure Sim (Y, YT _(V) ) is equal to or greater than a predetermined threshold _Thjudge. It is determined that it corresponds to the state.

  The recognition information transmission unit 21 transmits the recognition information regarding the result of determination by the determination unit 20 to the terminal 3. For example, when the recognition system 100 is a system that performs user authentication of the terminal 3, the user's holding state of the terminal 3 is registered in advance in the recognition device as the first holding state, and user authentication is performed. The recognition process in the recognition device 1 is executed with the current gripping state as the second gripping state. In this case, the recognition information can be information related to the result of user authentication.

  Next, processing contents of the recognition method in the recognition system 100 and the recognition apparatus 1 will be described with reference to FIG.

  First, the detection unit 31 detects second grip data (S11). The second grip data is data that is a basis of determination and verification data in the grip state recognition process, and is acquired based on the second grip state. Subsequently, the transmission unit 32 transmits a second grip data group including time-series grip data to the recognition device 1 (S12).

  When the second grip data group is transmitted from the terminal 3, the grip data acquisition unit 11 acquires the second grip data group acquired based on the second grip state (S13). Next, the noise removing unit 12 removes various noises included in the grip data included in the second grip data group (S14). Subsequently, the time-variable grip data calculating unit 13 calculates time-variable grip data based on the second grip data group from which noise has been removed by the noise removing unit 12 in step S14 as second time-variable grip data (S15).

  Next, the corresponding probability calculation unit 19 uses the first gripping state information stored in the gripping state information storage unit 18 based on the second time-varying gripping data calculated by the time-varying gripping data calculation unit 13, A corresponding probability indicating the probability that the time variation indicated by the second time variation gripping data corresponds to the time variation in the first gripping state is calculated (S16). Subsequently, the determination unit 20 determines that the second gripping state corresponds to the first gripping state when the corresponding probability calculated by the corresponding probability calculating unit 19 is equal to or greater than a predetermined threshold (S17). And the recognition information transmission part 21 transmits the recognition information regarding the result of determination by the determination part 20 to the terminal 3 (S18).

  Subsequently, the recognition information receiving unit 33 receives the recognition information transmitted from the authentication device 1 (S19). And the execution part 34 performs the predetermined | prescribed process according to the recognition information received by the recognition information receiving part 33 (S20).

  As described above, since the terminal 3 can execute predetermined processing in accordance with the grip data indicating the grip state of the terminal 3 by the user, only the grip of the terminal 3 can be performed without forcing the user to perform a special operation. Thus, input to the terminal 3 is realized.

  Next, in the system for recognizing whether or not the gripping state of the terminal 3 by the user corresponds to a predetermined gripping state, the function of each functional unit in the update processing of the gripping state information for verification will be described. In the following description, the gripping state used as the basis of the gripping state information for update is referred to as the “third gripping state”, and the gripping state for verifying whether or not updating is performed using the gripping state information for updating is performed. Is the “fourth gripping state”.

  The grip data acquisition unit 11 acquires a grip data group acquired based on the third grip state as an update grip data group. The updated grip data group is data that is detected by the detection unit 31 and further transmitted by the transmission unit 32 in the terminal 3 gripped by the user in the third grip state.

  The noise removal unit 12 removes noise included in the grip data included in the update grip data group acquired by the grip data acquisition unit 11 in the same manner as the first grip data group.

  The time variation grip data calculation unit 13 calculates time variation grip data based on the update grip data group from which noise has been removed by the noise removal unit 12 as update time variation grip data.

  Similar to the calculation of the prior probability distribution based on the first time fluctuation gripping data, the prior probability distribution calculation unit 14 updates the prior probability distribution for update based on the update time fluctuation gripping data calculated by the time fluctuation gripping data calculation unit 13. Is calculated.

  Similarly to the calculation of the state transition probability distribution based on the first time variation grasping data, the state transition probability distribution calculating unit 15 is based on the update time variation grasping data calculated by the time variation grasping data calculating unit 13. A transition probability distribution is calculated.

  The posterior probability distribution calculation unit 16 calculates an update posterior probability distribution based on the update prior probability distribution calculated by the prior probability distribution calculation unit 14 and the update state transition probability distribution calculated by the state transition probability distribution calculation unit 15. calculate.

  The gripping state information storage unit 17 causes the gripping state information storage unit 18 to store the third gripping state information including the update posterior probability distribution calculated by the posterior probability distribution calculation unit 16.

  Note that the second gripping state determined to correspond to the first gripping state is regarded as the third gripping state, and the grip data described above is used by using the second gripping data group as the update gripping data group. Processing from acquisition of the update grip data group by the acquisition unit 11 to calculation of the update posterior probability distribution by the posterior probability distribution calculation unit 16 may be performed.

  Further, the grip data acquisition unit 11 acquires, as an update verification grip data group, a grip data group acquired based on the fourth grip state for verifying whether or not to update using the grip state information for update. To do. The update verification gripping data group is data detected by the detection unit 31 in the terminal 3 gripped by the user in the fourth gripping state and further transmitted by the transmission unit 32.

  The noise removal unit 12 removes noise included in the grip data included in the update verification grip data group acquired by the grip data acquisition unit 11 in the same manner as the first grip data group.

  The time variation grip data calculation unit 13 calculates time variation grip data based on the update verification grip data group from which noise has been removed by the noise removal unit 12 as update verification time variation grip data.

  The second corresponding probability calculation unit 22 uses the posterior probability distribution in the first gripping state information stored in the gripping state information storage unit 18 to update the time variation gripping for update verification calculated by the time variation gripping data calculation unit 13. A first corresponding probability indicating a probability that the time variation indicated in the data corresponds to the time variation in the first gripping state is calculated.

  In addition, the second corresponding probability calculation unit 22 uses the update posterior probability distribution calculated by the posterior probability distribution calculation unit 16, and the time variation indicated in the update verification time variation grasping data is in the third grasping state. A second corresponding probability indicating the probability corresponding to the time variation is calculated.

  The calculation of the first and second corresponding probabilities by the second corresponding probability calculating unit 22 is such that each of a plurality of time variation data included in the second time varying gripping data by the corresponding probability calculating unit 19 corresponds to the first gripping state. As with the calculation of the probability of

なお、式（４１）における左辺は第２の類似性尺度に該当し、右辺は第１の類似性尺度に該当する。 The second determination unit 23 determines whether or not the second corresponding probability is larger than the first corresponding probability calculated by the second corresponding probability calculation unit 22. Specifically, the second determination unit 23 calculates a first similarity scale indicating the degree of similarity between the fourth gripping state and the first gripping state based on the first corresponding probability, Based on the corresponding probability, a second similarity measure indicating the degree of similarity between the fourth gripping state and the third gripping state is calculated. The calculation of the first and second similarity measures by the second determination unit 23 is performed using the equations (38) and (39), similarly to the calculation of the similarity measure by the determination unit 20. The second determination unit 23, a data set group of the third holding state information including the update posterior probability distribution When DS _U, as shown in equation (41) and (42), first It is determined whether the second similarity measure is larger than the similarity measure.

Note that the left side in Equation (41) corresponds to the second similarity scale, and the right side corresponds to the first similarity scale.

  The update unit 24 grasps when the second determination unit 23 determines that the second corresponding probability (second similarity measure) is larger than the first corresponding probability (first similarity measure). The first gripping state information stored in the state information storage unit 18 is updated to third gripping state information including an update posterior probability distribution.

  Next, with reference to FIG. 8, the processing content in the update process of the holding | grip state information for collation is demonstrated.

  First, the detection unit 31 detects update grip data (S31). The update grip data is acquired based on the third grip state. Subsequently, the transmission unit 32 transmits an update grip data group including time-series grip data to the recognition device 1 (S32).

  When the update grip data group is transmitted from the terminal 3, the grip data acquisition unit 11 acquires the update grip data group acquired based on the third grip state (S33). Next, the noise removing unit 12 removes various noises included in the grip data included in the update grip data group (S34). Subsequently, the time-variable grip data calculation unit 13 calculates time-variable grip data based on the grip data group from which noise has been removed by the noise removing unit 12 in step S34 as update time-variable grip data (S35).

  Next, the prior probability distribution calculation unit 14 calculates an update prior probability distribution based on the update time variation grip data calculated by the time variation grip data calculation unit 13 (S36). Further, the state transition probability distribution calculation unit 15 calculates the update state transition probability distribution based on the update time variation grip data calculated by the time variation grip data calculation unit 13 (S37). Further, the posterior probability distribution calculation unit 16 is based on the update prior probability distribution calculated by the prior probability distribution calculation unit 14 and the update state transition probability distribution calculated by the state transition probability distribution calculation unit 15. A distribution is calculated (S38). Then, the gripping state information storage unit 17 stores the third gripping state information including the updated posterior probability distribution calculated by the posterior probability distribution calculating unit 16 in the gripping state information storage unit 18 (S39).

  On the other hand, the detection unit 31 detects update verification grip data (S41). The update verification grip data is acquired based on the fourth grip state. Subsequently, the transmission unit 32 transmits an update verification grip data group including time-series grip data to the recognition device 1 (S42).

  When the update verification grip data group is transmitted from the terminal 3, the grip data acquisition unit 11 acquires the update verification grip data group acquired based on the fourth grip state (S43). Next, the noise removing unit 12 removes various noises included in the grip data included in the update verification grip data group (S44). Subsequently, the time-variable grip data calculation unit 13 calculates time-variable grip data based on the grip data group from which noise has been removed by the noise removal unit 12 in step S44 as update verification time-variable grip data (S45).

  Next, the second corresponding probability calculation unit 22 uses the posterior probability distribution in the first gripping state information stored in the gripping state information storage unit 18, and the time variation indicated in the update verification time variation gripping data is: A first corresponding probability indicating a probability corresponding to time variation in the first gripping state is calculated. The second corresponding probability calculation unit 22 uses the update posterior probability distribution to indicate a probability that the time variation indicated in the update verification time variation grasping data corresponds to the time variation in the third grasping state. Is calculated (S46).

  Subsequently, the second determination unit 23 determines whether or not the second corresponding probability calculated in step S46 is larger than the first corresponding probability (S47). If the update unit 24 determines in step S47 that the second corresponding probability is greater than the first corresponding probability, the update unit 24 stores the first grip state information stored in the grip state information storage unit 18. Then, it is updated to the third gripping state information including the updated posterior probability distribution (S48).

  As described above, when the gripping state information is updated, appropriate gripping state information is stored in the gripping state information storage unit 18 as information for recognizing the gripping state. Therefore, for example, by adopting the second grip data based on the second grip state determined to correspond to the first grip state as the grip data for update, the grip state referred to for recognition of the grip state The information can be kept as appropriate information according to, for example, changes over time.

  According to the recognition device 1, the recognition method, and the recognition system of the present embodiment, the user can perform an input operation to the terminal 3 by recognizing the grip data indicating the grip state of the terminal 3. I can't be forced. In addition, the time variation data of the grip data is calculated, and the posterior probability distribution calculated by the statistical method from the calculated time variation data is stored as the grip state information for recognizing the grip state. It is robust against noise caused by circumstances specific to the detection device for detection and other sudden noises. Since the gripping state is recognized based on the gripping state information that is robust against various noises, the input to the terminal 3 based on the highly accurate recognition process is realized.

  The present invention has been described in detail based on the embodiments. However, the present invention is not limited to the above embodiment. The present invention can be variously modified without departing from the gist thereof.

DESCRIPTION OF SYMBOLS 1 ... Recognition apparatus, 100 ... Recognition system, 3 ... Terminal, 11 ... Grasp data acquisition part, 12 ... Noise removal part, 13 ... Time-variable grip data calculation part, 14 ... Prior probability distribution calculation part, 15 ... State transition probability distribution Calculation unit, 16 ... posterior probability distribution calculation unit, 17 ... gripping state information storage unit, 18 ... gripping state information storage unit, 19 ... corresponding probability calculation unit, 20 ... determination unit, 21 ... recognition information transmission unit, 22 ... second Corresponding probability calculation unit, 23 ... second determination unit, 24 ... update unit, 31 ... detection unit, 32 ... transmission unit, 33 ... recognition information reception unit, 34 ... execution unit.

Claims (11)

A recognition device for recognizing a gripping state of a terminal by a user based on a gripping data group including a plurality of gripping data detected in time series at the terminal and indicating a gripping state of the terminal by the user,
First grip data acquisition means for acquiring the grip data group acquired based on the first grip state as a first grip data group;
Time variation grip data comprising a group of time variation data indicating time variation of the grip data at each time included in the grip data group, wherein the first grip data group acquired by the first grip data acquisition means First time-variable gripping data calculating means for calculating the time-variable gripping data based on the first time-variable gripping data;
Prior probability distribution for calculating a prior probability distribution for estimating a past time fluctuation state from a previously observed time fluctuation state based on the first time fluctuation grip data calculated by the first time fluctuation grip data calculating means. A calculation means;
State transition probability distribution calculation for calculating a state transition probability distribution for estimating a current time fluctuation state from a past time fluctuation state based on the first time fluctuation gripping data calculated by the first time fluctuation gripping data calculating means. Means,
Based on the prior probability distribution calculated by the prior probability distribution calculating means and the state transition probability distribution calculated by the state transition probability distribution calculating means, the current time fluctuation state is calculated from the currently observed time fluctuation state. A posteriori probability distribution calculating means for calculating a posteriori probability distribution to be estimated;
Gripping state information storage means for storing first gripping state information including the posterior probability distribution calculated by the posterior probability distribution calculating means in the gripping state information storage means as information for recognizing the gripping state;
Second grip data acquisition means for acquiring the grip data group acquired based on the second grip state as a second grip data group;
Second time-variable grip data calculating means for calculating the time-variable grip data based on the second grip data group acquired by the second grip data acquiring means as second time-variable grip data;
Based on the second time variation grip data calculated by the second time variation grip data calculation means, the first grip state information stored in the grip state information storage means is used to indicate the second time variation grip data. A corresponding probability calculating means for calculating a corresponding probability indicating a probability that the time variation is equivalent to the time variation in the first gripping state;
A determination unit that determines that the second gripping state corresponds to the first gripping state when the corresponding probability calculated by the corresponding probability calculating unit is equal to or greater than a predetermined threshold;
A recognition device comprising: The posterior probability distribution calculating means calculates the posterior probability distribution using sequential Bayesian estimation,
The recognition device according to claim 1. The posterior probability distribution calculating means calculates the posterior probability distribution using a Kalman filter.
The recognition apparatus according to claim 2. The posterior probability distribution calculating means calculates the posterior probability distribution using a particle filter.
The recognition apparatus according to claim 2. Update grip data acquisition means for acquiring the grip data group acquired based on the third grip state as an update grip data group;
Update time-variable grip data calculating means for calculating the time-variable grip data based on the update grip data group acquired by the update grip data acquiring means as update time-variable grip data;
An update prior probability distribution calculating means for calculating the prior probability distribution based on the update time variable gripping data calculated by the update time variable gripping data calculating means as an update prior probability distribution;
An update state transition probability distribution calculating means for calculating the state transition probability distribution based on the update time change gripping data calculated by the update time change gripping data calculating means;
The update posterior probability distribution based on the update prior probability distribution calculated by the update prior probability distribution calculation means and the update state transition probability distribution calculation means calculated by the update state transition probability distribution calculation means Update posterior probability distribution calculating means for calculating as distribution,
Update verification grip data acquisition means for acquiring the grip data group acquired based on a fourth grip state as an update verification grip data group;
Update verification time variation grip data calculation means for calculating the time variation grip data based on the update verification grip data group acquired by the update verification grip data acquisition means as update verification time variation grip data;
Based on the update verification time fluctuation grip data calculated by the update verification time fluctuation grip data calculation means, the update is performed using the posterior probability distribution in the first grip state information stored in the grip state information storage means. Calculating a first corresponding probability indicating a probability that the time variation indicated by the updated verification time variation grasping data calculated by the verification time variation grasping data calculating unit corresponds to the time variation in the first grasping state; Using the update posterior probability distribution calculated by the update posterior probability distribution calculating means, the probability that the time variation shown in the update verification time variation grasping data corresponds to the time variation in the third grasping state is calculated. Second corresponding probability calculating means for calculating a second corresponding probability shown;
Second determination means for determining whether the second corresponding probability is larger than the first corresponding probability calculated by the second corresponding probability calculating means;
When the second determination means determines that the second corresponding probability is greater than the first corresponding probability, the first grip state information stored in the grip state information storage means is updated. Updating means for updating to the third gripping state information including the posterior probability distribution;
The recognition device according to any one of claims 1 to 4, further comprising: Before using the grip data group for the calculation of the time-varying grip data, in the grip data included in the grip data group, noise, pulse noise, and the grip data corresponding to the transient state related to detection of the grip data are detected. Noise removing means for removing at least one of noise that can be removed by wavelet shrinkage, noise caused by a direct current component related to detection of the grip data, and Gaussian noise;
The recognition apparatus of any one of Claims 1-5. The corresponding probability calculation means includes
Based on the posterior probability distribution included in the first grip state information, the probability that each of a plurality of time variation data included in the second time variation grip data corresponds to the first grip state is represented by the time variation. Calculate for each data,
The determination means includes
Calculating a similarity measure based on the ratio of the number of time variation data in which the probability calculated by the corresponding probability calculation means is equal to or greater than a predetermined threshold with respect to the number of time variation data included in the second time variation grasping data; When the similarity measure is equal to or greater than a predetermined threshold, it is determined that the second gripping state corresponds to the first gripping state.
The recognition apparatus of any one of Claims 1-6. A recognition method in a recognition apparatus for recognizing a gripping state of a terminal by a user based on a gripping data group including a plurality of gripping data detected in time series at the terminal and indicating a gripping state of the terminal by the user,
A first grip data acquisition step of acquiring the grip data group acquired based on the first grip state as a first grip data group;
Time variation grip data composed of a group of time variation data indicating time variation of the grip data at each time included in the grip data group, the first grip data group acquired in the first grip data acquisition step A first time-variable gripping data calculating step for calculating the time-variable gripping data based on the first time-variable gripping data;
Prior probability distribution for calculating a prior probability distribution for estimating a past time fluctuation state from a past time fluctuation state based on the first time fluctuation grip data calculated in the first time fluctuation grip data calculation step. A calculation step;
State transition probability distribution calculation for calculating a state transition probability distribution for estimating a current time fluctuation state from a past time fluctuation state based on the first time fluctuation gripping data calculated in the first time fluctuation gripping data calculating step. Steps,
Based on the prior probability distribution calculated in the prior probability distribution calculating step and the state transition probability distribution calculated in the state transition probability distribution calculating step, the current time fluctuation state is calculated from the currently observed time fluctuation state. A posterior probability distribution calculating step for calculating a posterior probability distribution to be estimated;
A gripping state information storing step of storing first gripping state information including the posterior probability distribution calculated in the posterior probability distribution calculating step in the gripping state information storage unit as information for recognizing the gripping state;
A second grip data acquisition step of acquiring the grip data group acquired based on the second grip state as a second grip data group;
A second time-variable grip data calculation step for calculating the time-variable grip data based on the second grip data group acquired in the second grip data acquisition step as second time-variable grip data;
Based on the second time-variable grip data calculated in the second time-variable grip data calculation step, the first grip state information stored in the grip state information storage means is used to indicate the second time-variable grip data. A corresponding probability calculating step for calculating a corresponding probability indicating a probability that the time variation is equivalent to the time variation in the first gripping state;
A determination step of determining that the second gripping state corresponds to the first gripping state when the corresponding probability calculated in the corresponding probability calculating step is equal to or greater than a predetermined threshold;
A recognition method. An update grip data acquisition step for acquiring the grip data group acquired based on the third grip state as an update grip data group;
An update time-variable grip data calculation step for calculating the time-variable grip data based on the update grip data group acquired in the update grip data acquisition step as update time-variable grip data;
An update prior probability distribution calculating step for calculating the prior probability distribution based on the update time variable gripping data calculated in the update time variable gripping data calculating step as an update prior probability distribution;
An update state transition probability distribution calculation step for calculating the state transition probability distribution based on the update time variation grip data calculated in the update time variation grip data calculation step;
Update posterior probability distribution based on update prior probability distribution calculated in update prior probability distribution calculation step and update state transition probability distribution calculated in update state transition probability distribution calculation step. Update posterior probability distribution calculation step to calculate as distribution,
An update verification grip data acquisition step of acquiring the grip data group acquired based on the fourth grip state as an update verification grip data group;
An update verification time variation gripping data calculation step for calculating the time variation grip data based on the update verification grip data group acquired in the update verification grip data acquisition step as update verification time variation grip data;
Based on the update verification time variation grip data calculated in the update verification time variation grip data calculation step, the update is performed using the posterior probability distribution in the first grip state information stored in the grip state information storage means. Calculating a first corresponding probability indicating a probability that the time variation indicated in the updated verification time variation grasping data calculated in the verification time variation grasping data calculating step corresponds to the time variation in the first grasping state; Using the update posterior probability distribution calculated in the update posterior probability distribution calculation step, the probability that the time variation indicated in the update verification time variation grasping data corresponds to the time variation in the third grasping state is calculated. A second corresponding probability calculating step for calculating a second corresponding probability shown;
A second determination step of determining whether or not the second corresponding probability is greater than the first corresponding probability calculated in the second corresponding probability calculating step;
When it is determined in the second determination step that the second corresponding probability is greater than the first corresponding probability, the first grip state information stored in the grip state information storage unit is updated. An update step for updating to the third gripping state information including the posterior probability distribution;
The recognition method according to claim 8, further comprising: A recognition system comprising: a terminal; and a recognition device that recognizes a gripping state of the terminal by the user based on a gripping data group that indicates a gripping state of the terminal by the user and includes a plurality of gripping data detected in time series by the terminal Because
The terminal
Detecting means for detecting the grip data;
Transmitting means for transmitting a grip data group including a plurality of grip data detected in time series by the detection means to the recognition device;
With
The recognition device is
First grip data acquisition means for acquiring the grip data group transmitted from the terminal and acquired based on the first grip state as the first grip data group;
Time variation grip data comprising a group of time variation data indicating time variation of the grip data at each time included in the grip data group, wherein the first grip data group acquired by the first grip data acquisition means First time-variable gripping data calculating means for calculating the time-variable gripping data based on the first time-variable gripping data;
Prior probability distribution for calculating a prior probability distribution for estimating a past time fluctuation state from a previously observed time fluctuation state based on the first time fluctuation grip data calculated by the first time fluctuation grip data calculating means. A calculation means;
State transition probability distribution calculation for calculating a state transition probability distribution for estimating a current time fluctuation state from a past time fluctuation state based on the first time fluctuation gripping data calculated by the first time fluctuation gripping data calculating means. Means,
Based on the prior probability distribution calculated by the prior probability distribution calculating means and the state transition probability distribution calculated by the state transition probability distribution calculating means, the current time fluctuation state is calculated from the currently observed time fluctuation state. A posteriori probability distribution calculating means for calculating a posteriori probability distribution to be estimated;
Gripping state information storage means for storing first gripping state information including the posterior probability distribution calculated by the posterior probability distribution calculating means in the gripping state information storage means as information for recognizing the gripping state;
Second grip data acquisition means for acquiring, as a second grip data group, the grip data group acquired from the terminal and acquired based on the second grip state;
Second time-variable grip data calculating means for calculating the time-variable grip data based on the second grip data group acquired by the second grip data acquiring means as second time-variable grip data;
Based on the second time variation grip data calculated by the second time variation grip data calculation means, the first grip state information stored in the grip state information storage means is used to indicate the second time variation grip data. A corresponding probability calculating means for calculating a corresponding probability indicating a probability that the time variation is equivalent to the time variation in the first gripping state;
A recognition system comprising: a determination unit that determines that the second gripping state corresponds to the first gripping state when the corresponding probability calculated by the corresponding probability calculating unit is equal to or greater than a predetermined threshold. The recognition device is
Recognizing information transmitting means for transmitting the recognizing information related to the determination result by the determining means to the terminal;
The terminal
Recognition information receiving means for receiving the recognition information transmitted from the recognition device;
Execution means for executing a predetermined process according to the recognition information received by the recognition information receiving means;
The recognition system according to claim 10.

Images