JP2017202047A - Feature amount extraction device, estimation device, method for the same and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To estimate a psychological state from a kinetic change of an eye obtained in an optional environment.SOLUTION: A feature amount extraction device obtains a third feature amount depending on a degree of difference between a first feature amount obtained based on a kinetic change of an eye in a first temporal section and a feature amount in a first temporal section estimated from temporal sequence information on a second feature amount obtained based on a kinetic change of the eye in a second temporal section.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for estimating a psychological state from a dynamic change of an eye.

  In Patent Documents 1 and 2, the degree of difference in the feature amount based on the movement of the eyeball or the change in the size of the pupil between the first time interval in which a predetermined sound can be heard and the second time interval in which the predetermined sound cannot be heard is disclosed. Based on this, a technique for estimating the degree of conspicuousness of a received sound is disclosed.

Japanese Patent No. 5718492 Japanese Patent No. 5718493

  However, in the technologies disclosed in Patent Documents 1 and 2, it is necessary to artificially set the first time interval in which sound can be heard and the second time interval in which sound cannot be heard. Therefore, the psychological state could not be estimated from the dynamic change of the eye obtained in any environment.

  The subject of this invention is estimating a psychological state from the dynamic change of the eye obtained in arbitrary environments.

  Time series information of the first feature value obtained based on the dynamic change of the eye in the first time interval and the second feature value obtained based on the dynamic change of the eye in the second time interval A third feature value corresponding to the degree of difference from the feature value in the first time interval estimated from the above is obtained.

  Thereby, the psychological state can be estimated from the dynamic change of the eye obtained in an arbitrary environment.

FIG. 1 is a block diagram illustrating an apparatus configuration of the embodiment. FIG. 2 is a flowchart for explaining the processing of the embodiment. FIG. 3 is a diagram illustrating the movement of the eyeball. FIG. 4 is a diagram for explaining the feature amount of the microsaccade. FIG. 5 is a diagram illustrating a change in pupil diameter of both eyes. FIG. 6 is a diagram for illustrating the feature amount of each time section. FIG. 7A is a diagram for explaining experimental conditions. The upper diagram in FIG. 7A represents the regular condition of the inter-stimulus interval distribution, the lower diagram represents the irregular condition in which the inter-stimulus interval distribution has a width of 100 ms, and the black rectangle represents the presentation timing of the pure tone. . Figure 7B is a graph illustrating the feature amount S _t obtained by the experiment.

Embodiments of the present invention will be described below.
[principle]
First, the principle of this embodiment will be described. In each embodiment, based on the “first feature amount” obtained based on the dynamic change of the eye in the “first time interval” and the dynamic change of the eye in the “second time interval”. A “third feature amount” corresponding to the degree of difference from the feature amount in the “first time interval” estimated from the obtained “time series information of the second feature amount” is obtained. The “third feature amount” is the actual amount of eye movement in the “first time interval” compared to the feature amount in the “first time interval” predicted using the “time series information of the second feature amount”. It represents how much the “first feature value” obtained based on the change is different. In other words, the “third feature amount” is an index that indicates how much the “first time interval” is unusual in view of the trend of the feature amount predicted from the “second time interval”. Therefore, by using the “third feature amount”, the dynamic change of the eye obtained in an arbitrary environment without artificially setting the environments of the “first time interval” and the “second time interval”. The psychological state of the subject can be estimated from For example, since the “third feature amount” represents at least one of the degree of attention and the degree of surprise of the subject, at least one of the degree of attention and the degree of surprise can be estimated from the feature amount including at least the “third feature amount”. In addition, the “third feature amount” is the trend of the feature amount predicted from the “time series information of the second feature amount” in the “second time interval” (for example, in an increasing trend, a decreasing trend, or in a steady state) In order to express the novelty from the viewpoint of (e.g.), the target person with higher accuracy than simply evaluating the degree of difference of the "first feature quantity" with respect to the average value of the feature quantity in the "second time interval", etc. Can be estimated.

  The “first time interval” may be a time interval later (future) than the “second time interval”, or may be a time interval earlier (past) than the “second time interval”. The time interval included in the “second time interval” may be used. The “first time interval” may be a time interval immediately after or immediately before the “second time interval”, or may be a time interval away from the “second time interval”.

  The “second time interval” is preferably a time interval longer than the “first time interval”. This is because the longer the “second time interval” can accurately predict the trend of the feature amount, and the novelty of the “first time interval” with respect to the trend can be evaluated with high accuracy. For example, the “second time interval” may include a plurality of “partial time intervals”, and the “first time interval” may be one “partial time interval”. An example of the “partial time section” is a frame that is a signal processing unit, a subframe that constitutes the frame, or the like. However, if the trend of the feature amount can be predicted from the “second time interval”, the lengths of the “second time interval” and the “first time interval” may be equal to each other. May be a time interval shorter than the “first time interval”.

  The “second feature amount time series information” is a time series composed of a plurality of feature amounts corresponding to a plurality of times (time) or partial time intervals in the “second time interval”. That is, the “second feature amount time series information” is a time series corresponding to a plurality of times or a plurality of partial time sections in the “second time section”. “Multiple times” may be discrete times that are adjacent to each other, may be discrete times that are not adjacent to each other, or may include discrete times that are adjacent to each other and discrete times that are not adjacent to each other. Good. Similarly, “a plurality of partial time intervals” may be partial time intervals adjacent to each other, may be partial time intervals that are not adjacent to each other, or may not be adjacent to partial time intervals that are adjacent to each other. And a time interval.

  The “first feature value” and the “second feature value” are feature values obtained based on dynamic changes in the eyes of the same subject. The “dynamic change of the eye” may be a movement of the eyeball itself (a time-dependent change in the position of the eyeball) or a movement of the pupil (a time-dependent change in the pupil diameter). The “feature” may be anything, may be a scalar corresponding to each time or partial time interval, or may be a vector composed of a plurality of elements corresponding to each time or partial time interval. There may be. However, the “first feature value” and the “second feature value” are the same type of feature value. Further, the “feature amount” needs to include an element that can predict the trend from the time series. This is because in each embodiment, since the psychological state such as the degree of attention and the degree of surprise of the subject is evaluated based on the difference from the trend of the feature quantity, the trend of the “feature quantity” must be predictable as a prerequisite. For example, the “first feature amount” and the “second feature amount” desirably include a feature amount based on the saccade of the eyeball. The “saccade” may be a micro saccade or a large saccade. Features based on eyeball saccade include feature values based on eye movement direction, feature values based on absolute value of eye movement amplitude, feature values based on eye movement attenuation coefficient, and natural angular frequency of eye movement. The feature amount based on the feature amount based on the generation timing of the saccade of the eyeball can be exemplified. In particular, the direction of eyeball movement and the eyeball movement attenuation coefficient in saccades are highly time-dependent and tend to predict trends. Therefore, it is desirable that the “first feature amount” and the “second feature amount” include a feature amount based on at least one of the eyeball movement direction and the eyeball movement attenuation coefficient in the saccade. Further, it is desirable that the “first feature amount” and the “second feature amount” include a feature amount based on the generation timing of the saccade of the eyeball. Thereby, it becomes easy to predict the trend of the change in the feature amount according to the progress of time, and the estimation accuracy of the psychological state of the target person is increased.

  The “first feature value” and the “second feature value” are values derived from the dynamic change of one eye (for example, the right eye) of the same subject and the movement of the other eye (for example, the left eye). A feature amount based on a relative amount with a value derived from a general change may be included. The relative amount of the dynamic change of both eyes shows the attributes and personality of the subject, and the psychological state of the subject can be estimated with high accuracy by using the trend of the feature amount based on such relative amount. In particular, the attributes and personality of the subject appear in the relative values of the movement of the pupils of both eyes. Therefore, the “first feature value” and the “second feature value” are relative amounts of a value derived from the movement of the pupil of one eye of the same subject and a value derived from the movement of the pupil of the other eye. It is desirable to include a feature amount based on it. The “first feature amount” and the “second feature amount” are feature amounts based on a relative amount between a value derived from the saccade of one eye and a value derived from the saccade of the other eye of the same subject. May be included. “Relative amount of α and β” is, for example, the difference between α and β, the value obtained by subtracting β from α, the value obtained by subtracting α from β, the value obtained by dividing α by β, or β divided by α. Or a function value of any of them. The “feature amount based on relative value” is, for example, “relative value” or a function value thereof, a vector having “relative value” or the function value as an element, or any function value thereof.

  There is no limitation on the method of predicting the feature amount in the “first time interval” using the “time series information of the second feature amount”. For example, a prediction model such as an autoregressive model or a recurrent neural network, or a probability model such as a hidden Markov model is used to predict the feature quantity in the “first time interval” from the “time series information of the second feature quantity”. May be. The “subject” may be a human being or an animal other than a human being as long as the eye dynamically changes.

  As described above, by using the feature amount including the “third feature amount”, it is possible to estimate the psychological state such as the degree of attention and the degree of surprise of the subject. There is no particular limitation on this estimation method. For example, it may be estimated that the degree of attention and the degree of surprise are higher as the “third feature amount” is larger, or the degree of attention and the degree of surprise are higher when the “third feature amount” is greater than or equal to the threshold value. If this is not the case, it may be estimated that the degree of attention or the degree of surprise is low. Alternatively, a model that estimates a value representing a psychological state by performing supervised or unsupervised machine learning using a feature amount including a “third feature amount” obtained in advance or in real time as a learning data is newly obtained. Further, a value representing the psychological state may be estimated by applying a feature amount including the “third feature amount” to the model. For example, multiple regression analysis, k-means, support vector machine (SVM), simple clustering, hidden Markov model, neural network, deep learning, etc. can be used.

[First Embodiment]
Next, a first embodiment will be described with reference to the drawings.
<Configuration and processing>
As illustrated in FIG. 1, the system of the present embodiment includes a feature amount extraction device 11 and an estimation device 12. The feature amount extraction device 11 includes an eyeball information acquisition unit 111, a feature amount extraction unit 112, and a feature amount calculation unit 113, and the estimation device 12 includes an estimation unit 121. The feature amount calculation unit 113 includes a prediction model generation unit 113a and a prediction error feature amount calculation unit 113b. Each of the feature quantity extraction device 11 and the estimation device 12 includes, for example, a processor (hardware processor) such as a central processing unit (CPU) and a memory such as random-access memory (RAM) and read-only memory (ROM). Is a device configured by a predetermined or dedicated computer executing a predetermined program. The computer may include a single processor and memory, or may include a plurality of processors and memory. This program may be installed in a computer, or may be recorded in a ROM or the like in advance. In addition, some or all of the processing units are configured using an electronic circuit that realizes a processing function without using a program, instead of an electronic circuit (circuitry) that realizes a functional configuration by reading a program like a CPU. May be. In addition, an electronic circuit constituting one device may include a plurality of CPUs.

<< Eyeball Information Acquisition Unit 111 (FIG. 2: Step S111) >>
The eyeball information acquisition unit 111 acquires time-series information regarding “dynamic changes in eyes” at each discrete time of the subject 100, and sends the acquired time-series information regarding dynamic changes in the eyes to the feature amount extraction unit 112. Output. The acquired “dynamic change of the eye” may be the movement of the eyeball itself of the subject 100, the movement of the pupil, or both of them. The eyeball information acquisition unit 111 may acquire time-series information regarding dynamic changes of both eyes, or may acquire time-series information regarding dynamic changes of either eye.

  The time-series information regarding the “movement of the eyeball itself” of the subject 100 is obtained based on an image obtained by photographing the eye of the subject 100 with an imaging device (for example, an infrared camera). The eyeball information acquisition unit 111 performs, for example, image processing on a captured video, thereby obtaining a time series of eyeball positions for each frame (for example, a sampling interval of 1000 Hz) that is a predetermined time interval. Obtain as information. The eyeball information acquisition unit 111 may be realized by an imaging device and a computer that executes an image processing algorithm, or a computer that executes an algorithm that performs image processing on an image input from the imaging device using the imaging device as an external device. It may be realized by. Alternatively, the eyeball information acquisition unit 111 may measure the movement of the eyeball using an electric potential measurement method using electrodes, and may acquire time-series information regarding “movement of the eyeball itself” based on the measurement result. In this case, the eyeball information acquisition unit 111 may be realized by a measurement device (including electrodes) and a computer that executes an algorithm for calculating the position of the eyeball based on the potential measured by the measurement device, or the measurement device may be The external device may be realized by a computer that executes an algorithm for calculating the position of the eyeball based on the potential input from the measurement device. FIG. 3 illustrates the movement of one eyeball. The horizontal axis of FIG. 3 represents time [second], and the vertical axis represents the viewing angle [degree].

  The time-series information regarding the “pupil movement” of the subject 100 is obtained based on an image obtained by photographing the eyes of the subject 100 with an imaging device (for example, an infrared camera). For example, the subject 100 is allowed to gaze at one point, and the pupil at that time is imaged by an infrared camera. The eyeball information acquisition unit 111 acquires a time series of pupil sizes for each frame (for example, a sampling interval of 1000 Hz) by performing image processing on the captured video. For example, the eyeball information acquisition unit 111 can fit a circle to the pupil of an image obtained by photographing the pupil, and use the radius of the fitted circle as the pupil diameter. Since the pupil diameter fluctuates minutely, it is preferable that the eyeball information acquisition unit 111 uses a pupil diameter value that is smoothed (smoothed) for each predetermined time interval. FIG. 5 illustrates the movement of the pupils of the right eye and the left eye (change in pupil diameter). The horizontal axis of FIG. 5 represents time [second], and the vertical axis represents the pupil diameter. This pupil diameter is expressed as z-score where the average of all the pupil diameter data acquired at each time is 0 and the standard deviation is 1. However, the “time series information regarding pupil movement” acquired by the eyeball information acquisition unit 111 may not be the time series of the pupil diameter expressed in z-score, but may be the time series of the pupil diameter value itself. Alternatively, it may be a time series of the area and diameter of the pupil, or any time series of values corresponding to the size of the pupil.

<< Feature Extraction Unit 112 (Step S112) >>
The feature amount extraction unit 112 uses the feature amount based on the “eye dynamic change” in a predetermined time section F _t (partial time section) from the acquired time-series information regarding the “eye dynamic change” ( y _{t, 1} ,..., y _{t, d} ) (first feature amount) and feature amounts based on “dynamic change of eyes” in each of the predetermined time intervals F _{t−1 ,.} (y _t−1,1 ,..., y _{t−1, d} ),..., (y _{tp, 1} ,..., y _{tp, d} ) (second feature value) are extracted and output. Here, i = t, t-1,..., Tp is an index representing the discrete time corresponding to the time interval F _i , t is an index representing the discrete time, and d is the number of elements of the feature quantity, that is, the dimension. It is a positive integer, and p is an integer of 1 or more, but is preferably 2 or more. i and t correspond to a new discrete time as the magnitude of the value increases. p may be a constant or a variable. As illustrated in FIG. 6, the time interval F _t is the first time interval T ₁ , and the interval consisting of the p time intervals F _t−1 ,..., F _tp is the second time interval T ₂ . When the feature quantity is a scalar, d = 1, and when the feature quantity is a vector, d is an integer of 2 or more. For example, at each t, the feature amount extraction unit 112 uses a scalar or a vector having a feature amount based on at least one of “the movement of the eyeball” and “the movement of the pupil” of the subject 100 as a feature amount (y _{i, 1} , ..., y _{i, d} ) (where i = t, t-1, ..., tp).

Features based on “movement of the eyeball itself”:
Examples of the feature amount based on “movement of the eyeball itself” include feature amounts of “microsaccade” and “large saccade”. In the case of using a feature amount of "micro saccade" and "Large saccades", or the time interval F _i comprises a micro saccade only one always respectively, or set to include large saccades only one To do. At this time, in order to increase the accuracy of the model, it is desirable to prevent microsaccades and large saccades from being erroneously detected and omissions as much as possible.

Microsaccade features:
“Microsaccade” refers to minute jumping eye movements that appear in eye movements. When a person is gazing at a certain point, the eyeball does not stop moving completely, but three types of eye movements called drift (trend), trema, Micro saccade (may be called flick). Drift is a small smooth movement, trema is a very small high-frequency vibration, and microsaccade is a small jumping movement. A microsaccade is illustrated using FIG. In FIG. 3, the microsaccades MS _{1 to} MS ₃ are shown with bold lines. Microsaccades are eye movements that appear (involuntarily) regardless of the individual's intentions, in a state of gazing at a certain point, once in 1 to 2 seconds. That is. Microsaccades can be obtained from either the horizontal or vertical component of motion. In this embodiment, based on the property that the microsaccade is deflected in the horizontal direction, only the component in the horizontal direction is used for simplicity. However, the direction component of the microsaccade that can be used in the present invention is not limited to the horizontal direction. The “horizontal direction” is not limited to a direction parallel to the ground, and is a horizontal direction with respect to the face of the subject 100 (an eyeball arrangement direction, which may be referred to as a horizontal direction or a width direction). ) Or the direction defined as the horizontal direction in the eyeball information acquisition unit 111.

  For example, the feature quantity extraction unit 112 calculates a primary difference series for the time series of eyeball positions, and sets the time when the absolute value of the primary difference series exceeds a predetermined first threshold as the microsaccade start time. What is necessary is just to detect as (occurrence time). However, if the length of time that the absolute value of the primary difference series exceeds the predetermined threshold does not last longer than the predetermined value (usually about 3 ms), it is excluded from detection. Further, when a reference amplitude A described later is equal to or larger than a predetermined threshold (normal viewing angle of about 2 °), it is excluded from detection as a large saccade. The feature amount extraction unit 112 may use a moving average value in an appropriate range when calculating the primary difference series, for example, when it is determined that the acquired position information of the eyeball includes a lot of noise. The threshold used for detection is preferably a value about 6 times the standard deviation of the difference series.

Microsaccade feature values include a value Z based on the microsaccade generation timing, a value D corresponding to the motion direction, an absolute value of the reference amplitude A | A |, a maximum speed V _max , a duration D _m , and an overshoot. Absolute value of amplitude A _o | A _o |, overshoot speed V _o , rise time K, damping rate λ, damping coefficient ζ, natural angular frequency ω _n , per unit time (for example, 1 second) of microsaccade and which may or may not be of occurrences R _m. As described above, it is particularly desirable to use the value Z based on the microsaccade generation timing, the value D corresponding to the direction of motion, the absolute value | A | of the reference amplitude A, and the attenuation coefficient ζ.

The value Z based on the generation timing of the micro-saccades that occur in a time interval F _i, for example, may be a start time M _i time interval F _i, micro saccade corresponding to the reference become time interval RF _t May be the time difference | RM _t −M _i | between the occurrence time RM _t and the start time M _i , or the function value g (M _i ) or g of the start time M _i or the time difference | RM _t −M _i | (| RM _t −M _i |). The reference time interval RF _t corresponds to the time interval F _t , for example, RF _t = F _t−1 . The function value g (M _i ) or g (| RM _t −M _i |) is not limited. For example, the larger the representative value M _i or the time difference | RM _t −M _i | A function value of a function having no singular point can be used. Examples of the function value g (M _i ) are 1 / M _i and exp (−M _i ). “Exp” represents an exponential function based on the Napier number. Examples of g (| RM _t −M _i |) are 1 / | RM _t −M _i | and exp (− | RM _t −M _i |). However, when M _i and | RM _t −M _i | are 0, 1 / M _i and 1 / | RM _t −M _i | are ∞. Therefore, g (M _i ) and g (| RM _t −M _i |) may be set to 0 when M _i and | RM _t −M _i | are 0. The value Z based on the generation timing of the microsaccade generated in such a time interval F _i may be used as any element of the feature amount (y _{i, 1} ,..., Y _{i, d} ).

The value D corresponding to the movement direction of the microsaccade generated in the time interval F _i is a value determined for each movement direction. For example, the value D corresponding to the movement direction may take one of two values corresponding to the left and right movement directions, may take one of four values corresponding to the left, right, up and down movement directions, Any of the n values corresponding to the movement direction may be taken. As an example, the value D corresponding to the movement direction in the right direction (the direction from the left eye to the right eye) is set to the first value (for example, −1), and the movement direction in the left direction (the direction from the left eye to the right eye). A value D corresponding to the second value (for example, 1) is set. A value D corresponding to the motion direction of the microsaccade generated in such a time interval F _i may be used as any element of the feature amount (y _{i, 1} ,..., Y _{i, d} ).

と表される。 Next, the microsaccade reference amplitude A, maximum speed V _max , duration D _m , overshoot amplitude A _o , overshoot speed V _o , rise time K, and attenuation rate λ will be described with reference to FIG. To do.
(1) Reference amplitude A: the amount of movement when the movement of the eyeball by the microsaccade converges.
(2) Maximum speed V _max : The maximum speed until reaching the reference amplitude A + the overshoot amplitude A _o .
(3) Duration D _m : the length of the time interval in which microsaccade is occurring. The start time of the microsaccade is a time when the absolute value of the first-order difference series exceeds a predetermined threshold, and the end time of the microsaccade is the time when the first return to the reference amplitude A is reached after reaching the overshoot amplitude. .
(4) Overshoot amplitude A _o : The amount of the portion that exceeds (overshoots) the reference amplitude A by the microsaccade. The overshoot is a phenomenon in which the waveform protrudes beyond the reference amplitude A at the rising portion of the waveform, or the protruding waveform. In other words, the overshoot amplitude is the amount of the protruding portion.
(5) Overshoot speed V _o : This is the maximum speed when attempting to converge from the reference amplitude A + the overshoot amplitude A _o to the reference amplitude A.
(6) Rise Time K: a reference amplitude A + overshoot reaches the amplitude A _o (rise) time it takes. The reference amplitude A + overshoot amplitude A _o time to reach of the same value as the time it takes from the maximum velocity V _max reach speeds V _o overshoot.
(7) Attenuation rate λ: ratio of overshoot amplitude A _o to reference amplitude A It may be the ratio of overshoot speed V _o to maximum speed V _max ,

It is expressed.

と表される。固有角振動数ω_ｎはマイクロサッカードの応答の速さを表す指標に相当し、減衰係数ζはマイクロサッカードの応答の収束性を表す指標に相当する。 Microsaccade damping coefficient ζ and natural angular frequency ω _n are

It is expressed. The natural angular frequency ω _n corresponds to an index representing the response speed of the microsaccade, and the damping coefficient ζ corresponds to an index representing the convergence of the microsaccade response.

The feature extraction unit 112 fits the microsaccade damping coefficient ζ, natural angular frequency ω _n , and reference amplitude A by fitting the position of the eyeball while the microsaccade is occurring, and optimizes it by the least square method or the like You may calculate by doing.

  Since the value of the attenuation coefficient ζ of the microsaccade tends to change depending on the movement in the left-right direction, the feature amount extraction unit 112 uses the representative value of the attenuation coefficient of the left-side microsaccade, the micro-saccade in the right direction, The representative value of the saccade attenuation coefficient may be calculated separately.

The absolute value of the reference amplitude A of the micro-saccades as described above occurring in a time interval F _i | A |, the maximum velocity V _max, the absolute value of the duration D _m, of the overshoot amplitude A _o | A _o |, over At least one of the chute speed V _o , rise time K, damping rate λ, damping coefficient ζ, and natural angular frequency ω _{n is used} as any element of the feature quantity (y _{i, 1} ,..., Y _{i, d} ). Also good.

Large saccade features:
“Large saccade” means a jumping eye movement with a larger amplitude than that of microsaccade. Generally, when the amplitude is a viewing angle of 2 degrees or more, a large saccade is less than 2 degrees and a microsaccade is a microsaccade. The feature amount extraction unit 112 may detect the time when the reference amplitude A is equal to or greater than a predetermined threshold as the start time when the large saccade occurs. Examples of large saccade feature values include a value Z based on the occurrence timing of large saccade, a value D corresponding to the direction of motion, a reference amplitude A, a maximum velocity V _max , a duration D _m , an overshoot amplitude A _o , These are the overshoot speed V _o , rise time K, damping rate λ, damping coefficient ζ, natural angular frequency ω _n , number of occurrences R _m per unit time, number of occurrences, and the like. These specific examples are obtained by replacing “microsaccade” in the example of the feature amount of the microsaccade described above with “large saccade”. At least one of the features of the large saccade generated in the time interval F _i may be any element of the features (y _{i, 1} ,..., Y _{i, d} ). For the reasons described above, it is particularly desirable to use the value Z based on the generation timing of the large saccade, the value D according to the direction of motion, the absolute value | A | of the reference amplitude A, and the attenuation coefficient ζ.

Features based on “pupil movement”:
As illustrated in FIG. 5, the size of the pupil is not constant but changes. The size of the pupil is enlarged (mydriasis) by the pupil dilator that is controlled by the sympathetic nervous system, and contracted (miosis) by the pupil sphincter that is controlled by the parasympathetic nervous system. Changes in the size of the pupil are mainly classified into three types: light reflection, convergence reflection, and emotional changes. The light reflection is a reaction in which the size of the pupil changes in order to control the amount of light incident on the retina. A miosis occurs for strong light, and a mydriasis occurs in a dark place. Convergence reflex is a reaction in which the pupil diameter changes with the movement of both eyes inward or outward (convergence movement) when focusing. Arise. The emotional change is a reaction that occurs to the external stress regardless of any of the above. When my sympathetic nerve is dominant due to anger, surprise, or active activity, mydriasis occurs and relaxes. When the parasympathetic nerve dominates, miosis occurs. As the feature amount based on “pupil movement”, a feature amount of a miosis or a feature amount of a mydriasis can be used.

Miotic features:
The time at which the miosis starts (hereinafter referred to as the miosis start point) is detected by extracting the maximum point from the time series of pupil sizes. The time at which the miosis ends (hereinafter referred to as the miosis end point) is the earlier of the point at which the mydriasis start for the first time after the start of miosis or the point at which the blinks start for the first time after the start of miosis. Amplitude A _c of miosis is the difference in the pupil diameter from the miosis start point to miosis end point. The miosis duration D _c is the time difference from the miosis start point to the miosis end point. The average miosis speed V _c is (amplitude A _c ) / (duration D _c ). Amplitude A _c of miosis occurring in a time interval F _i, the duration D _c of miosis, velocity V _c of the average miosis, based on the "movements of the pupil" of the like number of occurrences of miosis in a time interval F _i It can be used as a feature amount. The feature amount extraction unit 112 uses at least one of the amplitude A _{c of} the miosis generated in the time interval F _i , the duration D _{c of} the miosis, the speed V _{c of the} average miosis, and the number of occurrences of miosis as a feature amount ( y _{i, 1} ,..., y _{i, d} ). When multiple miosis is detected in the time interval F _i , representative values of the miosis amplitude A _c , miosis duration D _c , and average miosis velocity V _{c obtained} for each miosis Any of these may be used as any element of the feature amount (y _{i, 1} ,..., Y _{i, d} ). In order to prevent erroneous detection due to noise, if the duration of miosis is less than a predetermined threshold (for example, 10 ms) or if the amplitude of miosis is less than a predetermined threshold, the miosis is excluded from detection. May be.

Mydriatic features:
The time at which the mydriatic starts (hereinafter referred to as the mydriatic start point) is detected by extracting a minimum point from the time series of the pupil diameter. The time at which the mydriatic ends (hereinafter referred to as the mydriatic end point) is the earlier of the time when the miosis starts for the first time after the start of mydriasis or the time when the blink starts for the first time after the start of mydriasis. Amplitude A _d mydriatic is the difference between the pupil diameter from mydriasis start point to mydriasis end point. The mydriatic duration D _d is the time difference from the mydriatic start point to the mydriatic end point. The average speed V _{d of the} mydriasis is (amplitude A _d ) / (duration D _d ). The amplitude A _{d of} the mydriasis generated in the time interval F _i , the duration of the mydriasis D _d , the average mydriatic velocity V _d , the number of occurrences of the mydriasis, etc. based on the `` pupil movement '' in the time interval F _i It can be used as a feature amount. The feature amount extraction unit 112 uses at least one of the amplitude A _{d of} the mydriasis generated in the time interval F _i , the duration D _{d of} the mydriasis, the average mydriatic velocity V _d , and the number of occurrences of the mydriasis as a feature amount ( y _{i, 1} ,..., y _{i, d} ). When multiple mydriasis are detected in the time interval F _i , representative values of the mydriatic amplitude A _d , mydriatic duration D _d , and average mydriatic velocity V _{d obtained} for each mydriatic Any of these may be used as any element of the feature amount (y _{i, 1} ,..., Y _{i, d} ). In order to prevent false detection due to noise, if the duration of mydriasis is less than or equal to a predetermined threshold (for example, 10 ms), or if the amplitude of mydriasis is less than or equal to a predetermined threshold, that mydriatic is excluded from detection. May be.

As described above, the relative amount between the value derived from the dynamic change of one eye (for example, the right eye) of the subject 100 and the value derived from the dynamic change of the other eye (for example, the left eye). May be any element of the feature value (y _{i, 1} ,..., Y _{i, d} ). As the value derived from the dynamic change of the eye, the above-described microsaccade feature amount, large saccade feature amount, miosis feature amount, mydriatic feature amount, or the like can be used. However, the value derived from the dynamic change of one eye and the value derived from the other dynamic change are the same type of feature amount. Specific examples of the “relative amount” are as described above.

<< Feature Quantity Calculation Unit 113 (Step S113) >>
The feature amount (y _{i, 1} ,..., Y _{i, d} ) (where i = t, t−1,..., Tp) is sent to the feature amount calculation unit 113. For example, d = 5, characterized by a value Z based on the microsaccade generation timing, a value D according to the direction of motion, an absolute value | A | of the reference amplitude A, a damping coefficient ζ, and a natural angular frequency ω _n In the case of a quantity element, the elements of the feature quantity (y _{i, 1} ,..., Y _{i, 5} ) are as follows. Thus the first feature quantity _{(y t, 1, ...,} y t, 5) and the second feature quantity _{(y t-1,1, ...,} y t-1,5), ..., (y t-p _{, 1} ,..., Y _{t-p, 5} ) are feature quantities of the same type.

The feature amount calculation unit 113 includes a first feature amount (y _{t, 1} ,..., Y _{t, d} ) obtained based on the dynamic change of the eye of the subject 100 in the first time interval T ₁ , Time series information (y _t−1,1 ,..., Y _{t−1, d} ) of the second feature amount obtained based on the dynamic change of the eye of the subject 100 in the second time interval T ₂ , ..., (y _{t-p, 1} , ..., y _{t-p, d} ), the feature value (y ^ _t-1,1 , ..., y ^ _{t-1, d in} the first time interval T ₁ estimated from ) And a feature quantity S _t (third feature quantity) corresponding to the degree of difference. Hereinafter, this process will be exemplified. In this embodiment, a vector autoregressive model is used, and (y _t-1,1 , ..., y _{t-1, d} ), ..., (y _{t-p, 1} , ..., y _{t-p, d} ) to ( An example of estimating y ^ _t-1,1 , ..., y ^ _{t-1, d} ) is shown. However, this does not limit the invention. In addition, "^" in (y ^ _t-1,1 , ..., y ^ _{t-1, d} ) should be written directly above "y", but "^" “Y” may be written on the upper right. Since the first feature value and the second feature value are not necessarily feature values having a normal distribution, when a vector autoregressive model is used as in this embodiment, the feature value is previously obtained by a method such as Box-Cox conversion. It may be normalized.

選択された係数c₁,…,c_d，φ_1,1 ⁽¹⁾,…,φ_d,d ⁽¹⁾,…,φ_1,1 ^(p),…,φ_d,d ^(p)は、予測誤差特徴量計算部１１３ｂに送られる。 << Prediction Model Generation Unit 113a (Step S113a) >>
The prediction model generation unit 113a obtains (y _{t ', 1} , ..., y _{t', d} ), ..., (y _{t'-p, 1} , ..., y _{t'- p, d} ) as input and coefficients c ₁ , ..., c _d , φ _1,1 ⁽¹⁾ , ..., φ _{d, d} ⁽¹⁾ , ..., φ _1,1 ^{(p )} , ..., φ _{d, d} ^(p) are calculated and output. Here, t ≧ t ′. For example, t ′ = t, t−1, t−2,..., Tw, and w is an integer of w <t. For example, tw is an initial value of the time index (for example, tw = 1). Coefficients c ₁ , ..., c _d , φ _1,1 ⁽¹⁾ , ..., φ _{d, d} ⁽¹⁾ , ..., φ _1,1 ^(p) , ..., φ _{d, d} ^(p) are, for example, To minimize the size of prediction error (prediction residual) ε _{t ', 1} , ..., ε _{t', d} for all t '= t, t-1, t-2, ..., tw Selected. For example, the coefficient c _{1 is set} so that the sum of ε _{t ′, 1} ,..., Ε _{t ′, d} and the function value of t ′ = t, t-1, t-2,. , ..., c _d , φ _1,1 ⁽¹⁾ , ..., φ _{d, d} ⁽¹⁾ , ..., φ _1,1 ^(p) , ..., φ _{d, d} ^(p) are set.

The selected coefficients c ₁ ,…, c _d , φ _1,1 ⁽¹⁾ ,…, φ _{d, d} ⁽¹⁾ ,…, φ _1,1 ^(p) ,…, φ _{d, d} ^(p) are The error is sent to the prediction error feature amount calculation unit 113b.

ただし、

であり、

であり、［・］^Ｔは［・］の転置である。Σ_ｔはy^_tの分散共分散行列であり、Σ_ｔのi行j列の要素Σ_ｉｊは以下の通りである。

ただし、Ｅ［・］は［・］の期待値を表す。 << Prediction Error Feature Quantity Calculation Unit 113b (Step S113b) >>
The prediction error feature quantity calculation unit 113b uses coefficients c ₁ , ..., c _d , φ _1,1 ⁽¹⁾ , ..., φ _{d, d} ⁽¹⁾ , ..., φ _1,1 ^(p) , ..., φ _{d , d} ^(p) , and feature quantity (y _{i, 1} , ..., y _{i, d} ) (where i = t, t-1, ..., tp) are input and correspond to t as follows: A feature quantity S _t (third feature quantity) is obtained and output.

However,

And

[•] ^T is the transpose of [•]. Σ _t is a variance-covariance matrix of y ^ _t , and elements Σ _ij of i rows and j columns of Σ _t are as follows.

However, E [•] represents the expected value of [•].

ここで、μはy_tの平均ベクトルであり、Σ_t’はy_tの分散共分散行列である。特徴量S_tは、第２特徴量から推定された確率分布における第１特徴量の出現確率が０に近いほど大きくなって１に近づき、出現確率が１に近いほど小さくなって０に近づく。つまり、特徴量S_tは、新奇な状態であって対象者１００の注意度合いや驚き度合いが高いほど大きな値となり、逆に、対象者１００の注意度合いや驚き度合いが低いほど小さな値をとる。 The feature amount _St is the first feature amount (y _{t, 1} ,..., Y _{t, d} ) obtained based on the dynamic change of the eye of the subject 100 in the first time interval T ₁ . Time series information (y _t−1,1 ,..., Y _{t−1, d} ) of the second feature amount obtained based on the dynamic change of the eye of the subject 100 in the 2-hour section T _{2 ,.} , (y _{t-p, 1} , ..., y _{t-p, d} ), the feature value (y ^ _t-1,1 , ..., y ^ _{t-1, d} ) in the first time interval T ₁ It becomes a value according to the probability of appearing in the distribution of (Fig. 6). Here, the expression (1) is generally based on the probability density function P _t of the multivariate normal distribution of the feature quantities (y _{t, 1} ,..., Y _{t, d} ) as follows.

Here, μ is an average vector of y _t , and Σ _t ′ is a variance covariance matrix of y _t . Feature amount S _t is the probability of occurrence of the first feature quantity at the estimated probability distribution of the second feature quantity approaches 1 increases closer to 0, the occurrence probability approaches 0 becomes smaller toward 1. That is, the feature amount S _t is a novel state attention degree and surprises the degree of the subject 100 becomes higher large value, conversely, attention degree and surprises the degree of the subject 100 takes a low enough small value.

<< Estimation unit 121 (step S121) >>
The feature amount _St is sent to the estimation unit 121 of the estimation device. Estimation unit 121, using at least the feature amount S _t, estimates the psychological state of the subject 100 such as attention degree and / or surprise degree, and outputs the estimation result.

There is no limitation on the method for estimating the psychological state of the subject 100. As a simple method, the estimation unit 121, the feature amount S _t and the threshold Th ₁ (however, _1> Th 1> 0) are compared, and the subject 100 if S _t ≧ Th ₁ caution condition and / or An estimation result indicating that the user is in a surprise state is output, and if not, an estimation result indicating that the subject 100 is not in an attention state and / or a surprise state is output.

Estimation unit 121 the feature amount S _t and a plurality of threshold Th _max, ..., Th _{1 (however, 1> Th max>...>} Th 1> 0, max is an integer of 2 or more) is compared with the feature amount S _t May be output as an estimation result indicating a level indicating the degree of attention and / or the degree of surprise of the subject 100. Alternatively, the estimation unit and 121 feature amount S _t itself may output a monotonic function (monotone increasing function or a monotonically decreasing function) for the feature amount S _t as an estimation result indicating the attentional state and / or surprise state.

In the feature amount S _t and the above-mentioned time interval F _t using a vector VF _t be either preparative elements of the feature quantity based on the "dynamic changes in the eye", the subject 100 such as attention degree and / or surprise degree A psychological state may be estimated. For example, a vector VF _t at each t clustered by k-means or the like, may output information indicating the cluster to which each vector VF _t belongs as an estimation result.

[Second Embodiment]
In the first embodiment, the prediction model generation unit 113a performs coefficients c ₁ ,..., C _d , φ _1,1 ⁽¹⁾ , ..., φ _{d, d} ⁽¹⁾ , ..., φ _1,1 ^{(p) in} real time. , ..., φ _{d, d} ^(p) were updated. However, the coefficients c ₁ , ..., c _d , φ _1,1 ⁽¹⁾ , ..., φ _{d, d} ⁽¹⁾ , ..., φ _1,1 ^(p) , ..., φ _{d, d} ^(p) May be calculated and these may not be updated. In the following, differences from the items described so far will be mainly described, and the items already described are referred to by the same reference numerals and the description is simplified.

As illustrated in FIG. 1, the system according to the present embodiment includes a feature amount extraction device 21 and an estimation device 12. The feature quantity extraction device 21 includes an eyeball information acquisition unit 111, a feature quantity extraction unit 112, and a feature quantity calculation unit 213. The feature amount calculation unit 213 includes a prediction model generation unit 113a, a prediction error feature amount calculation unit 113b, and a storage unit 213c. In this embodiment, at the pre-processing stage, the prediction model generation unit 113a is obtained at each past t ′ (where t> t ′) (y _{t ′, 1} ,..., Y _{t ′, d} ),. , (y _{t'-p, 1} , ..., y _{t'-p, d} ) as inputs, coefficients c ₁ , ..., c _d , φ _1,1 ⁽¹⁾ , ..., φ _{d, d} ⁽¹⁾ ,..., φ _1,1 ^(p) ,..., φ _{d, d} ^(p) are calculated and stored in the storage unit 213c. In the process at each t, after steps S111 and 112 are executed, the process of step S113b is executed without executing the process of step S113a. However, in step S113b, the prediction error feature quantity calculation unit 113b extracts coefficients c ₁ ,..., C _d , φ _1,1 ⁽¹⁾ ,..., Φ _{d, d} ⁽¹⁾ ,. φ _1,1 ^(p) ,..., φ _{d, d} ^(p) , and feature quantity (y _{i, 1} ,..., y _{i, d} ) sent from the feature quantity extraction unit 112 (where i = t , t−1,..., tp) as inputs, and obtains and outputs a feature quantity S _t (third feature quantity) corresponding to t as described above. Thereafter, the process of step S121 is executed. Thereby, the real-time prediction model generation process can be omitted, and the process can be speeded up.

[Third Embodiment]
This embodiment is a modification of the first or second embodiment. In this embodiment, at least the feature quantity _St is applied to a prediction model learned in advance, and the psychological state of the subject 100 such as the degree of attention and / or the degree of surprise is estimated.

As illustrated in FIG. 1, the difference between the third embodiment and the first and second embodiments is that the system includes an estimation device 32 instead of the estimation device 12. The estimation device 32 includes an estimation unit 321, a model generation unit 322, and a storage unit 323. In this embodiment, at the stage of pre-processing, the model generating unit 322, each of the past t "(although, t>t") including at least the feature amount feature amounts S _{t "obtained} in (e.g., the feature amount S _{t "} Or vector VF _t" ) as an input, and a subject 100 such as a degree of attention and / or a degree of surprise from a feature amount (for example, feature amount _St or vector VF _t ) including at least the feature amount St obtained at _t. A psychological state prediction model that outputs an estimation result representing the psychological state of the human being is generated, and information (model parameter or the like) specifying the psychological state prediction model is stored in the storage unit 323. This learning is a feature that includes at least the feature amount _{St ″} Supervised learning using learning data in which a quantity and a correct answer label representing a psychological state are associated, or unsupervised learning using only a feature quantity including at least a feature quantity _{St ”} as learning data. No, but just The type of feature quantity used as learning data is the same as the type of feature quantity obtained by t. Examples of psychological state prediction models include support vector machines, hidden Markov models, neural networks, and deep learning. In the processing at each t, the feature amount obtained at t is applied to the psychological state prediction model set based on the information extracted from the storage unit 323 by the estimation unit 321 after the processing at step S113b. applying the feature quantity including at least a S _t, and outputs an estimation result indicating the psychological state of the subject 100 such as attention degree and / or surprising degree (Figure 2: step S321).

[Experimental example]
Next, an example will be introduced in which a feature quantity S _t (surprise value: surprise) is calculated from the microsaccade measured in an actual experiment using Formula (1), and an evaluation based on a psychophysical experiment is performed. In the example experiment, a psychophysical evaluation of the feature amount S _t, was measured according the reaction time to the detection of the sound column disappears.

<Psychophysical experiment: Detection of loss of sound sequence>
Six healthy adults (mean age 27.6 ± 1.2 years, 2 women) participated in the experiment. In the experiment, 1000 Hz pure tone (duration 200 ms) was presented at random times at regular (upper figure in FIG. 7A) or irregular (lower figure in FIG. 7A) intervals. The subject was instructed to press the button as soon as possible when he felt that the presentation of the sound string had ended. The number of pure tones presented was changed randomly at each trial using three conditions of 8, 10, and 12. The time intervals between presentations of pure tones (interval of presentation start time) are all 500 ms (upper figure in FIG. 7A: rule presentation conditions), 500 ± 50 ms, 500 ± 100 ms, 500 ± 150 ms, 500 ± 200 ms (figure A total of 5 conditions (lower figure of 7A: irregular presentation conditions) were randomly switched for each trial. In the irregular presentation condition, the presentation time interval follows a uniform distribution within the above-mentioned width, and is set so that the same time interval does not exist in one trial. However, in order to unify the conditions for the reaction, the presentation time interval of the last sound of each trial and the previous sound was set to 500 ms. Ten repetitions were performed for each condition of the presentation time interval and the number of presentations, and a total of 150 trials were performed for each subject.

音列消失検知にかかる反応時間は、規則的な音列ほど早かった。規則的な間隔で呈示される音列は、消失のタイミングがはっきりとしており、不規則的な間隔で呈示される音列に比べて消失の検知にかかる反応時間が早くなるものと考えられる。また、音列消失検知にかかる反応時間は、消失までの呈示回数が多いほど早かった。これは試行の繰り返しによる効果を反映しているものと考えられる。すなわち、呈示回数が少ないほど音列が消失する確率は経験的には低くなるため、判断に遅れが生じるものと考えられる。 Table 2 shows the average reaction time taken to detect the disappearance of the sound string under each condition. However, the average reaction time required to detect the disappearance of the sound string was indicated by “average ± standard deviation”, and “second” was used as the unit.

The reaction time required to detect the disappearance of the sound string was faster for regular sound strings. It is considered that the sound strings presented at regular intervals have clear timings of disappearance, and the reaction time required for detection of disappearance is faster than the sound strings presented at irregular intervals. In addition, the reaction time required to detect the disappearance of the sound string was faster as the number of times until the disappearance was increased. This is considered to reflect the effect of repeated trials. That is, as the number of presentations decreases, the probability that the sound string disappears becomes empirically low, and it is considered that the determination is delayed.

FIG. 7B shows the feature quantity S _t (surprise value: surprise) obtained from the microsaccade measured before and after the sound string disappearance time is zero. As shown in FIG. 7B, although it decreases once immediately after the disappearance of the sound train, it can be seen that a large surprise is detected over 1500-2500 ms. In this case, the condition (presentation number, distribution width of the presentation time interval) and according Mean reaction times to the sound column disappeared in the formula (1) of the calculated micro saccade by the feature amount S _t of (1500-2500Ms) The average value showed a weak negative correlation (r = -0.42). Therefore, humans perceive the degree of "surprise", the feature amount S _t obtained from the micro saccade is considered to have a certain degree expressed.

ただしγは任意の定数である（例えば、０＜γ≦１）。また、前述の式(2)は第１特徴量が多変量正規分布に従っていることを前提としている。しかし、これは本発明を限定するものではない。特徴量S_tの計算にあたっては、第２特徴量の時系列情報に基づいて推定された分布における第１特徴量の確率P_tを計算した上で、一般に

とすることもできる。また、少なくとも特徴量S_tを用い、注意度合いや驚き度合い以外の対象者１００の心理状態を推定してもよい。例えば、少なくとも特徴量S_tを用い、対象者１００のリラックス度合いや緊張度合いなどの心理状態を推定してもよい。 [Other variations]
In addition, this invention is not limited to the above-mentioned embodiment. For example, instead of the equation (1), the feature amount S _t (third feature amount) may be obtained by the following equation.

However, γ is an arbitrary constant (for example, 0 <γ ≦ 1). Further, the above-described equation (2) is based on the premise that the first feature amount follows a multivariate normal distribution. However, this does not limit the invention. In the calculation of the feature amount S _t, in terms of calculating the first feature amount of the probability P _t in the estimated distribution on the basis of the time-series information of the second feature quantity, generally

It can also be. Further, at least the feature amount _St may be used to estimate the psychological state of the target person 100 other than the attention level and the surprise level. For example, using at least the feature amount S _t, it may estimate the psychological state of such relaxation degree and tension degree of the subject 100.

  The various processes described above are not only executed in time series according to the description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. Needless to say, other modifications are possible without departing from the spirit of the present invention.

  When the above configuration is realized by a computer, the processing contents of the functions that each device should have are described by a program. By executing this program on a computer, the above processing functions are realized on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium. An example of a computer-readable recording medium is a non-transitory recording medium. Examples of such a recording medium are a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, and the like.

  This program is distributed, for example, by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own storage device, and executes a process according to the read program. As another execution form of the program, the computer may read the program directly from the portable recording medium and execute processing according to the program, and each time the program is transferred from the server computer to the computer. The processing according to the received program may be executed sequentially. The above-described processing may be executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by an execution instruction and result acquisition without transferring a program from the server computer to the computer. Good.

  In the above embodiment, the processing functions of the apparatus are realized by executing a predetermined program on a computer. However, at least a part of these processing functions may be realized by hardware.

11, 21 Feature amount extraction device 12, 32 Estimation device

Claims (11)

  Time series information of the first feature value obtained based on the dynamic change of the eye in the first time interval and the second feature value obtained based on the dynamic change of the eye in the second time interval A feature quantity extraction device that obtains a third feature quantity according to the degree of difference from the feature quantity in the first time interval estimated from. The feature amount extraction apparatus according to claim 1,
The feature quantity extraction device, wherein the first feature quantity and the second feature quantity include a feature quantity based on a saccade of an eyeball. The feature quantity extraction device according to claim 2,
The feature quantity extraction device, wherein the first feature quantity and the second feature quantity include a feature quantity based on at least one of an eyeball movement direction and an eyeball movement attenuation coefficient. The feature quantity extraction device according to claim 2 or 3,
The feature quantity extraction device, wherein the first feature quantity and the second feature quantity include a feature quantity based on a generation timing of an eyeball saccade. The feature quantity extraction device according to any one of claims 1 to 4,
The first feature amount and the second feature amount include a feature amount based on a relative amount between a value derived from a dynamic change of one eye and a value derived from a dynamic change of the other eye. Quantity extraction device. The feature quantity extraction device according to any one of claims 1 to 5,
The first feature amount and the second feature amount include feature amounts based on a relative amount between a value derived from the movement of the pupil of one eye and a value derived from the movement of the pupil of the other eye. apparatus. The feature quantity extraction device according to any one of claims 1 to 6,
The third feature quantity is a feature quantity extraction device that represents at least one of a degree of attention and a degree of surprise.   The first feature amount estimated from time-series information of at least a first feature amount based on a dynamic change of the eye in the first time interval and a second feature amount based on the dynamic change of the eye in the second time interval. An estimation device that estimates at least one of a degree of attention and a degree of surprise using a third feature amount corresponding to a degree of difference from a feature amount in a time interval.   Time series information of the first feature value obtained based on the dynamic change of the eye in the first time interval and the second feature value obtained based on the dynamic change of the eye in the second time interval A feature quantity extraction method for obtaining a third feature quantity according to the degree of difference from the feature quantity in the first time interval estimated from   The first feature amount estimated from time-series information of at least a first feature amount based on a dynamic change of the eye in the first time interval and a second feature amount based on the dynamic change of the eye in the second time interval. An estimation method for estimating at least one of a degree of attention and a degree of surprise using a third feature amount corresponding to a degree of difference from a feature amount in a time interval.   The program for functioning a computer as the feature-value extraction apparatus in any one of Claim 1 to 7, or the estimation apparatus of Claim 8.

Images