JP2019111291A - Attention level estimation device, attention level estimation model learning device, attention level estimation method, attention level estimation model learning method, and program - Google Patents

Abstract

To estimate an attention level on the basis of microsaccade.SOLUTION: A model storage unit 10 stores information indicating a reference attention level and information indicating a decrease in an attention level. An eyeball information acquisition unit 11 acquires a dynamic change in an eye of a subject. A microsaccade detection unit 12 detects a time of occurrence of a microsaccade from the dynamic change in the eye. An estimation unit 13 estimates the attention level from the time of the occurrence of the microsaccade on the basis of the information indicating the reference attention level and the information indicating the decrease in the attention level.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a technique for estimating a state of attention (attention level) based on movement of a human eye (microsaccade).

  Non-Patent Document 1 discloses that there is a correlation between the direction of the microsaccade in the task of “character identification” and the speed of reaction to the presented task. Specifically, it has been reported that the reaction speed of the subject to the presented task becomes slow near the time of occurrence of each microsaccade. In addition, it is disclosed that the frequency of occurrence of microsaccade decreases as the task becomes difficult, and the direction of the microsaccade matches well with the direction of attention.

Alexander Pastukho and Jochen Braun, “Rare but precious: Microsaccades are highly informative about attentional allocation”, Vision Research, Vol. 50, Issue 12, pp. 1173-1184, 2000.

  Although the non-patent document 1 discloses the correlation between the direction of the microsaccade and the direction of the attention, the correlation between the state of attention (attention level) and the microsaccade is not clarified. Therefore, it was not possible to estimate the state of attention that changes from moment to moment based on the microsaccade and the movement characteristics (response speed and accuracy of the response) accompanying it.

  An object of the present invention is to realize a technique for estimating a level of caution based on a microsaccade in view of the above-described points.

  In order to solve the above problems, the attention level estimation device according to the first aspect of the present invention includes a model storage unit that stores information indicating a reference attention level and information indicating a decrease in attention level; The eye information acquisition unit that acquires the dynamic change of the micro-saccade detecting unit that detects the micro-saccade occurrence time from the dynamic change of the eye, the information indicating the reference caution level and the decrease in the alert level And an estimation unit configured to estimate the attention level from the micro-saccade occurrence time based on the information.

  In order to solve the above problems, the attention level estimation model learning device according to the second aspect of the present invention relates to a display control unit for displaying a predetermined image on a display device, and a dynamic change of the subject's eyes. An eye information acquisition unit for acquiring sequence information, a feature extraction unit for extracting a feature based on microsaccades from time-series information on dynamic change of eyes, and a predetermined process performed by a subject in response to a predetermined image And a measuring unit that measures an index representing the reaction rate from the time when the predetermined image was displayed and the time when the predetermined input was made, and the feature amount and reaction rate based on the microsaccade And a learning unit that learns information representing a decrease in the attention level due to the microsaccade based on the index.

  According to the present invention, it is possible to estimate the ever-changing attention level based on the micro saccade.

FIG. 1 is a diagram for explaining the attention level lowering function. FIG. 2 is a diagram illustrating a functional configuration of the attention level estimation device. FIG. 3 is a diagram illustrating the processing procedure of the attention level estimation method. FIG. 4 is a conceptual diagram for illustrating learning processing and estimation processing. FIG. 5A is a conceptual diagram for explaining the movement of the fixation point. FIG. 4B is a view for explaining how time-series data of viewing angle is converted into a marked point process. FIG. 4C is a diagram for explaining a feature amount based on microsaccades. FIG. 6 is a diagram illustrating a functional configuration of the attention level estimation model learning device. FIG. 7 is a diagram illustrating a processing procedure of the attention level estimation model learning method. FIG. 8A is a diagram showing experimental results for each trial of a certain subject. FIG. 8B is a diagram showing experimental results of the entire subject.

  Hereinafter, embodiments of the present invention will be described in detail. In the drawings, components having the same functions are denoted by the same reference numerals and redundant description will be omitted.

<Principle>
It is considered that the reaction speed to a task or the accuracy of the reaction when a human performs a predetermined task depends on the state (attention level) of the attention to the task. In other words, the higher the attention level, the higher the reaction speed to the task and the accuracy of the reaction tend to be. On the other hand, it is known that when a microsaccade occurs, the level of attention drops at the time near the point where the attention is directed, and the reaction speed to the task and the accuracy of the reaction decrease. Therefore, the present invention proposes a method of modeling a level of attention that changes from moment to moment, focusing on microsaccades. This makes it possible to estimate the attention level at each time.

Attention level modeling
Let f ₀ (t) be a function representing an average attention level not considering microsaccades. Here, t represents time. Hereinafter, f ₀ (t) will be referred to as a reference attention level function. The reference attention level function f ₀ (t) may be a function that maintains a value near the reference attention level, and may be, for example, a fixed value or a sine wave that repeatedly increases and decreases around the fixed value. May be a function. The design (learning) method of the reference attention level function f ₀ (t) will be described later.

Let L (τ) be a function representing the decrease in the level of attention accompanying the occurrence of microsaccades. L (τ) is a function that monotonously decreases the attention level immediately after the time at which the microsaccade occurs and then gradually increases the attention level gradually with the passage of time. τ represents an elapsed time from the occurrence time of the corresponding microsaccade. Hereinafter, L (τ) will be referred to as a caution level lowering function. The attention level reduction function L (τ), as shown in FIG. 1, has a large gradient when monotonically decreasing (ie, the attention level drops sharply) and a small gradient when monotonically increasing (ie, the attention level gently). Is preferred) function. For example, the attention level lowering function L _i (τ) corresponding to the microsaccade generated at time t _i is a function as follows.

Here, α and l _peak are parameters that determine the form of the function L _i (τ). As shown in FIG. 1, α is the time until the attention level drops most after the microsaccade occurrence, and l _peak is the decrease width when the attention level falls most. α and l _peak are expressed by the following function using micro _saccade parameters A _i , ζ _i and ω _i .

Incidentally, A _i is the amplitude of the micro-saccades subjects that occurred at time t _i, ζ _i is the damping coefficient, omega _i represents a natural angular frequency. Details of the micro saccade parameters will be described later.

For example, the attention level L _i of the subject can be estimated by the following equation using the parameters (τ _i , A _i , ζ _i , ω _i ) of the latest microsaccades. Here, the parameters A _i , ζ _i and ω _i of the latest microsaccades are the amplitude, the attenuation coefficient, and the natural angular frequency of the microsaccades generated most recently at the time of the evaluation level of attention level. , Τ _i represents the time difference from the microsaccade to the evaluation time.

The parameters (c ₁ , c ₂ ,..., c ₈ ) for determining α and l _peak are set for each individual, and for example, an optimal value estimated by learning described later may be set. .

The attention level model f (t) taking all microsaccades into consideration is compared with the reference attention level function f ₀ (t) at each occurrence time t ₀ , t ₁ , t ₂ , ... of the microsaccade. The level reduction function L _i (τ) (i = 0, 1, 2,...) Is modeled as a function obtained by multiplying the convolutional function.

δ (t) is a delta function, which satisfies the following properties.

First Embodiment
The first embodiment of the present invention uses the attention level estimation model that learns the relationship between the feature value based on the dynamic change of the eye of the subject and the attention level, and the target is obtained from the dynamic change of the subject's eye. Apparatus and method for estimating a level of attention of a person.

  The “subject” may be a human or non-human animal as long as the eye has a dynamically changing eye. The “dynamic change of the eye” may be the movement of the eye itself (the change of the position of the eye over time) or the movement of the pupil (the change of pupil diameter over time). The “feature amount” may be anything, may be a scalar, or may be a vector composed of a plurality of elements. The “feature amount” includes, for example, a feature amount (information corresponding to the feature of “saccade”) based on “saccade” appearing in the movement of the eye. The feature quantities based on "saccades" appearing in eye movement include feature quantities based on eye movement direction, feature quantities based on absolute value of eye movement amplitude, feature quantities based on eye movement attenuation coefficient, eye movement The feature quantity based on the natural angular frequency, the feature quantity based on the generation timing of the saccade of the eyeball, and the like can be exemplified. "Saccade" includes micro saccade and large saccade, but in the present embodiment, an example of micro saccade will be described. The “feature amount” is a value derived from a dynamic change of one eye (eg, the right eye) of the same “subject” and a value derived from a dynamic change of the other eye (eg, the left eye) It may also include feature quantities based on relative quantities with The “relative amount between α and β” is, for example, the difference between α and β, a value obtained by subtracting β from α, a value obtained by subtracting α from β, a value obtained by dividing α by β, or dividing β by α Values, or any function value of them. The “feature value based on relative value” is, for example, “relative value” or its function value, “relative value” or a vector having the function value as an element, or any function value thereof.

  The “attention level estimation model” is a model that represents a relationship between a “feature amount” variable and an information variable corresponding to the “attention level” based on the “dynamic change of eyes”. The “attention level estimation model” may be sequentially updated when obtaining the “estimation result”, or may be obtained in advance before obtaining the “estimation result”. The “attention level estimation model” is, for example, a statistical model or a probabilistic model, and is a model that represents a conditional distribution of information variables corresponding to the “attention level” when a “feature amount” variable is given. It may be a state space model representing a relationship between a variable of “feature amount” and information corresponding to “attention level”. For example, when the information corresponding to the "attention level" is r, the attention level estimation model represents 表 す that represents a feature based on the dynamic change of the eye (for example, a mark 表 す that represents a feature of a microsaccade) The statistical model (for example, h (r,)) itself) obtained based on the frequency h (r,)) (for example, the conditional intensity function which is the occurrence rate per unit time) of occurrence of the event having. In addition, “attention level estimation model” using multiple regression analysis, k-means, support vector machine (SVM), simple clustering, hidden Markov model, neural network, deep learning, etc. may be used.

  The attention level estimation device 1 according to the first embodiment includes a model storage unit 10, an eye information acquisition unit 11, a microsaccade detection unit 12, and an estimation unit 13, as shown in FIG. The attention level estimation device 1 executes the processing of each step shown in FIG. 3 to implement the attention level estimation method of the first embodiment.

  The attention level estimation device 1 is configured by reading a special program into a known or dedicated computer having, for example, a central processing unit (CPU: Central Processing Unit), a main storage device (RAM: Random Access Memory), etc. It is a special device. The caution level estimation device 1 executes each process, for example, under the control of the central processing unit. The data input to the caution level estimation device 1 and the data obtained by each process are stored, for example, in the main storage device, and the data stored in the main storage device is read out to the central processing unit as necessary. Is used for other processing. At least a part of each processing unit of the attention level estimation device 1 may be configured by hardware such as an integrated circuit. Each storage unit included in the attention level estimation device 1 is, for example, a main storage device such as a random access memory (RAM), an auxiliary storage device including a semiconductor memory element such as a hard disk, an optical disk, or a flash memory. Alternatively, it can be configured by middleware such as a relational database or key value store.

  The attention level estimation method performed by the attention level estimation device 1 according to the first embodiment will be described below with reference to FIG.

The model storage unit 10 stores information representing a reference attention level and information representing a decrease in attention level due to the micro saccade. The information representing the reference attention level is a parameter of the above-mentioned reference attention level function f ₀ (t). The information representing the decrease in the attention level due to the microsaccade is a parameter of the above-mentioned attention level decrease function L (τ). As described above, since the parameters α and l _peak of the attention level lowering function L (τ) are given by the equations (2) and (3), the parameters of the equations (2) and (3) (c ₁ , c ₂ , ..., c ₈ ) may be stored in the model storage unit 10, or α and l _peak may be stored. The information indicating the reference attention level and the information indicating the decrease in attention level by the micro saccade may be given in advance by some method, for example, it is determined by learning using the attention level estimation model learning device 2 described later. Can.

  In step S11, the eye information acquisition unit 11 acquires information on the dynamic change of the eye of the subject 9 at each discrete time t. The “dynamic change of the eye” may be the movement of the eye itself (the change of the position of the eye over time), the movement of the pupil (the change of pupil diameter over time), or both of them. May be The eye information acquisition unit 11 may acquire time-series information on dynamic change of both eyes, or may acquire time-series information on dynamic change of one of the eyes. The eye information acquisition unit 11 outputs the acquired information on the dynamic change of the eye to the micro saccade detection unit 12.

  The time-series information on the “movement of the eyeball itself” of the subject 9 is obtained based on an image obtained by photographing the eye of the subject 9 with an imaging device (for example, an infrared camera). The eye information acquisition unit 11 performs image processing on the captured image, for example, to obtain a time series of eye position in each frame (for example, a sampling interval of 1000 Hz) which is a predetermined time section, a time series related to the movement of the eye Acquire as information. The eye information acquisition unit 11 may be realized by an imaging device and a computer that executes an image processing algorithm, or a computer that executes an algorithm that processes an image input from the imaging device using the imaging device as an external device. It may be realized by Alternatively, the eyeball information acquiring unit 11 may measure the movement of the eyeball using the potential measurement method using the electrode, and may acquire the time-series information on the “movement of the eyeball itself” based on the measurement result. In this case, the eye information acquisition unit 11 may be realized by a measuring device (including an electrode) and a computer that executes an algorithm for calculating the position of the eye based on the potential measured by the measuring device. The external device may be realized by a computer or the like that executes an algorithm that calculates the position of the eyeball based on the potential input from the measurement device.

  The time-series information on the “pupil movement” of the subject person 9 is obtained based on the image obtained by photographing the eye of the subject person 9 with an imaging device (for example, an infrared camera). In this case, the eye information acquisition unit 11 performs image processing on the captured image to acquire a time series of pupil sizes for each frame (for example, a sampling interval of 1000 Hz) as time series information on the movement of the pupil. . The eye information acquisition unit 11 can, for example, fit a circle to the pupil of an image obtained by photographing the pupil, and use the radius of the fitted circle as the pupil diameter. Since the pupil diameter fluctuates minutely, it is preferable that the eye information acquisition unit 11 use the value of the pupil diameter smoothed for each predetermined time interval. The “time-series information on the movement of the pupil” acquired by the eye information acquisition unit 11 may be any time-series of values corresponding to the size of the pupil. For example, “time-series information on pupil movement” may be a time-series of pupil diameter represented by z-score, or may be a time-series of pupil diameter values themselves, or pupil area It may be a time series of or diameter.

  In step S12, the microsaccade detection unit 12 detects the presence or absence of the microsaccade based on the information on the dynamic change of the eye output by the eye information acquisition unit 11 every time t. When detecting the occurrence of the microsaccade, the microsaccade detection unit 12 outputs the time (that is, the occurrence time of the microsaccade) to the estimation unit 13.

The microsaccade refers to fine jumping eye movement that appears in the movement of the eye. When a human is gazing at a certain point, the eyeball is not completely stopping movement, and three types of eye movement called drifting (also called drift, trend), trema, It is a micro-saccade (it may be called Flick). Drift is a small smooth movement, Trema is a very small high frequency oscillation, a micro saccade is a small jumping movement. The microsaccade is illustrated using FIGS. In the top row of FIG. 4 and FIG. 5A, the micro saccades MS _{1 to} MS ₃ are shown with bold lines. A micro-saccade is an eye movement that appears (involuntarily) about once in 1 to 2 seconds while focusing on a certain point, and it is a small jumping movement It is The microsaccade can be obtained from either the horizontal or vertical component of the motion. In the present embodiment, only the horizontal component is used for simplicity, based on the property of the microsaccade to be deflected in the horizontal direction. However, the directional component of the microsaccade that can be used in the present invention is not limited to the horizontal direction. The “horizontal direction” does not mean to limit the direction parallel to the ground, but may be the horizontal direction with respect to the face of the target person 9 (the arrangement direction of the eyeballs, and may be referred to as the lateral direction or width direction). And the eye direction information acquisition unit 11 includes the direction defined as the horizontal direction.

In step S13, the estimation unit 13 generates the microsaccade output from the microsaccade detection unit 12 based on the information indicating the reference attention level stored in the model storage unit 10 and the information indicating the decrease in attention level. Estimate the attention level from the time. More specifically, the reference attention level function f ₀ (t) is derived from information representing the reference attention level, and the attention level reduction function L _i (τ) is derived from the information representing a drop in attention level due to the microsaccade generated at time t _i . Function, and a function of the attention level reduction function L _i (τ) (i = 0, 1, 2,...) At each occurrence time t ₀ , t ₁ , t ₂ ,. Based on equation (5), an estimated value f (t) of the attention level at the time to be estimated is obtained using the level function f ₀ (t).
For example, when (c ₁ , c ₂ ,..., C ₈ ) and the reference attention level function f ₀ (t) are stored in the model storage unit 10, the estimation unit 13 reads them from the model storage unit 10. it parameter _{(c 1, c 2, ...} , c 8) and a reference note level function f ₀ (t), determined from information on the dynamic changes in the subject's eye 9 acquired at an eyeball information acquiring unit 11 From the feature quantities A _i , ζ _i , ω _{i of} all the microsaccades, estimated values f of the attention level are obtained by the equations (2) to (6).

Second Embodiment
The information representing the reference attention level stored in the model storage unit 10 of the attention level estimation device 1 and the information representing a decrease in the attention level due to the micro saccade may be given in advance by any method, for example, as described below The attention level estimation model learning device 2 can perform the learning by learning.

  As shown in FIG. 6, the attention level estimation model learning device 2 of the second embodiment includes a display control unit 21, an eye information acquisition unit 22, a feature quantity extraction unit 23, an input unit 24, a measurement unit 25, a learning unit 26, And a model storage unit 10. The attention level estimation model learning method of the second embodiment is realized by the attention level estimation model learning device 2 executing the processing of each step shown in FIG. 7.

  The attention level estimation model learning device 2 is configured by reading a special program into a known or dedicated computer having, for example, a central processing unit (CPU: Central Processing Unit), a main storage device (RAM: Random Access Memory), etc. Is a special device. The attention level estimation model learning device 2 executes each process, for example, under the control of the central processing unit. The data input to the attention level estimation model learning device 2 and the data obtained by each process are stored, for example, in the main storage device, and the data stored in the main storage device is read to the central processing unit as necessary. It is issued and used for other processing. At least a part of each processing unit of the attention level estimation model learning device 2 may be configured by hardware such as an integrated circuit. Each storage unit included in the attention level estimation model learning device 2 is, for example, an auxiliary storage configured by a main storage device such as a RAM (Random Access Memory) or a semiconductor memory device such as a hard disk or an optical disk or a flash memory. It can be configured by a device or middleware such as a relational database or key value store.

  The attention level estimation model learning method performed by the attention level estimation model learning device 2 according to the second embodiment will be described below with reference to FIG. 7.

In step S21, the display control unit 21 performs control to randomly present an image of an object to the display device 8 such as a monitor. The display device 8 displays a predetermined image at a predetermined position in the display screen at a predetermined timing according to the control of the display control unit 21. Images to be presented are prepared in advance and stored in the attention level estimation model learning device 2. The display control unit 21 simultaneously outputs the information of the time T _O displaying the image of the object to the display device 8 feature amount extraction unit 23, the measuring unit 25, and the learning section 26.

  The display device 8 is installed in front of the subject person 9. The display control unit 21 displays, for example, a cross gazing point at the center of the screen of the display device 8, and controls to display a black point on the gazing point at a predetermined time interval and for a predetermined time length. The subject person 9 gazes at the center of the screen of the display device 8 and presses a predetermined button (corresponding to the input unit 24) as soon as a black dot is displayed. Repeat the above task several times.

  The image which the display control unit 21 presents to the target person 9 is not limited to the above, and may be, for example, a video of a scene where a car travels on a road taken from the rear of the car. In this case, it may be a task of pressing a predetermined button (corresponding to the input unit 24) when the target person 9 perceives that the vehicle has been braked. In short, the display control unit 21 controls the subject person 9 to present a predetermined visual stimulus at random times, and when the subject person 9 presents the visual stimulus, a predetermined task (in the above-described example) Press the button). As described above, it is a task that requires the target person 9 to pay attention to a predetermined stimulus (in this example, a visual stimulus), and it can be performed if the speed and accuracy of the reaction can be achieved. Any task can be done. For example, auditory stimulation or tactile stimulation may be provided instead of visual stimulation, and tasks to be performed may be such as performing predetermined operations other than pressing a button.

  In step S22, the eye information acquisition unit 22 acquires information on the dynamic change of the eye of the object person 9 at each discrete time t, similarly to the eye information acquisition unit 11 of the attention level estimation device 1, and extracts the feature value. Output to unit 23.

In step S23, the feature amount extraction unit 23, the time-series information on dynamic changes in eye eyeball information acquiring unit 22 outputs the time T _O near the display device 8 displays the image of the object on the display screen Feature amount κ (t) representing dynamic change of the eye in a predetermined time interval of For example, the feature amount extraction unit 23 calculates a feature amount of a scalar or a vector having marks _{t t, 1} , ..., _{t t, d} representing features based on the micro- _saccade of the subject 9 in each time interval F _t . Extract as κ (t) = (κ _{t, 1} ,..., _{t t, d} ) (FIGS. 5A and 5B). However, d is an integer of 1 or more, d = 1 if the feature quantity κ (t) is a scalar, and d ≧ 2 if the feature quantity κ (t) is a vector. When expressing a micro saccade by a marked point process, the feature quantity 量 (t) is not necessarily the same length (the same d). When expressed by a marked point process, since the feature quantity ((t) represents a set of microsaccade marks generated in the time interval F _t , it is not common for d to be the same at all discrete times t . If no microsaccade occurs in the time interval _Ft , then d = 0, and in this case, the feature quantity κ (t) is an empty set. That is, the feature value κ (t) of the empty set represents that no microsaccade has occurred in the time interval F _t . Alternatively, if the micro saccade has not occurred within the time interval F _t, the feature quantity κ (t) is may be set to a special constant. The feature quantity extraction unit 23 outputs the obtained feature quantity κ (t) to the learning unit 26.

  In this embodiment, as the feature amount based on the movement of the eyeball itself, the feature amount based on microsaccade is used. In this case, the feature quantity extraction unit 23 calculates, for example, a primary difference sequence for the time series of the positions of the eyeballs, and the micro saccade detects the time when the absolute value of the primary difference sequence exceeds a predetermined first threshold. It may be detected as the start time (occurrence time) of However, if the length of time in which the absolute value of the primary difference sequence exceeds a predetermined threshold does not last longer than a predetermined value (usually about 3 milliseconds), it is excluded from detection. Moreover, when the below-mentioned reference amplitude A is more than a predetermined | prescribed threshold value (normal viewing angle 2 degree extent), it is excluded from detection as a large saccade. The feature quantity extraction unit 23 may use a moving average value within an appropriate range when calculating the first-order difference series, when it is determined that the acquired position information of the eyeball contains much noise. It is preferable to use a value of about six times the standard deviation of the difference series as the threshold used for detection.

The characteristics of the micro-saccade, a value based on the generation timing of the micro-saccades Z, the value D corresponding to the direction of motion, the absolute value of the reference amplitude A | A |, the maximum velocity V _max, duration D _m, overshoot Absolute value of amplitude A _o | A _o |, speed of overshoot V _o , rise time K, damping rate λ, damping coefficient ζ, natural angular frequency ω _n , microsaccade per unit time (for example, 1 second) The occurrence frequency R _{m and the} like can be exemplified, and at least one of them can be used as an element (mark) of the feature amount based on the microsaccade.

The value Z based on the generation timing of the micro-saccades that occur in a time interval F _t is, for example, may be a start time M _i time interval F _t, micro saccade corresponding to the reference become time interval RF _t The time difference between the occurrence time RM _t and the start time M _i may be | RM _t −M _i |, or the start time M _i or the function value g (M _i ) or g of the time difference | RM _t −M _i | It may be (| RM _t -M _i |). The reference time interval RF _t corresponds to the time interval F _t , for example RF _t = F _t−1 . The function value g (M _i ) or g (| RM _t −M _i |) is not limited, but for example, it becomes smaller as the representative value M _i or time difference | RM _t −M _i | Function values of functions having no singularity can be used. Examples of the function value g (M _i ) are 1 / M _i and exp (−M _i ). "Exp" represents an exponential function based on the number of Napiers. Examples of g (| RM _t -M _i |) include 1 / | RM _t -M _i | and exp (-| RM _t -M _i |). However, when M _i and | RM _t -M _i | are 0, 1 / M _i and 1 / | RM _t -M _i | are ∞. Therefore, g (M _i ) and g (| RM _t -M _i |) may be set to 0 when M _i and | RM _t -M _i | are zero. The value Z based on the generation timing of the microsaccade generated in such a time interval F _t may be used as one of the elements of the feature amount κ (t).

The value D according to the movement direction of the micro _saccade generated in the time interval F _t is a value determined for each movement direction. For example, the value D corresponding to the movement direction may take either of two values corresponding to the left and right movement directions, or may take one of four values corresponding to the left, right, upper and lower movement directions. Any of n values corresponding to the motion direction may be taken. As an example, a value D corresponding to the movement direction in the right direction (the direction from the left eye to the right eye) is taken as a first value (for example, −1), and the movement direction in the left direction (the direction from the left eye to the right eye) A value D corresponding to T is set to a second value (for example, 1). The value D corresponding to the movement direction of the microsaccade generated in such a time interval F _t may be used as one of the elements of the feature quantity κ (t).

Next, referring to FIG. 5C, the microsaccade reference amplitude A, maximum velocity V _max , duration D _m , overshoot amplitude A _o , overshoot velocity V _o , rise time K, decay rate λ explain.
(1) Reference amplitude A: The amount of movement when the movement of the eyeball due to the micro saccade converges.
(2) Maximum velocity V _max : This is the maximum velocity to reach the reference amplitude A + the amplitude A _o of the overshoot.
(3) Duration time D _m : The length of the time interval in which the microsaccade occurs. The start time of the microsaccade is the time when the absolute value of the primary difference sequence exceeds a predetermined threshold, and the end time of the microsaccade is the time when it returns to the reference amplitude A for the first time after reaching the overshoot amplitude. .
(4) the amplitude A _o overshoot (overshoot phenomenon): reference amplitude A has been exceeded (excesses) by a micro saccade is the amount of parts. The overshoot is a phenomenon in which the waveform protrudes beyond the reference amplitude A at the rising portion of the waveform or a waveform that protrudes. In other words, the overshoot amplitude is the amount of the protruding portion.
(5) Overshoot velocity V _o : This is the maximum velocity when trying to converge from the reference amplitude A + overshoot amplitude A _o to the reference amplitude A.
(6) Rise time K: time taken to reach (rise) reference amplitude A + overshoot amplitude A _o . The time taken to reach the reference amplitude A + the amplitude A _o of the overshoot is equal to the time taken to reach the speed V _o of the overshoot from the maximum speed V _max .
(7) Attenuation rate λ: A ratio of overshoot amplitude A _o to reference amplitude A. It may be the ratio of the overshoot speed V _{o to} the maximum speed V _max ,

It is expressed as

Damping coefficient マ イ ク ロ of microsaccade, natural angular frequency ω _n is

It is expressed as The natural angular frequency ω _n corresponds to an index representing the speed of response of the microsaccade, and the damping coefficient ζ corresponds to an index representing the convergence of the response of the microsaccade.

The feature amount extraction unit 23 optimizes the attenuation coefficient マ イ ク ロ of the microsaccade, the natural angular frequency ω _n , and the reference amplitude A by the least squares method etc. by fitting the position of the eye while the microsaccade occurs. You may calculate by doing.

  Since the attenuation coefficient 23 of the microsaccade tends to change its value depending on the motion in the left and right direction, the feature quantity extraction unit 23 is a representative value of the attenuation coefficient of the microsaccade in the left direction, The representative value of the saccade attenuation coefficient may be calculated separately.

Absolute value | A | of the reference amplitude A of the above _microsaccade generated in the time interval F _t , maximum velocity V _max , duration D _m , absolute value of overshoot amplitude A _o | A _o | At least one of the shoot speed V _o , the rise time K, the attenuation factor λ, the attenuation coefficient ζ, and the natural angular frequency ω _n may be used as one of the elements of the feature quantity 量 (t).

In step S24, the input unit 24 receives a predetermined input from the object person 9. It is assumed that the target person 9 looks at the display screen of the display device 8 and performs a task such as pressing a button when an image of the object is displayed on the display device 8. The input unit 24 outputs the time T _{IN at} which a predetermined input (here, a button pressed) is received to the measuring unit 25.

In step S25, the measuring unit 25, the time difference T _O of the time T _O displaying an image of the object received from the display control unit 21 on the display screen, the time T _IN that has received the input received from the input unit 24 Calculate T _IN , and output an index indicating the speed of the reaction speed to the learning unit 26 based on the time difference.

In step S26, the learning unit 26 uses the learning data, which is a set of an index representing the attention level calculated from the index indicating the speed of the reaction speed output by the measuring unit 25, and the feature value at that time. The attention level function f ₀ (t) and the parameters of the attention level reduction function L (τ) are learned. Here, the term "feature quantity when" a feature amount of time T _O near obtained from the feature extractor 23.

The learning data displays a predetermined image on the display device 8, performs a plurality of trials of receiving an input from the target person 9, and generates an index indicating the reaction speed for each trial and the time obtained from the feature extraction unit 23 It assumed to be to obtain a characteristic quantity of the set of T _O vicinity. As a result, it is possible to obtain learning data that is a set of a set of feature amounts for each trial and an index representing an attention level.

  As mentioned above, it is believed that the higher the attention level, the faster the reaction rate. That is, it is possible to obtain an index representing the attention level from the index representing the reaction speed, assuming that the index representing the reaction speed and the index representing the attention level have a negative correlation. For example, a broad monotonous decreasing function value related to an index indicating reaction speed may be used as an index value of an index indicating attention level. Alternatively, if the indicator indicating the reaction rate is less than the predetermined value, the indicator indicating that the caution level is high is used as the caution level indicator, and if the indicator indicating the reaction rate is greater than the predetermined value, the caution level is low. The indicator to be represented may be the indicator of the attention level. For example, the reaction rate RT can be converted to the caution level L by calculating the following equation.

Here, RT _min is the minimum value of the reaction rate in the learning data, and RT _max is the maximum value of the reaction rate in the learning data. According to this conversion equation, the caution level L always takes a value of 0 to 1. The attention level L thus obtained is referred to as "observation level observation value".

First, learning of parameters of the attention level lowering function L (τ) will be described. The learning unit 26 learns α and l _peak which determine the form of the attention level lowering function L (τ) using the learning data. This processing corresponds to learning the parameters (c ₁ , c ₂ ,..., C ₈ ) of the function that determines α and l _peak . The parameters (c ₁ , c ₂ ,..., C ₈ ) can be set for each individual as described in <Caution level modeling>. For example, the parameters (c ₁ , c ₂ ,..., C ₈ ) are set to appropriate initial values, and calculated by equation (4) based on α and l _peak obtained by equations (2) and (3) The least squared error between the estimated value ^ L _{i of} the attention level and the attention level L (this is called "attention level observation value") obtained by applying the training data to equation (7) , the parameter _{(c 1, c 2, ...} , c 8) by repeating continue updating the parameters _{(c 1, c 2, ...} , c 8) to learn. Here, A _i , ζ _i , and ω _i in Equations (2) and (3) are the amplitude and attenuation of the microsaccade that occurred most immediately at the time when the image of the object was displayed in the i-th trial, respectively. Let the coefficient, the natural angular frequency, and τ _i be the time from the microsaccade until the image of the object is displayed. The reference learning without the least square error, may be used such as a distance between the estimated value ^ L _i of caution level and attention level observations. In short, the parameters (c ₁ , c ₂ ,..., C ₈ ) may be repeatedly updated so that the attention level observation value and the attention level estimation value ^ L _i approach each other.

The learning unit 26 also learns the parameters of the reference attention level function f ₀ (t). For example, the average value of the attention level indicators corresponding to the indicators indicating the reaction rates obtained in a plurality of trials as described above is used as the parameter indicating the reference attention level.

The learning unit 26 stores the parameter of the reference attention level function f ₀ (t) obtained by learning and the parameter of the attention level lowering function L (τ) in the model storage unit 10. The parameters of the attention level reduction function L (τ) may be α and l _peak or may be parameters specifying α and l _peak (c ₁ , c ₂ ,..., C ₈ ).

<Experimental result>
The experiment was conducted with the following contents to evaluate the effect of the present invention.

  The screen is placed 74 cm in front of the subject, and black spots are displayed in the center of the screen at intervals of 1.5 to 3.5 seconds for 0.5 seconds each. When the black point is not displayed, it is assumed that a cross point is displayed in the center of the screen. The subject gazes at the fixation point in the center of the screen during the experiment, and responds by pressing the button as soon as possible when a black dot is displayed. The diameter of the black point is set to 2.0 degrees as a viewing angle. The above task was tried 32 times for each of 25 subjects. The eye movement was measured using Eyelink CL manufactured by SR Research, and the sampling frequency was set to 1000 Hz.

  The experimental data collected was analyzed as follows. First, the microsaccade most recently generated with respect to the timing at which the black dot was displayed in each trial was detected one by one. The time difference between the display start time of the black point and the microsaccade occurrence time closest to it was taken as RT. In addition, the microsaccade amplitude, damping coefficient and natural angular frequency were calculated. However, trials where microsaccades did not occur within the last 3 seconds were excluded. The above equation (7) was calculated for each experimental data for which the reaction time RT could be obtained, and the reaction time RT was converted to a caution level L.

  The experimental results are shown in FIG. FIG. 8A is a graph comparing the observed value and the estimated value of attention level for each trial in a certain subject. In the example of FIG. 8A, the correlation coefficient R between the observation value at the attention level and the estimation value is 0.6709. FIG. 8B is a scatter plot of trial-to-trial observations and estimates for the entire subject. One point in the graph represents one trial. When correlation coefficients between observed and estimated attention levels were calculated for each subject, 88% of subjects showed significant correlation. The average value of the correlation coefficient was R = 0.5221 ± 0.1968, and the maximum value was 0.8730.

  As described above, according to the attention level estimation device of the present invention, it has been proved that the attention level can be estimated with high accuracy.

<Example of application>
By being able to estimate the attention level for a task, the following can be made. For example, in an operation requiring high accuracy, when the worker's attention level decreases, an operation error is likely to occur, which may be a cause of an accident. Therefore, according to the present invention, it is possible to prevent an accident in advance by estimating a caution level at predetermined time intervals and prompting a break when the caution level falls below a predetermined level. The same thing can be considered in driving a vehicle, etc., because a drop in caution level may cause an accident.

  As mentioned above, although embodiment of this invention was described, a specific structure is not restricted to these embodiment, Even if there is a change of design suitably etc. in the range which does not deviate from the meaning of this invention, Needless to say, it is included in the present invention. The various processes described in the embodiment are not only executed chronologically according to the order described, but may be executed in parallel or individually depending on the processing capability of the apparatus executing the process or the necessity.

[Program, recording medium]
When various processing functions in each device described in the above embodiments are implemented by a computer, the processing content of the function that each device should have is described by a program. By executing this program on a computer, various processing functions in each of the above-described devices are realized on the computer.

  The program describing the processing content can be recorded in a computer readable recording medium. As the computer readable recording medium, any medium such as a magnetic recording device, an optical disc, a magneto-optical recording medium, a semiconductor memory, etc. may be used.

  Further, the distribution of this program is carried out, for example, by selling, transferring, lending, etc. a portable recording medium such as a DVD, a CD-ROM, etc. in which the program is recorded. Furthermore, this program may be stored in a storage device of a server computer, and the program may be distributed by transferring the program from the server computer to another computer via a network.

  For example, a computer that executes such a program first temporarily stores a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. Then, at the time of execution of the process, this computer reads the program stored in its own storage device and executes the process according to the read program. Further, as another execution form of this program, the computer may read the program directly from the portable recording medium and execute processing according to the program, and further, the program is transferred from the server computer to this computer Each time, processing according to the received program may be executed sequentially. In addition, a configuration in which the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes processing functions only by executing instructions and acquiring results from the server computer without transferring the program to the computer It may be Note that the program in the present embodiment includes information provided for processing by a computer that conforms to the program (such as data that is not a direct command to the computer but has a property that defines the processing of the computer).

  Further, in this embodiment, although the present apparatus is configured by executing a predetermined program on a computer, at least a part of the processing contents may be realized as hardware.

DESCRIPTION OF SYMBOLS 1 attention level estimation device 2 attention level estimation model learning device 8 display device 9 target person 10 model storage unit 11 eye information acquisition unit 12 micro saccade detection unit 13 estimation unit 21 display control unit 22 eye information acquisition unit 23 feature amount extraction unit 24 input unit 25 measurement unit 26 learning unit

Claims (9)

A model storage unit for storing information indicating a reference attention level and information indicating a decrease in attention level;
An eye information acquisition unit for acquiring a dynamic change of the subject's eye;
A microsaccade detection unit configured to detect a microsaccade occurrence time from the dynamic change of the eye;
An estimation unit that estimates an attention level from the occurrence time of the micro saccade based on the information indicating the reference attention level and the information indicating a decrease in the attention level;
Attention level estimation device including. The attention level estimation device according to claim 1, wherein
The information representing the reference attention level is a function representing an average attention level,
The information representing the reduction in attention level is a function representing the reduction in attention level accompanying the occurrence of microsaccade,
The estimation unit is configured to estimate the attention level by convoluting a function representing a drop in the attention level at the occurrence time of the microsaccade with a function representing the average attention level.
Attention level estimation device. The attention level estimation device according to claim 2, wherein
The function representing the lowering of the attention level is a function that monotonously decreases the attention level immediately after the time at which the microsaccade occurs and then gradually increases the attention level gradually with the passage of time.
Attention level estimation device. The attention level estimation device according to claim 3, wherein
The function representing the above attention level reduction has a large gradient at monotonically decreasing and a small gradient at monotonically increasing.
Attention level estimation device. A display control unit for displaying a predetermined image on a display device;
An eye information acquisition unit that acquires time-series information on dynamic changes of the subject's eyes;
A feature amount extraction unit for extracting a feature amount based on a micro saccade from time-series information on the dynamic change of the eye;
An input unit that receives a predetermined input made by the subject in response to the predetermined image;
A measuring unit configured to measure an index representing a reaction rate from the time when the predetermined image is displayed and the time when the predetermined input is received;
A learning unit that learns information representing a drop in attention level by the microsaccade based on the feature amount based on the microsaccade and the index indicating the reaction speed;
Attention level estimation model learning device including. The attention level estimation model learning device according to claim 5, wherein
The above learning unit
Attention level estimate value which is a function value of a function having as parameters the information indicating the reduction of the attention level and the feature quantity based on the microsaccade, and the attention level which is a broad monotonic decreasing function value related to the index representing the reaction speed The information representing the drop in the attention level is learned by repeatedly updating the information representing the drop in the attention level so that the difference with the observation value of
Based on the reference attention level which is a predetermined attention level, the function which uses the information indicating the lowering of the attention level and the feature quantity based on the microsaccade as a parameter makes a sudden attention after the occurrence time of the immediately preceding micro saccade. An attention level estimation model learning device characterized in that the level is lowered and the attention level is gradually increased monotonously with the passage of time. The model storage unit stores information representing a reference attention level and information representing a decrease in attention level,
The eye information acquisition unit acquires a dynamic change of the subject's eye,
The micro saccade detection unit detects the micro saccade occurrence time from the dynamic change of the eye,
The estimation unit estimates the attention level from the occurrence time of the micro saccade based on the information indicating the reference attention level and the information indicating the decrease of the attention level.
Attention level estimation method. The display control unit displays a predetermined image on the display device,
The eye information acquisition unit acquires time-series information on dynamic changes of the subject's eyes,
A feature amount extraction unit extracts a feature amount based on a micro saccade from the time-series information on the dynamic change of the eye,
The input unit receives a predetermined input that the subject makes in response to the predetermined image,
The measuring unit measures an index representing a reaction rate from the time when the predetermined image is displayed and the time when the predetermined input is received,
The learning unit learns information representing a decrease in attention level due to the micro saccade based on the feature amount based on the micro saccade and the index indicating the reaction speed.
Attention level estimation model learning method.   A program for causing a computer to function as the attention level estimation device according to any one of claims 1 to 4 or the attention level estimation model learning device according to claim 5 or 6.

Images