JPH0819520A - Eye movement analyzing method - Google Patents

Abstract

PURPOSE:To sense fatigue of an eye ball accurately and quickly by measuring the movement of the eye, producing relevant digital data, and calculating the entropy of the eye movement. CONSTITUTION:When the movement to display a specified video image on a display part 6 is commenced, a CPU 2 controls a measuring part 7, and the measurement is started. An LED 31 in the measuring part 7 casts an infrared ray onto an eye of the user, and the reflected light by the monitored region is received by phototransistors 32, 33, and the sum and difference of/between the two outputs are calculated, result wherefrom is fed to an A/D converter 8. The A/D converter 8 makes sampling of signals with the set sampling frequency. When the sampling data attains the set number, the CPU 2 computes the entropy of the data which has been acquired. The obtained entropy is plotted with the elapse of the time. If the entropy is increasing with time, the CPU 2 judges that the fatigue of the eye is advanced.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an eye movement analysis method and, more particularly, to an eye movement analysis method for obtaining information on fatigue of the eyes of a user such as a head mounted display (HMD).

[0002]

2. Description of the Related Art Recently, head mounted displays (H
MD) is becoming popular. The HMD is provided with a display unit. When a video image or the like is displayed on the HMD, the display position of the display unit is close to the eyes of the user. Therefore, the display unit is small and substantially large. It is possible to obtain the same effect as when displayed in.

If an image having parallax between the left and right eyes is displayed as the display image, a stereoscopic image can be displayed. As a result, a so-called virtual reality (virtual reality space) can be formed.

However, since such an HMD is used by a user who mounts it on his / her head so that he / she wears so-called eyeglasses, he / she always looks at the displayed image, and during that time, he / she is in a normal life. I cannot see the space and my eyes tend to get tired easily. Therefore, it is desired to detect eye fatigue and notify the user of this when eye fatigue is detected.

[0005]

In order to detect the fatigue of the eyeball, for example, it is known to find the time derivative of the eyeball movement of both eyes and detect the fatigue of the eyeball from the cross-correlation.

However, the conventional method has a problem that it is difficult to accurately and promptly determine the fatigue of the eyeball.

The present invention has been made in view of such a situation, and it is possible to accurately and quickly detect eye fatigue.

[0008]

The eye movement analysis method according to claim 1 measures the movement of the eyeball to generate digital data corresponding to the movement of the eyeball, and from the digital data,
It is characterized in that the entropy of eye movement is calculated and information on eye fatigue is obtained from the entropy.

According to the eye movement analysis method of the second aspect, the movement of the eyeball is measured, digital data corresponding to the movement of the eyeball is generated, the standard deviation of the movement of the eyeball is calculated from the digital data, and the standard deviation is calculated. The feature is that information on eye fatigue is obtained from the deviation.

The horizontal movements of the left and right eyeballs are measured to generate digital data corresponding to the movements of the eyeballs, and the correlation coefficient of the horizontal movements of the left and right eyeballs is calculated from the digital data. It is characterized in that information on eye fatigue is obtained from the value of the relation number.

In this case, it is possible to further calculate the entropy of the eye movement from the digital data, predict the eye fatigue from the correlation coefficient, and quantify the eye fatigue from the entropy.

Alternatively, the standard deviation of the movement of the eyeball can be further calculated from the digital data, the fatigue of the eyeball can be predicted from the correlation coefficient, and the fatigue of the eyeball can be quantified from the standard deviation. .

The eye movement analysis method according to a sixth aspect of the present invention measures the movement of the eyeball, generates digital data corresponding to the movement of the eyeball, and extracts a predetermined number of significant digits from the digital data. It is characterized in that a predetermined value is calculated from the obtained digital data of the effective digit number to obtain information regarding eye fatigue.

The eye movement analysis method according to claim 7 measures the movement of the left and right eyeballs, calculates a predetermined value from digital data of the left eyeball, and calculates a predetermined value from digital data of the right eyeball. However, it is characterized by comparing the calculated values of the left and right eyes and determining the effective eye.

In this case, the value calculated from the digital data can be used as the entropy or standard deviation, and the eye fatigue can be determined based on the time dependence of these values.

Further, a value calculated from digital data may be used as a correlation coefficient represented by, for example, Spearman's rank correlation coefficient, and eye fatigue may be predicted from the magnitude of this correlation coefficient. .

[0017]

According to the eye movement analysis method of the first aspect, the entropy is calculated from the digital data corresponding to the movement of the eyeball, and the information regarding the fatigue of the eyeball is obtained from this entropy.

Further, in the eye movement analysis method according to the second aspect, the standard deviation of the movement of the eyeball is calculated from the digital data corresponding to the movement of the eyeball, and the information regarding the fatigue of the eyeball is obtained from the standard deviation.

Further, in the eye movement analysis method according to the third aspect, the correlation coefficient of the horizontal movement of the left and right eyeballs is calculated from the digital data corresponding to the movement of the eyeballs, and the value of this correlation coefficient is calculated. Provides information on eye fatigue.

Therefore, in either case, eye fatigue can be detected accurately and quickly.

In the eye movement analysis method according to the sixth aspect, a predetermined effective digit number is extracted from the digital data corresponding to the movement of the eyeball, and, for example, entropy, Standard deviation, correlation coefficient, etc. are calculated. Therefore, it becomes possible to quickly obtain information regarding eye fatigue.

In the eye movement analysis method according to the seventh aspect, entropy and standard deviation are obtained from digital data of the movement of the left and right eyes. Then, from the value, it is determined which of the left and right eyeballs is the dominant eye. Therefore, it becomes possible to detect the eyeball fatigue more accurately.

[0023]

1 shows an example of the configuration of a head mounted display (HMD) to which the eye movement analysis method of the present invention is applied. In this embodiment, various commands can be input to the CPU 2 by operating the input unit 1. The CPU 2 executes various kinds of processing according to the programs stored in the ROM 3. The RAM 4 appropriately stores data necessary for the CPU 2 to execute various processes.

The display unit 6 is controlled by the control unit 5 so that video data reproduced from a recording medium (not shown) or the like is appropriately displayed. The measuring unit 7 measures the movement of the user's eyeball and outputs the measurement result to the A / D converter 8. A / D converter 8
Is configured to A / D convert the input signal and supply it to the CPU 2.

FIG. 2 shows the external structure of the portion mounted on the head of the HMD. As shown in FIG.
1 and the locking portion 12, the whole is configured in the shape of spectacles, and the locking portion 12 is locked to the upper part of the user's ear, and the screen 11 is arranged in front of the user's eye. ing. The screen 11 is provided with display units 6L and 6R so as to face the left and right eyes of the user. The display units 6L and 6R form the display unit 6 shown in FIG.

Further, measuring portions 7L and 7R are arranged below the screen 11 so as to face the left and right eyes of the user. The measuring units 7L and 7R compose the measuring unit 7 in FIG.

FIG. 3 shows an example of the configuration of the measuring unit 7L seen from the plane (7R is similarly configured). As shown in the figure, LE that irradiates the user's eyes with infrared rays
D31 is provided, and phototransistors 32 and 33 as light receiving elements are arranged on the left and right of the D31. These phototransistors 32 and 33 are adapted to receive the infrared reflected light from the LED 31 which is reflected in a predetermined area of the eye.

As the LED 31, for example, an LED having a relatively wide directivity which projects infrared rays in a range of ± 21 degrees can be used, and as the phototransistors 32 and 33, for example, an infrared ray in a range of ± 10 degrees is received. A phototransistor having a relatively sharp directivity can be used.

FIG. 4 shows an example of the structure for detecting the movement of the eyeball in the vertical (up and down) direction. As shown in the figure, the infrared light emitted from the LED 31 is applied to the region 51 and the region 52 located near the lower end of the black eye 41. Phototransistors 32 and 33
Are adapted to receive the reflected light from the regions 51 and 52, respectively.

Of course, instead of irradiating the light emitted from the LED 31 only to the regions 51 and 52, the light is applied to the entire eye, and only the reflected light from the regions 51 and 52 is phototransistor. It is also possible to receive light by 32 and 33.

The reflected light from the regions 51 and 52 is received by the phototransistors 32 and 33, and the output thereof is supplied to the summing amplifier 53 and added. When the black eye 41 moves downward, the proportion of the black eye 41 in the areas 51 and 52 increases, and the proportion of the white eye 42 relatively decreases. Since the reflectance of the black eye 41 is smaller than that of the white eye 42, the output of the summing amplifier 53 becomes smaller.

On the other hand, when the black eye 41 moves upward, the proportion of the black eye 41 in the areas 51 and 52 decreases and the proportion of the white eye 42 relatively increases. As a result, the output of the summing amplifier 53 becomes large.

Therefore, from the output of the summing amplifier 53, the black eye 4
The movement of 1 in the vertical (up and down) direction can be detected.

On the other hand, as shown in FIG. 5, the phototransistors 32 and 33 are adapted to receive the reflected light of the left-side region 61 and the right-side region 62 of the black eye 41 in the horizontal direction in the figure, respectively. , Phototransistors 32 and 33
When the differential amplifier 63 calculates the difference between the outputs of
When the black eye 41 moves to the left in FIG. 5, for example,
The proportion of the black eye 41 in the area 61 increases,
The output of the phototransistor 32 becomes small. On the contrary, the proportion of the black eye 41 in the area 62 becomes small, and the output of the phototransistor 33 becomes large. As a result, the output of the differential amplifier 63 decreases (increases in the negative direction).

On the other hand, when the black eye 41 moves to the right in FIG. 5, the ratio of the black eye 41 in the area 62 increases, and the output of the phototransistor 33 decreases. On the other hand, in the area 61, the proportion occupied by the black eye 41 becomes small and the output of the phototransistor 32 becomes large. As a result, the output of the differential amplifier 63 increases in the positive direction.

As described above, the output of the differential amplifier 63 changes in accordance with the horizontal movement of the black eye 41.

In FIG. 5, the regions 61 and 62 are
The regions 51 and 52 shown in FIG. 4 are arranged at positions different from each other. However, even if the outputs of the regions 51 and 52 are supplied to the differential amplifier 63 as they are, the output of the differential amplifier 63 causes a black eye. It is possible to obtain a signal corresponding to the horizontal movement of 41. However, as shown in FIG.
Compared with the case where the reflected light from the regions 61 and 62 located on the left and right in the horizontal direction of 1 is received, the sensitivity is slightly lowered.

Next, with reference to the flow charts of FIGS. 6 and 7, the operation for measuring eye fatigue will be described.

In order to measure eye fatigue, the pretreatment shown in FIG. 6 is performed in advance before the measurement process is actually started.

First, in step S1, the input unit 1 is operated to input predetermined measurement conditions. As the measurement conditions, for example, the measurement range of the movement of the black eye 41, the sampling frequency of the measurement, and the like are input.

Next, the process proceeds to step S2, in which the significant digit of the measurement data is input by operating the input unit 1. Further, in step S3, the number of calculation data (the number of sampling data used when calculating values such as entropy, standard deviation, and correlation coefficient described later) is also input by operating the input unit 1.

That is, the signals output from the summing amplifier 53 and the differential amplifier 63 of the measuring section 7 are input to the A / D converter 8 and sampled at the sampling frequency set in step S1. The sampled data is RA
Once stored in M4.

As will be described later, the CPU 2 uses this data to calculate the entropy, standard deviation, rank correlation coefficient and the like. In performing this calculation,
Among the effective digit numbers of the digital data obtained as a result of the A / D conversion by the A / D converter 8, only the effective digit number set in step S2 is used. If all the digits obtained when A / D converted by the A / D converter 8 are used for calculation, more accurate calculation can be performed. Will be costly.
Therefore, by using only the number of significant digits set in step S2 for the calculation, a more rapid calculation is performed.

Further, the number of data in the range in which such entropy, standard deviation or rank correlation coefficient is calculated is
The data of the range set in step S3 is used.
For example, in step S1, the sampling frequency is 2
If 00Hz is set and 200 is set as the number of calculation data, when 200 sampling data are collected, entropy, standard deviation or rank correlation coefficient of the 200 sampling data is calculated. . Therefore, in this case, one entropy, standard deviation, or rank correlation coefficient is obtained per second.

When the above initial settings are made in advance, when the operation of displaying a predetermined video image on the display unit 6 is started, the CPU 2 starts the process shown in the flowchart of FIG. First, in step S11, the CPU
2 controls the measuring unit 7 to start the measurement. As described above, the LED 31 of the measuring unit 7 irradiates the user's eyes with infrared rays, and the reflected light from the regions 51, 52 or the regions 61, 62 is received by the phototransistors 32, 33. Then, the sum and difference of the outputs of the phototransistors 32 and 33 are
The summing amplifier 53 and the differential amplifier 63 perform calculation, and the calculation result is output to the A / D converter 8.

The A / D converter 8 samples the signal input from the measuring section 7 at the sampling frequency set in step S1. Of the sampling data output from the A / D converter 8, the CPU 2 extracts only the number of significant digits set in step S2, transfers it to the RAM 4, and stores it. When the number of sampling data set in step S3 is obtained, the measurement for the next cycle is continued as it is, and step S1 is performed.
The process proceeds to step 2 and the calculation processing of a predetermined number of sampling data obtained up to that point is performed. In this example, the entropy H of the data obtained so far is
₁ (S) is calculated. H ₁ (S) = − ΣP _i log ₂ P _i

Here, P _i represents the probability of taking the state i, which is sampling data in the case of this embodiment.

Next, in step S13, a time dependence plot process is executed. That is, the entropy obtained as described above is plotted according to the passage of time. As a result, for example, the characteristics shown in FIG. 8 are obtained.

As shown in FIG. 8, the entropy of the output of the summing amplifier 53 of the left eye or the output of the differential amplifier 63 in the state in which the HMD is mounted gradually increases with time.

On the other hand, when the user observes a predetermined real index (real target) without using the HMD and the entropy of the left and right eyes of the user is calculated, as shown in FIG. The value is almost constant regardless of the progress. That is, when using the HMD, it is considered that the fatigue of the eyeball is advanced to that extent. Therefore, when the entropy increases with the lapse of time, the CPU 2 determines that the eyeball fatigue is increasing. When the degree of progress reaches a preset reference value, the CPU 2 controls the control unit 5 to cause the display unit 6 to display a predetermined message prompting the user to stop using the HMD. Is displayed. This makes it possible to prevent the user from not being aware of eye fatigue.

As shown in FIG. 8, the entropy of the left eye when wearing the HMD is larger than the entropy of the right eye. Normally, humans have one of the right eye and the left eye as the dominant eye, and the dominant eye is relatively easy to concentrate on the index, but the non-effective eye concentrates on the index. Is relatively difficult. as a result,
Usually, the entropy of the non-effective eye is larger than the entropy of the effective eye. Because the more entropy value is not concentrated, the larger it is.

Therefore, the entropy of the left eye and the entropy of the right eye are compared, and the eye corresponding to the smaller entropy can be determined as the effective eye of the user. The determination of the time dependence, the effective eye, and the eye of the non-effective eye, it is possible to perform both, if originally focused on the entropy of the effective eye that is easy to concentrate on the index, It is possible to detect the fatigue of the eyeballs of the user more reliably.

As described above, after the time-dependent plot processing is performed in step S13, the process proceeds to step S14, it is determined whether or not the end of use of the HMD is instructed, and the end is instructed. If not,
Returning to step S11, the same processing is repeatedly executed. When the end of use of the HMD is instructed,
The eye fatigue detection process is also terminated.

In the above embodiment, the output of the summing amplifier 53 and the differential amplifier 63 are used to calculate the entropy.
The entropy may be calculated for each of the outputs of, but only one of them may be calculated. Further, the entropy can be calculated for each of the movements of the left eye and the right eye, but it is also possible to use only one of them. However, as described above, when determining the dominant eye, it is necessary to calculate the entropy of the movement of the left and right eyes.

In the above embodiment, step S12.
In the above, the entropy is calculated, but it is also possible to calculate the standard deviation. in this case,
The time dependence plot in step S13 is performed for the standard deviation, and FIG. 9 shows an example in that case. As shown in the figure, when using the HMD, it can be seen that the standard deviation increases with the passage of time.

On the other hand, as shown in FIG. 10, when the user observes the actual index, the standard deviation is almost constant regardless of the passage of time.

From this, when the standard deviation increases with the passage of time, it can be determined that the eye fatigue of the user is progressing.

Furthermore, in step S12, the correlation coefficient of the movement of the left and right eyes can be calculated. Therefore, in this case, in step S11,
Horizontal movements of at least both the left and right eyes are measured. That is, the output of the differential amplifier 63 in FIG. 5 is sampled in each of the left eye and the right eye.

Then, n pieces of data (Li, Ri) consisting of a set of data Li obtained by moving the left eye in the horizontal direction and data Ri obtained by moving the right eye in the horizontal direction are arranged in descending order. , Rank data (r _Li , r _Ri ). Then, from this rank data, the Spearman rank correlation coefficient r _S is calculated according to the following equation. r _S = {Σr _Li r _Ri − (Σr _Li Σr _Ri / n)} / [{Σ
r ² _Li − (Σr _Li ) ² / n} {Σr ² _Ri − (Σr _Ri ) ² /
n}] ^1/2

FIG. 11 shows the result of plotting the time dependence of the rank correlation coefficient thus obtained in step S13. As shown in the figure, H
The rank correlation coefficient in the case of using MD is negative or small even if it is positive (close to 0). On the other hand, the rank correlation coefficient of the actual index has a relatively large positive value (close to 1).

In terms of human eye movement, the left and right eyes simultaneously move horizontally in phase (when the left eye moves to the right (or left), the right eye also moves to the right (or left)) Soccer And a following eye movement that moves in reverse phase (when the left eye moves to the right (or left), the right eye moves to the left (or right)). The rank correlation coefficient is positive in the case of saccade and negative in the case of following eye movement. Comparing saccade and following eye movement, the following eye movement tends to be more tired than saccade. Therefore, when the value of the rank correlation coefficient is negative or a positive value close to 0, it can be predicted that the eyeball tends to be fatigued.

Therefore, when the value of the rank correlation coefficient becomes negative or a positive value close to 0, it can be judged as eye fatigue.

Alternatively, when the value of the rank correlation coefficient becomes negative or a positive value close to 0, it is predicted that the eyeball tends to be fatigued, and then, as described above, the entropy or standard The deviation can be used to quantitatively determine the degree of eye fatigue.

In the above embodiment, the Spearman rank correlation coefficient is used. However, when the rank correlation coefficient is used in this way, no matter how large the actual data value becomes, Is converted into a value corresponding to the rank, so that the amount of data in the case of performing a substantial calculation can be substantially reduced, and the processing can be speeded up accordingly.

Of course, as the correlation coefficient, it is possible to use the normal correlation coefficient as it is, instead of the rank correlation coefficient. In that case, as compared with the case of using the rank correlation coefficient, the data The amount becomes large, and the calculation takes a long time.

[0066]

According to the eye movement analysis method described in claims 1 to 3, the entropy, the standard deviation, or the correlation coefficient of the movement of the eyeball is calculated from the digital data corresponding to the movement of the eyeball. Therefore, it is possible to accurately and promptly obtain information regarding eye fatigue.

According to the eye movement analysis method of the sixth aspect, since the predetermined value is calculated from the extracted digital data of the effective digits, it is possible to quickly obtain the information regarding the eye fatigue. Becomes

Further, according to the eye movement analysis method of the seventh aspect, the effective eye is determined from the calculated values of the left and right eyes, so that the information regarding eye fatigue can be obtained more accurately. Is possible.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration example of a head mounted display to which an eye movement analysis method of the present invention is applied.

FIG. 2 is a perspective view showing a configuration of a portion mounted on the head of the head mounted display.

FIG. 3 is a plan view illustrating the principle of the measuring unit 7L of FIG.

4 is a diagram illustrating the principle of detecting vertical movement of an eyeball in the measuring unit 7L of FIG.

5 is a diagram illustrating the principle of detecting horizontal movement of the eyeball in the measuring unit 7L of FIG.

FIG. 6 is a flowchart illustrating an initial setting process executed when eye fatigue is detected.

FIG. 7 is a flowchart illustrating an eyeball fatigue detection process according to the embodiment of FIG.

FIG. 8 is a diagram illustrating time dependence of entropy.

FIG. 9 is a diagram illustrating time dependence of standard deviation when using an HMD.

FIG. 10 is a diagram illustrating the time dependence of the standard deviation of the actual index.

FIG. 11 is a diagram illustrating the time dependence of Spearman's rank correlation coefficient.

[Explanation of symbols]

 1 Input Section 2 CPU 3 ROM 4 RAM 5 Control Section 6 Display Section 7 Measuring Section 8 A / D Converter 11 Screen 12 Locking Section 31 LED 32, 33 Phototransistor 41 Black Eye 42 White Eye 51, 52 Area 53 Summing Amplifier 61 , 62 area 63 differential amplifier

Claims (11)

[Claims] 1. An eye movement is measured, digital data corresponding to the movement of the eye is generated, an entropy of the movement of the eye is calculated from the digital data, and information on fatigue of the eye from the entropy. A method for analyzing eye movements, which comprises: 2. An eye movement is measured, digital data corresponding to the eye movement is generated, a standard deviation of the eye movement is calculated from the digital data, and the eye fatigue is calculated from the standard deviation. A method for analyzing eye movements, which comprises obtaining information about 3. The horizontal movements of the left and right eyeballs are measured to generate digital data corresponding to the movements of the eyeballs, and the correlation coefficient of the horizontal movements of the left and right eyeballs is generated from the digital data. Is calculated, and information regarding fatigue of the eyeball is obtained from the value of the correlation coefficient. 4. The entropy of the movement of the eyeball is further calculated from the digital data, the fatigue of the eyeball is predicted from the correlation coefficient, and the fatigue of the eyeball is quantified from the entropy. The eye movement analysis method according to claim 3. 5. The standard deviation of the movement of the eyeball is further calculated from the digital data, the fatigue of the eyeball is predicted from the correlation coefficient, and the fatigue of the eyeball is quantified from the standard deviation. The eye movement analysis method according to claim 3. 6. An eye movement is measured, digital data corresponding to the movement of the eye is generated, a predetermined effective digit number is extracted from the digital data, and the extracted effective digit number of the digital data is extracted. An eye movement analysis method, which calculates a predetermined value from data and obtains information about the eye fatigue. 7. The movement of the left and right eyeballs is measured, a predetermined value is calculated from the digital data of the left eyeball, and a predetermined value is calculated from the digital data of the right eyeball, An eye movement analysis method, which comprises determining the effective eye by comparing the calculated values of the eyeball. 8. The eye movement analysis method according to claim 6, wherein the value calculated from the digital data is entropy, and fatigue of the eyeball is determined from the time dependence of the entropy. 9. The eye movement analysis according to claim 6, wherein the value calculated from the digital data is a standard deviation, and the fatigue of the eyeball is determined from the time dependence of the standard deviation. Method. 10. The eye movement according to claim 6, wherein the value calculated from the digital data is a correlation coefficient, and fatigue of the eyeball is predicted from the magnitude of the correlation coefficient. analysis method. 11. The eye movement analysis method according to claim 3, 4, 5 or 10, wherein the correlation coefficient is Spearman's rank correlation coefficient.