JP2012137639A - Head-mounted display, control program, and control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a head-mounted display capable of accurately detecting a fatigue degree of the eyes and notifying the detection result to a user, and to provide a control program and a control method.SOLUTION: A CPU displays a saccade inducing pattern on a display image (S18) so as to cause a saccade motion in the eyes of a wearer. The eyes generating the saccade motion causes a rapid ocular movement. The CPU detects the ocular movement (S19) and calculates the speed, moving distance, and a following rate of the sight lines of the eyeballs (S20). The CPU determines a degree of the asthenopia individually on the respective calculated parameters and comprehensively determines the degree of the asthenopia from the determination result (S21). The CPU displays a warning message on the display image according to the determined level. Therefore, the wearer viewing the warning message can realize the fatigue of the eyes.

Description

  The present invention relates to a head mounted display, a control program, and a control method.

  2. Description of the Related Art Conventionally, there is an image display device that projects image light according to a display image onto a user's eyes so that the user can visually recognize the display image. As an example, a head-mounted display (HMD) including a mounting unit that is mounted on a user's head is known. When the HMD is mounted on the head, as long as the eyes are open, the display image is viewed no matter where they are facing. For example, an image display device capable of preventing a user's eye strain in consideration of the use environment is known (see, for example, Patent Document 1). This image display apparatus determines the degree of eye fatigue based on the difference between the brightness of external light and the brightness of the display image, and notifies the user of suspension of use.

JP 2010-164782 A

  However, the image display device described in Patent Document 1 estimates the degree of eye fatigue based on the difference between the brightness of external light and the brightness of the display image, and directly measures the degree of eye fatigue. I don't mean. Therefore, there is a possibility that the user is notified of suspension of use even though fatigue does not actually accumulate in the eyes. In addition, because the degree of eye fatigue varies from person to person, the degree of eye fatigue estimated uniformly from the difference between the brightness of external light and the brightness of the displayed image does not accurately match the degree of fatigue of each individual. there is a possibility. If the threshold for determination can be changed, it is considered possible to make a determination according to an individual. However, it is unclear how to change the threshold of judgment by associating personal conditions such as eye fatigue and the difference between the brightness of external light and the brightness of the display image. There was also a problem that it took time and effort to decide.

  The present invention has been made to solve the above problems, and provides a head mounted display, a control program, and a control method that can accurately detect the degree of eye fatigue and notify the user of the detection result. With the goal.

  The head mounted display according to the first aspect of the present invention is a head mounted display capable of displaying an image superimposed on a real image in a wearer's visual field, the display unit displaying an image in the visual field, and the wearer An eye movement detection unit that detects the eye movement of the head, and a control unit that controls the operation of the head mounted display. The control unit is a soccer that induces saccade movement on the eyeball of the wearer by the display unit. A saccade-induced image display control step for displaying a saccade-induced image in the field of view, and the saccade-induced image is detected by the eye movement detection unit in a state where saccade motion is induced in the eyeball of the wearer by the saccade-induced image. A saccade motion state determination step for determining a saccade motion state of the wearer's eyeball based on eye movement; Fatigue that determines the degree of fatigue of the eyeball of the wearer based on a comparison between the saccade motion state determined in the motion state determination step and the threshold value stored in the threshold value storage unit that stores the threshold value of the saccade motion state A degree determination step, and a determination information output step for outputting determination information, which is information of a determination result of the fatigue degree in the fatigue degree determination step, to the outside.

  In the head mounted display which concerns on a 1st aspect, in order to detect a wearer's eye fatigue, a control part displays a saccade induced image on a wearer's visual field by a display part. The wearer's eyeball induces saccade movement. The eye movement detection unit detects the eye movement at that time. The control unit determines the saccade motion state of the wearer's eyeball based on the eyeball motion detected by the eyeball motion detection unit in a state where the saccade motion is induced in the wearer's eyeball. The control unit determines the degree of fatigue of the wearer's eyeball based on a comparison between the determined saccade motion state and the threshold value stored in the threshold value storage unit. The control unit outputs the determination result of the eyeball fatigue level to the outside. Therefore, since the wearer can be aware of the degree of fatigue of the eyeball, it is possible to appropriately determine whether to continue using or to rest according to the degree of eye fatigue. Thus, since the saccade motion state of the wearer's eyes is determined and the degree of eye fatigue is determined, it is possible to accurately grasp the wearer's eye state and notify the wearer of the result.

  Further, in the first aspect, when the control unit executes the saccade-induced image display control step, an image that induces saccade motion by changing the line of sight of the wearer in one direction of the visual field. The saccade-inducing image may be displayed. In the saccade-induced image, the wearer's line of sight is changed in one direction of the visual field, which causes rapid eye movement. Therefore, a saccade movement can be easily induced in the wearer's eyes.

  Further, in the first aspect, the control unit, when executing the saccade-induced image display control step, displays an image that induces saccade movement by changing the line of sight of the wearer in the horizontal direction of the visual field. The saccade-inducing image may be displayed. Therefore, the wearer's line of sight can be changed in an area that does not interfere with the image of characters, figures, etc. displayed in the field of view.

  In the first aspect, the control unit may further execute a threshold value registration step of storing the threshold value in the threshold value storage unit for each individual who is the wearer. Since the control unit can store the threshold value for each individual, even if there is an individual difference in the nature of the eye, the fatigue level of the wearer's eyeball can be determined using the threshold value corresponding to the wearer.

  Further, in the first aspect, the control unit, when executing the saccade motion state determination step, based on the eyeball motion detected by the eyeball motion detection unit, as the saccade motion state, the line of sight of the eyeball At least one of the speed, the moving amount, and the following degree may be determined. The speed of eye gaze, the amount of movement, and the degree of follow-up are effective as parameters characterizing the saccade motion state. By considering at least one of these, the saccade movement state of the eye can be accurately detected.

  Further, in the first aspect, the control unit detects the eye movement detection unit in a state in which saccade movement is induced in the eyeball of the wearer when executing the saccade movement state determination step. Based on the eyeball movement, the saccade movement state may determine all of the eye gaze speed, movement amount, and tracking degree. The speed of eye gaze, the amount of movement, and the degree of follow-up are effective as parameters characterizing the saccade motion state. Considering all of these, the saccade motion state of the eye can be detected more accurately.

  In the first aspect, the threshold storage unit stores threshold values for each item of the eye gaze speed, movement amount, and tracking degree, and the control unit executes the fatigue level determination step. In addition, when two or more items of the eye-gaze speed, amount of movement, and degree of following determined in the saccade motion state determination step are less than the threshold stored in the threshold storage unit, fatigue occurs. You may make it determine with a state. Therefore, it is possible to finely determine the degree of fatigue of the wearer's eyes.

  In the first aspect, the threshold storage unit stores, as the threshold, a first threshold and a second threshold smaller than the first threshold for each item, and the control unit stores the fatigue level. When the determination step is executed, if two or more items of the eye movement, the movement amount, and the tracking degree determined in the saccade movement state determination step are less than the second threshold, the fatigue You may make it determine with a state. Therefore, the degree of fatigue of the wearer's eyes can be determined more finely.

  A control program according to a second aspect of the present invention includes a display unit capable of displaying an image superimposed on a real image in a wearer's visual field, and an eye movement detection unit that detects the eye movement of the wearer. A saccade-triggered image display control step for displaying a saccade-triggered image for inducing a saccade motion on the eyeball of the wearer on the computer by the display unit on the computer. A saccade that determines a saccade motion state of the eyeball of the wearer based on the eyeball motion detected by the eyeball motion detection unit in a state where a saccade motion is induced in the eyeball of the wearer based on the eyeball trigger image. An exercise state determination step, the saccade exercise state determined in the saccade exercise state determination step, and the sucker A fatigue level determination step for determining a fatigue level of the eyeball of the wearer based on a comparison with the threshold value stored in a threshold value storage unit that stores a threshold value of the exercise state, and the determination of the fatigue level in the fatigue level determination step A determination information output step of outputting determination information as result information to the outside is executed.

  In the control program according to the second aspect, the saccade-induced image display control step displays a saccade-induced image in the field of view in order to detect fatigue of the wearer's eyes. The wearer's eyeball induces saccade movement. The eye movement detection unit detects the eye movement at that time. The saccade motion state determination step determines the saccade motion state of the wearer's eyeball based on the eyeball motion detected by the eyeball motion detection unit in a state where the saccade motion is induced in the wearer's eyeball. The fatigue level determination step determines the fatigue level of the wearer's eyeball based on a comparison between the determined saccade motion state and the threshold value stored in the threshold value storage unit. The determination information output step outputs the determination result of the eyeball fatigue level to the outside. Therefore, since the wearer can be aware of the degree of fatigue of the eyeball, it is possible to appropriately determine whether to continue using or to rest according to the degree of eye fatigue. Thus, since the saccade motion state of the wearer's eyes is determined and the degree of eye fatigue is determined, it is possible to accurately grasp the wearer's eye state and notify the wearer of the result.

  A control method according to a third aspect of the present invention includes a display unit capable of displaying an image superimposed on a real image in a wearer's visual field, and an eye movement detection unit that detects the eye movement of the wearer. A method for controlling a mount display, comprising: a saccade-induced image display control step for displaying a saccade-induced image that induces a saccade movement on the eyeball of the wearer in the field of view by the display unit; and the saccade-induced image The saccade motion state determination for determining the saccade motion state of the wearer's eye based on the eye motion detected by the eye motion detection unit in a state where saccade motion is induced in the eye of the wearer. Storing the step, the saccade motion state determined in the saccade motion state determination step, and the threshold value of the saccade motion state Based on a comparison with the threshold value stored in the value storage unit, a fatigue level determination step for determining the fatigue level of the wearer's eyeball, and determination information that is information on a determination result of the fatigue level in the fatigue level determination step And a determination information output step for outputting to the outside.

  In the control method according to the third aspect, the saccade-induced image display control step displays a saccade-induced image in the field of view in order to detect fatigue of the wearer's eyes. The wearer's eyeball induces saccade movement. The eye movement detection unit detects the eye movement at that time. The saccade motion state determination step determines the saccade motion state of the wearer's eyeball based on the eyeball motion detected by the eyeball motion detection unit in a state where the saccade motion is induced in the wearer's eyeball. The fatigue level determination step determines the fatigue level of the wearer's eyeball based on a comparison between the determined saccade motion state and the threshold value stored in the threshold value storage unit. The determination information output step outputs the determination result of the eyeball fatigue level to the outside. Therefore, since the wearer can be aware of the degree of fatigue of the eyeball, it is possible to appropriately determine whether to continue using or to rest according to the degree of eye fatigue. Thus, since the saccade motion state of the wearer's eyes is determined and the degree of eye fatigue is determined, it is possible to accurately grasp the wearer's eye state and notify the wearer of the result.

It is a perspective view of RSD5. In the display image 8, it is a figure which shows the state in which the saccade induction patterns 41 and 42 are lighted on and off alternately. It is a conceptual diagram which shows the positional relationship of the eyeball E and the saccade induction patterns 41 and 42 displayed on the display image 8. FIG. It is a figure which shows the state where the warning message A is displayed in the display image 8. FIG. It is a block diagram which shows the electric constitution of RSD5. 3 is a conceptual diagram showing various storage areas of a flash memory 14. FIG. It is a conceptual diagram of the speed / fatigue determination table 1421. It is a conceptual diagram of the movement amount / fatigue determination table 1431. It is a conceptual diagram of a following ratio / fatigue determination table 1441. FIG. 6 is a conceptual diagram of a first / second threshold storage area 145. It is a conceptual diagram of the warning message table 1461. It is a flowchart of the eye strain notification process by CPU11. 13 is a flowchart showing a continuation of FIG.

  Hereinafter, a retinal scanning display device 5 according to an embodiment of the present invention will be described with reference to the drawings. These drawings are used to explain technical features that can be adopted by the present invention. The configuration of the apparatus, the flowcharts of various processes, and the like that are described are not intended to be limited to only that, but are merely illustrative examples.

  First, the physical configuration of the retinal scanning display device 5 will be described with reference to FIG. The retinal scanning display device 5 (Retinal Scanning Display, hereinafter referred to as “RSD5”) is a type of head mounted display that is used by being worn on the user's head. The RSD 5 includes a projection device 200 that projects image light onto the user's eyes, and a control device 100 that controls the operation of the projection device 200. The projection device 200 is used while being worn on a user's glasses 220. The projection device 200 can cause the wearer to visually recognize the display image 8 by irradiating light toward the pupil from the vicinity of the wearer's eye. The projection device 200 is detachably connected to the control device 100 via a cable 190 in which an optical fiber 191 (see FIG. 5) and signal lines 192, 193, 194, 195, and 196 (see FIG. 5) are integrated. ing.

  The projection device 200 includes an emission device 210 and a half mirror 250. The emission device 210 includes scanning mirrors (a high-speed scanning mirror 32 and a low-speed scanning mirror 34 described later shown in FIG. 5). The scanning mirror scans laser light (hereinafter referred to as image light) modulated in accordance with a data signal (hereinafter referred to as image signal) of an image that is visually recognized by the wearer. The scanned image light is output from the emission device 210 toward the half mirror 250.

  The half mirror 250 reflects the image light output from the output device 210 toward the wearer's pupil. The half mirror 250 has a predetermined reflectance (for example, 50%). The half mirror 250 transmits a part of the external light from the outside and guides it to the wearer's pupil. That is, the half mirror 250 guides image light incident from the side of the wearer and external light from the outside to the wearer's pupil. Therefore, the wearer can visually recognize the actual view and the display image 8 based on the image light. The half mirror 250 may be a prism. The emission device 210 may be integrated with the glasses 220. Instead of the half mirror 250, a known mirror that does not transmit external light may be used.

  The control device 100 generates image light based on the image signal. The generated image light is output to the emission device 210 via the cable 190. The control device 100 controls the drive of the scanning mirror included in the emission device 210 via the cable 190. As a result, the control device 100 controls the scanning of the image light executed in the emission device 210. The control device 100 includes a connector 207 for connecting to the PC 300 (see FIG. 5). An example of the connector 207 is a USB (registered trademark) connector.

  The user can store new image data in the control device 100 connected to the PC 300 by operating the PC 300. The user can set or change various parameters stored in the control device 100 connected to the PC 300 by operating the PC 300.

  The RSD 5 of the present embodiment detects the saccade movement of the wearer's eyes, and can notify the wearer of the suspension of use according to the degree of eye fatigue determined based on the detection result. In this embodiment, the saccade movement is “impulsive rapid eye movement that directs the line of sight to a point when a sudden change in the user's line of sight position or a new visual stimulus occurs”. .

  As shown in FIG. 2, a saccade induction pattern display area 40 is provided in the horizontal direction at the bottom of the display image 8. On the upper side, a warning message display area 50 is provided in the horizontal direction. In the saccade induction pattern display area 40, saccade induction patterns 41 and 42 for inducing a saccade movement to the wearer's eyes are displayed. The saccade induction patterns 41 and 42 are circular points, for example, and are at positions separated from each other by a predetermined interval in the horizontal direction. Furthermore, as shown in FIG. 3, the saccade induction patterns 41 and 42 are at positions where the viewing angle of the wearer's eyeball E is ± 10 degrees. The saccade induction patterns 41 and 42 are repeatedly turned on and off alternately. The wearer unconsciously follows the saccade induction patterns 41 and 42 that light up with his / her eyes. The RSD 5 sets the state of the eye movement at this time as the saccade movement state and measures the amount of the state.

  The RSD 5 determines the degree of eye strain by comparing the amount of state of the eyeball that is regularly measured with threshold values (first threshold value and second threshold value described later) stored in advance. For example, if the RSD 5 determines that the wearer's eyeball is tired, the warning message display area 50 of the display image 8 displays “The eye is getting tired. Take a break soon. Please display a warning message. The wearer who sees this notices that his eyes are getting tired, and can take an appropriate action according to his eye fatigue, such as suspending use and taking a break.

  Next, parameters that characterize the saccade motion state will be described. As parameters characterizing the saccade motion state, the line-of-sight speed (X), the movement amount (Y), and the following degree (Z) are used. Each of these parameters is for quantitatively measuring the saccade motion state.

  Next, a method for measuring the speed, the movement amount, and the following degree will be described. In the present embodiment, in the display image 8, the saccade induction patterns 41 and 42 are alternately turned on and off every second for 10 seconds. The speed at which the wearer's line of sight moves at this time (deg / sec) and the amount of movement (deg) are measured. The degree of follow-up is the rate (%) at which saccade movement occurred. Whether or not the saccade movement has occurred is determined in a state where the saccade induction patterns 41 and 42 are alternately displayed, the line-of-sight speed is a predetermined value or more, and the line-of-sight movement amount is a predetermined amount or more. If so, it may be determined that a saccade movement has occurred. As a method of calculating the tracking degree, for example, the saccade induction patterns 41 and 42 are displayed ten times, and the number of times that the saccade movement has occurred in the eyeball is counted, and the ratio is calculated. Note that these calculation processes are executed by the CPU 11 described later of the control device 100.

  Next, a gaze detection method will be described. In this embodiment, gaze detection is performed in order to measure the saccade motion state. Various known detection methods can be applied. For example, as a technique using an external camera, there are a “face feature modeling method” and a “pupil three-dimensional model method” (cornea / pupil / sclera reflection method). At present, it is preferable to use the latter in terms of accuracy. In the latter method, there is a difference between using a Purkinje image, pupil reflection, or seeing the difference between white eyes and black eyes. In this embodiment, it is performed by the eyeball imaging camera 30 (see FIG. 5) provided in the projection device 200. Near-infrared light from an infrared light source is superimposed on the image light output from the emission device 210 by a wavelength selection filter or the like inserted in the optical path. The center of the pupil imaged by the eyeball imaging camera 30 having photosensitivity for the near-infrared region is the line-of-sight position on the captured image. This technique is easy to use because no special calibration is required.

  Next, the electrical configurations of the control device 100 and the projection device 200 will be described with reference to FIG. The control device 100 includes a CPU 11, a ROM 12, a RAM 13, a flash memory 14, and a communication I / F 15. The CPU 11 reads out various programs stored in the ROM 12 to execute each process described later. Various programs are stored in the ROM 12. The RAM 13 temporarily stores data necessary for each process. The flash memory 14 stores various data, various tables, and the like which will be described later. The communication I / F 15 performs timing control during communication with the PC 300 connected via the connector 207 (see FIG. 1).

  The control device 100 includes an LD (Laser Diode) 23 and an LD driving unit 22. The LD 23 includes a blue output LD, a green output LD, and a red output LD. The LD 23 outputs image light composed of blue, green, and red laser beams to the optical fiber 191. The LD driving unit 22 controls the LD 23 based on the image signal received from the CPU 11 and causes the LD 23 to output image light. The control device 100 includes a high-speed scanner driving unit (hereinafter referred to as “HS driving unit”) 24 and a low-speed scanner driving unit (hereinafter referred to as “LS driving unit”) 25. It has.

  The HS driving unit 24 controls driving of a high-speed scanning mirror (hereinafter referred to as “HS”) 32 included in the projection apparatus 200 via a signal line 192. Specifically, the HS drive unit 24 performs drive control by resonantly swinging the HS 32 at a predetermined frequency. The LS driving unit 25 controls driving of a low-speed scanning mirror (hereinafter referred to as “LS”) 34 included in the projection apparatus 200 via a signal line 194. Specifically, the LS drive unit 25 performs drive control by forcibly driving the LS 34 in a sawtooth manner without resonance.

  The projection apparatus 200 includes HS32 and LS34. The HS 32 resonates and oscillates under the control of the HS drive unit 24 and scans the image light output from the LD 23 at a predetermined frequency in the vertical direction (for example, the vertical direction of the image). The LS 34 is forcibly driven in a non-resonant manner in a sawtooth shape under the control of the LS drive unit 25, and scans image light scanned by the HS 32 at a predetermined frequency in the horizontal direction (for example, the left-right direction of the image). The image light is scanned two-dimensionally by the scanning of HS32 and LS34.

  The projection device 200 includes lenses 31 and 33, an eyepiece lens 35, and a half mirror 250. The lens 31 guides the image light output from the optical fiber 191 to the HS 32. The lens 33 guides the image light reflected by the HS 32 to the LS 34. The eyepiece 35 guides the image light reflected by the LS 34 to the half mirror 250. The half mirror 250 guides image light to the pupil.

  In the above description, the HS 32 scans the image light in the vertical direction and the LS 34 scans the image light in the horizontal direction. For example, the HS 32 scans the image light in the horizontal direction, and the LS 34 scans the image light in the vertical direction. You may scan.

  The projection apparatus 200 includes an HS detection unit 36 and an LS detection unit 37. The HS detector 36 detects the driving state (resonance frequency, amplitude, phase, etc.) of the HS 32. The LS detection unit 37 detects the driving state of the LS 34. The HS detection unit 36 is connected to the CPU 11 via the signal line 193. The CPU 11 can recognize the driving state of the HS 32 detected by the HS detection unit 36. The LS detection unit 37 is connected to the CPU 11 via the signal line 195. The CPU 11 can recognize the driving state of the LS 34 detected by the LS detection unit 37.

  Next, various storage areas of the flash memory 14 will be described with reference to FIG. The flash memory 14 includes an image data storage area 141, a speed / fatigue determination table storage area 142, a movement amount / fatigue determination table storage area 143, a follow-up / fatigue determination table storage area 144, a first / second threshold value. A storage area 145 and a warning message table storage area 146 are provided at least.

  The image data storage area 141 stores image data such as characters, figures, and symbols to be displayed on the display image 8. The speed / fatigue determination table storage area 142 stores a speed / fatigue determination table 1421 (see FIG. 7) for determining the degree of eye strain based on the measured line-of-sight speed. The movement amount / fatigue determination table storage area 143 stores a movement amount / fatigue determination table 1431 (see FIG. 8) for determining the degree of eye strain based on the measured movement amount of the line of sight. The tracking degree / fatigue determination table storage area 144 stores a tracking degree / fatigue determination table 1441 (see FIG. 9) for determining the degree of eye strain based on the calculated tracking degree of the line of sight. The first / second threshold storage area 145 is a first threshold value and a second threshold value used for determining the degree of eye strain of the eye in three stages with respect to the line-of-sight speed, the movement amount, and the following degree (see FIG. 10). Is stored. The warning message table storage area 146 stores a warning message table 1461 (see FIG. 11).

  Next, the speed / fatigue determination table 1421 will be described with reference to FIG. The speed / fatigue determination table 1421 stores the eyestrain degrees A, B, and C in association with each other in three stages of the speed of the line of sight when the saccade motion is occurring. Eye strain is A, B, and C in order from the lighter to the heavier. That is, the degree of eye strain A is the least tired and the degree of eye strain C is the most tired. The first threshold value for speed is X1, and the second threshold value is X2. X2 is a value smaller than X1. In the speed / fatigue determination table 1421, the degree of eye strain A is in the range where the speed is X1 or more, the degree of eye strain B in the range where the speed is X2 or more and less than X1, and the eye is in the range where the speed is less than X2. I remember the degree of fine fatigue C. That is, the speed / fatigue determination table 1421 reflects that the eyeball movement becomes dull and the line-of-sight speed decreases as fatigue accumulates in the eyeball.

  Next, the movement amount / fatigue determination table 1431 will be described with reference to FIG. The movement / fatigue determination table 1431 divides the amount of movement (amplitude) of the line of sight when a saccade movement is occurring into three stages, and stores the degree of eye strain A, B, and C in association with each other. . As with the speed / fatigue determination table 1421, the degree of eye strain is A, B, and C in order from the lighter to the heavier. The first threshold value of the movement amount is Y1, and the second threshold value is Y2. Y2 is a value smaller than Y1. In the movement amount / fatigue determination table 1431, the eye strain A is in the range where the movement amount is Y1 or more, the eye fatigue degree B is in the range in which the movement amount is Y2 or more and less than Y1, and the speed is less than Y2. On the other hand, the degree of eye strain C is memorized. That is, the movement / fatigue determination table 1431 reflects that the movement of the eyeball becomes dull and the movement amount (amplitude) of the line of sight becomes shorter as fatigue accumulates in the eyeball.

  Next, the following degree / fatigue determination table 1441 will be described with reference to FIG. The tracking degree / fatigue determination table 1441 stores the tracking degree of the line of sight when the saccade movement is occurring in three stages, and stores the eyestrain degrees A, B, and C in association with each other. As with the speed / fatigue determination table 1421, the degree of eye strain is A, B, and C in order from the lighter to the heavier. The first threshold value of the following degree is Z1, and the second threshold value is Z2. Z2 is a value smaller than Z1. In the tracking degree / fatigue determination table 1441, the eye fatigue degree A is in the range where the tracking degree is Z1 or more, the eye fatigue degree B is in the range where the following degree is Z2 or more and less than Z1, and the tracking degree is less than Z2. In contrast, the eye strain C is stored. In other words, the follow-up / fatigue determination table 1441 reflects that as the eye accumulates fatigue, the movement of the eyeball becomes dull, and the follow-up degree of the line of sight with respect to the saccade induction patterns 41 and 42 (see FIG. 2) decreases. ing.

  Next, the first and second threshold values stored in the threshold value storage area 145 will be described with reference to FIG. The threshold storage area 145 includes a first eye used for determining the degree of eye strain for the sight line speed (X), the movement amount (Y), and the tracking degree (Z) as parameters of the saccade motion state. A threshold value and a second threshold value are stored. These first threshold value and second threshold value are changed from time to time based on the speed of the wearer's line of sight, the amount of movement, and the degree of follow-up calculated when the RSD 5 is activated. A method for calculating the first threshold and the second threshold will be described later.

  Next, the warning message table 1461 will be described with reference to FIG. The warning message table 1461 is obtained by aggregating the determination results A, B, and C respectively determined using the speed / fatigue determination table 1421, the movement amount / fatigue determination table 1431, and the follow-up degree / fatigue determination table 1441. Based on the result, the determination of the overall degree of eye strain is divided into three levels of levels 1 to 3, and a warning message to notify the wearer is stored according to each level.

  Level 1 corresponds to the case where the degree of eye strain “B” is one and the eye strain “C” is not present. In this case, since it is presumed that the wearer's eyes are not yet tired, there is no need to notify the wearer of the suspension of use. Therefore, no warning message is stored for level 1. Level 2 corresponds to a case where the degree of eye strain “B” is two or more. In this case, since it is estimated that the wearer's eyes are getting tired, it is necessary to notify the wearer of the suspension of use. Therefore, for level 2, a warning message “The eye is getting tired. Take a break soon” is stored.

  Level 3 corresponds to a case where the degree of eye strain “C” is two or more. In this case, since it is estimated that the wearer's eyes are considerably tired, it is necessary to notify the wearer of suspension of use. Therefore, for level 3, a warning message “Eye is considerably tired. Stop using it and take a rest.” Is stored. In this manner, the warning message displayed on the display image 8 (see FIGS. 2 and 4) can be changed according to the level of eye strain. Note that the overall determination of the degree of eye strain may be determined as level 2 if, for example, there are one parameter of the degree of eye strain “A”, “B”, and “C”.

  Next, the “eye strain notification process” executed by the CPU 11 is executed with reference to the flowcharts of FIGS. When the control device 100 is activated, the CPU 11 calls the “eye strain notification program” stored in the program ROM 12 (see FIG. 5) and executes this processing.

  The CPU 11 displays the saccade induction patterns 41 and 42 in the saccade induction pattern display area 40 (see FIG. 2) of the display image 8 as an initial state of the wearer's eyes (S11). As described above, the saccade induction patterns 41 and 42 are alternately turned on and off every second for 10 seconds. The wearer's eyes cause an impulsive rapid eye movement that directs the line of sight toward the saccade-inducing patterns 41, 42. The eyeball is in a saccade motion state.

  CPU11 detects a gaze position from a wearer's eyeball which is a saccade movement state, and detects eyeball movement from the detected gaze position (S12). The CPU 11 detects the position of the center of the pupil from the captured image of the eye captured by the eyeball imaging camera 30 (see FIG. 5), and recognizes the detected position as the line-of-sight position on the captured image. The CPU 11 detects the impulsive rapid eye movement of the wearer based on the movement of the wearer's line-of-sight position. The CPU 11 calculates the gaze speed (X0), the movement amount (Y0), and the tracking rate (Z0) of the wearer as the initial state of the eyes when not tired yet (S13). The CPU 11 temporarily stores the calculated X0, Y0, and Z0 as “reference threshold values” in the RAM 13 (see FIG. 5) (S14).

The CPU 11 calculates the first threshold value (X1, Y1, Z1) and the second threshold value (X2, Y2, Z2) of the speed, the movement amount, and the tracking ratio based on the reference threshold value temporarily stored in the RAM 13 (S15). . The calculation method of the first threshold value and the second threshold value is as follows. The first threshold coefficient for calculating the first threshold is α, and the second threshold coefficient for calculating the second threshold is β. α and β are real numbers in the range of 0-1. In this case, the first threshold value and the second threshold value of the speed (X) are calculated by the following equations.
First threshold (X1) = X0 × α
Second threshold (X2) = X0 × β
Y1 and Z1 are similarly calculated by multiplying Y0 and Z0 by a threshold coefficient α. Similarly, Y2 and Z2 are calculated by multiplying Y0 and Z0 by a threshold coefficient β.

A specific example will be described. For example, it is assumed that the reference threshold value of each parameter stored in the RAM 13 is the following value. These values are calculated in a state where saccade motion is induced in the wearer's eye when the control device 100 is activated.
X0 = 400 (deg / sec), Y0 = 18 (deg), Z0 = 90 (%)

The CPU 11 calculates the first threshold value and the second threshold value based on the reference threshold values (X0, Y0, Z0) of each parameter stored in the RAM 13. For example, the first threshold coefficient α = 0.7 and the second threshold coefficient β = 0.3. The CPU 11 multiplies the reference threshold value of each parameter by the first threshold coefficient α and the second threshold coefficient β, thereby obtaining the first threshold value (X1, Y1, Z1) and the second threshold value (X2, Y2, Z2) of each parameter. Is calculated as follows.
・ X1 = 280 (deg / sec)
・ X2 = 120 (deg / sec)
・ Y1 = 12.6 (deg)
・ Y2 = 5.4 (deg)
・ Z1 = 63 (%)
・ Z2 = 27 (%)

  In this way, the CPU 11 stores the first threshold value (X1, Y1, Z1) and the second threshold value (X2, Y2, Z2) calculated for each parameter in the threshold storage area 145 (see FIG. 10) of the flash memory 14. Remember each one. The first threshold value and the second threshold value stored in the threshold value storage area 145 are a speed / fatigue determination table 1421 (see FIG. 7), a movement amount / fatigue determination table 1431 (see FIG. 8), and a follow-up / fatigue determination table 1441 (FIG. 9), the first threshold value and the second threshold value.

  Next, as shown in FIG. 13, the CPU 11 starts a timer counter t (S16). The CPU 11 determines whether a predetermined time (for example, 10 minutes) has elapsed (S17). The predetermined time is not limited to 10 minutes, and may be shorter or longer. When the CPU 11 determines that the predetermined time has not yet elapsed (S17: NO), the CPU 11 returns to S17 and enters a standby state.

  When the CPU 11 determines that the predetermined time has elapsed (S17: YES), the CPU 11 displays the saccade induction patterns 41 and 42 (see FIG. 2) in the saccade induction pattern display area 40 of the display image 8 (S18). Similarly to the above, the CPU 11 detects the eye movement from the movement of the line of sight of the wearer in the saccade movement state in the captured image of the eyeball imaging camera 30 (S19). The CPU 11 calculates the wearer's line-of-sight speed (X), movement amount (Y), and follow-up ratio (Z) from the detected eye movements (S20).

  The CPU 11 calculates the speed / fatigue determination table 1421, the movement amount / fatigue determination table 1431, and the tracking degree / fatigue determination table for the calculated wearer's line-of-sight speed (X), movement amount (Y), and tracking ratio (Z). 1441 is used to determine the degree of eye strain for each parameter (S21).

  A specific example will be described. The first threshold value and the second threshold value of each parameter are the calculated values of the specific examples described above. For example, when the calculated line-of-sight speed (X) is 200 (deg / sec), referring to the speed / fatigue determination table 1421 shown in FIG. 7, the second threshold (120) or more and less than the first threshold (280). is there. Therefore, the degree of eye strain based on the speed can be determined as “B”. For example, when the calculated movement amount (Y) of the line of sight is 10.4 (deg), referring to the movement amount / fatigue determination table 1431 shown in FIG. 8, the second threshold value (5.4) or more and the first threshold value (12 .6) is less. Therefore, the degree of eye strain based on the amount of movement can be determined as “B”. For example, when the calculated tracking degree (Z) of the line of sight is 15 (%), referring to the tracking degree / fatigue determination table 1441 shown in FIG. 9, it is less than the second threshold (27). Therefore, the degree of eye strain based on the following degree can be determined as “C”.

  The CPU 11 comprehensively determines the determination result of the degree of eye strain of each parameter, and determines whether the degree of eye strain is level 1 (S22). Level 1 is a case where the degree of eye strain “B” is one and there is no eye strain “C” as shown in FIG. When the CPU 11 determines that the degree of eye strain is level 1 (S22: YES), the CPU 11 resets the timer counter t without displaying a warning message on the display image 8 based on the warning message table 1461 (see FIG. 11). (S28). The CPU 11 returns to S16, starts the timer counter t again, waits for the elapse of a predetermined time, and repeats the above processing.

  When the CPU 11 determines that the degree of eye strain is not level 1 (S22: NO), it subsequently determines whether it is level 2 (S23). Level 2 is a case where the degree of eye strain “B” is two or more as shown in FIG. When the CPU 11 determines that the degree of eye strain is level 2 (S23: YES), the CPU 11 displays the warning message A in the warning message display area 50 of the display image 8 based on the warning message table 1461 (see FIG. 11) (S24). ). The warning message A prompts the wearer to take a break. The wearer sees the warning message A displayed on the display image 8 and realizes that his eyes are tired. The wearer turns off the power of the RSD 5 and pauses use, and takes appropriate actions such as taking a break. It comes to take.

  When the degree of eye strain “C” is two or more, the CPU 11 determines that the degree of eye strain is level 3 (S23: NO), and displays the display image 8 based on the warning message table 1461 (see FIG. 11). The warning message B is displayed in the warning message display area 50 (S25). The warning message B strongly urges the wearer to stop using it. The wearer sees the warning message B displayed on the display image 8 and realizes that his eyes are considerably tired. Immediately, the wearer turns off the power of the RSD 5 to suspend use and takes a proper rest. Take action.

  Subsequently, the CPU 11 determines whether or not the RSD 5 is powered off based on the input of the wearer (S26). When the CPU 11 determines that the RSD 5 is powered off based on the input of the wearer (S26: YES), the CPU 11 powers off the RSD 5 (S27) and ends the process. The CPU 11 may turn off only the display of the display image 8 of the half mirror 250 instead of turning off the power of the RSD 5. If the CPU 11 determines that the RSD 5 is not powered off based on the input of the wearer (S26: NO), the CPU 11 resets the timer counter t (S28). The CPU 11 returns to S16, starts the timer counter t, waits for the elapse of a predetermined time, and repeats the above processing.

  In the above description, the projection apparatus 200 shown in FIG. 1 corresponds to the “display unit” of the present invention. The eyeball imaging camera 30 shown in FIG. 1 corresponds to the “eyeball movement detection unit” of the present invention. The CPU 11 shown in FIG. 5 corresponds to a “control unit” of the present invention. The first / second threshold storage area 145 of the flash memory 14 shown in FIG. 6 corresponds to the “threshold storage section” of the present invention.

  The process of S18 shown in FIG. 12 corresponds to the “saccade-induced image display control step” of the present invention. The process of S20 shown in FIG. 13 corresponds to the “saccade motion state determination step” of the present invention. The process of S21 corresponds to the “fatigue level determination step” of the present invention. The processes of S24 and S25 correspond to the “determination information output step” of the present invention. The processing of S11 to S15 in FIG. 12 corresponds to the “threshold storage processing step” of the present invention.

  As described above, the RSD 5 according to the present embodiment displays the saccade induction patterns 41 and 42 on the display image 8 and forcibly causes the saccade movement in the wearer's eyes. The eye that has caused the saccade movement causes a rapid eye movement and becomes a saccade movement state. The CPU 11 of the control device 100 detects the eye movement at this time from the line of sight of the wearer. As the parameters of the saccade motion state, the line-of-sight speed, the amount of movement, and the tracking rate are calculated. Each parameter changes according to eye fatigue. The CPU 11 determines the degree of eye strain for each such parameter. The CPU 11 comprehensively determines the degree of eye strain based on the determination result determined for each parameter. The CPU 11 determines the degree of eye strain at levels 1, 2, and 3, and displays a warning message on the display image 8 according to the level. Thereby, since the wearer who sees the warning message can be aware of eye fatigue, it is possible to encourage appropriate use according to the degree of eye strain.

  In the present embodiment, in particular, in the process of S15, the first threshold value and the second threshold value used for determining the degree of eye strain in each parameter of S22 are calculated based on the reference threshold value calculated when the control device 100 is activated. ing. The reference threshold is the wearer's line-of-sight speed, amount of movement, and tracking rate detected when the control device 100 is activated. Since the first threshold value and the second threshold value are calculated based on this reference threshold value, it is possible to determine the degree of eye strain according to each wearer. Therefore, an appropriate fatigue determination can be performed on the wearer's eyes. Furthermore, the eye movement of the wearer's eyes varies depending on the time period in which they are used. In the present embodiment, since the reference threshold value is calculated every time the control device 100 is activated, it is possible to reliably detect eye strain due to the use of the RSD 5.

  In the present embodiment, in particular, the first threshold value and the second threshold value are used as the threshold values used for the determination of the degree of eye strain, so that the degree of eye strain of the wearer can be determined in more detail. Furthermore, the warning message according to the degree of eyestrain of a wearer can be displayed by determining in detail the degree of eyestrain.

  In addition, this invention is not limited to the said embodiment, A various deformation | transformation is possible. For example, in the above-described embodiment, the degree of eye strain is determined using the speed, the moving amount, and the following rate as each parameter of the saccade motion state. For example, determination may be made using any one of these parameters, and another parameter may be added to determine the degree of eye strain. In addition, when determining using one parameter, it is preferable to use the speed of a line of sight.

  Moreover, in the said embodiment, although the eye strain degree is each determined in three steps about each parameter of a saccade movement state, it may be two steps or more.

  In the above embodiment, the first threshold value and the second threshold value are calculated every time the control device 100 is activated. For example, when it is used next time, the degree of eye strain may be determined using the first threshold value and the second threshold value previously stored in the first / second threshold value storage area 145 of the flash memory 14. In this case, the first / second threshold storage area 145 needs to store the first threshold and the second threshold for each individual.

  For example, an identification ID as identification information for identifying an individual is stored in the flash memory 14 of the control device 100, and a first threshold value and a second threshold value are stored for each identification ID. At the next use, the wearer inputs his / her identification ID on the PC 300. The CPU 11 of the control device 100 can specify the first threshold value and the second threshold value corresponding to the input identification ID, and can use them for the determination of the degree of eye strain of the wearer. Further, the first threshold value and the second threshold value for each individual may be registered in advance in the first / second threshold storage area 145 of the flash memory 14.

  Moreover, in the said embodiment, as shown in FIG. 2, the saccade induction patterns 41 and 42 are each displayed in the position mutually separated in the horizontal direction. For example, the saccade induction patterns 41 and 42 may be separated in the up-down direction or the oblique direction. Preferably, any position that does not interfere with characters, figures, symbols, and the like displayed on the display image 8 may be used.

  In the above embodiment, when the degree of eye strain is “A”, the warning message is not displayed on the display image 8 because it is not necessary to warn the wearer of the suspension of use. For example, when the degree of eye strain is “A”, a message indicating that the user is not tired may be displayed. Further, the determination result of eye strain may be output to the PC 300. Further, the content of the warning message is not limited to the above embodiment.

  Moreover, in the said embodiment, although the speed | velocity | rate of the eyes | visual_axis of the eyeball, the movement amount, and the following degree were used as a parameter which characterizes a saccade movement state, you may use state quantities other than these.

  In the above embodiment, as the “determination information output step” of the present invention, “determination information” is output using the projection device 200. However, any other configuration can be used as long as the determination result can be notified to the wearer. The configuration can also be applied. For example, a method of outputting sound or providing LEDs (red / yellow / green) for displaying a determination result on the control device 100 can be used.

  In the above-described embodiment, the retinal scanning projection apparatus 200 has been described as an example of the “display unit” of the present invention. However, for example, a display using a two-dimensional display element such as an LCD, DMD, or organic EL is also applicable. Is possible.

5 Retina scanning display device 8 Display image 11 CPU
14 flash memory 30 eyeball camera 41, 42 saccade induction pattern 100 control device 200 projection device

Claims (10)

A head mounted display capable of displaying an image superimposed on a real image in the wearer's field of view,
A display for displaying an image in the field of view;
An eye movement detector for detecting the eye movement of the wearer;
A control unit for controlling the operation of the head mounted display,
The controller is
A saccade-induced image display control step for displaying, in the field of view, a saccade-induced image that induces saccade movement on the eyeball of the wearer by the display unit;
Based on the eye movement detected by the eye movement detection unit in a state where saccade movement is induced in the eye of the wearer by the saccade induction image, the saccade movement state of the eye of the wearer is determined. A saccade motion state determination step;
Based on the comparison between the saccade motion state determined in the saccade motion state determination step and the threshold value stored in the threshold storage unit that stores the threshold value of the saccade motion state, the degree of fatigue of the eyeball of the wearer is calculated. A step of determining fatigue level;
And a determination information output step of outputting determination information, which is information of a determination result of the fatigue level in the fatigue level determination step, to the outside.   When the control unit executes the saccade-induced image display control step, an image that induces saccade movement by changing the gaze of the wearer in one direction of the visual field is used as the saccade-induced image. The head mounted display according to claim 1, wherein the head mounted display is displayed.   The control unit, when executing the saccade-induced image display control step, an image that induces saccade movement by changing the gaze of the wearer in the horizontal direction of the visual field as the saccade-induced image The head mounted display according to claim 2, wherein the head mounted display is displayed. The control unit further includes:
The head-mounted display according to any one of claims 1 to 3, wherein a threshold value registration step of storing the threshold value in the threshold value storage unit for each individual who is the wearer is executed.   The control unit, when executing the saccade movement state determination step, based on the eye movement detected by the eye movement detection unit, as the saccade movement state, the eye gaze speed, the amount of movement, and 5. The head mounted display according to claim 1, wherein at least one of the following degrees is determined.   The controller is configured to execute the saccade motion state determination step based on the eyeball motion detected by the eyeball motion detection unit in a state where saccade motion is induced in the eyeball of the wearer. 6. The head-mounted display according to claim 5, wherein all of the eye gaze speed, amount of movement, and degree of follow-up are determined as a moving state. The threshold storage unit stores a threshold for each item of the eye gaze speed, movement amount, and tracking degree,
When the control unit executes the fatigue level determination step,
When two or more items of the line-of-sight speed, amount of movement, and degree of follow-up determined in the saccade motion state determination step are less than the threshold stored in the threshold storage unit, The head mounted display according to claim 6, wherein the head mounted display is determined. The threshold storage unit stores, as the threshold, a first threshold and a second threshold smaller than the first threshold for each item,
When the control unit executes the fatigue level determination step,
The fatigue state is determined when two or more of the eye movement, movement amount, and tracking degree items determined in the saccade movement state determination step are less than the second threshold value. The head mounted display according to claim 7. A control program for a head-mounted display comprising a display unit capable of displaying an image superimposed on a real image and an eye movement detection unit for detecting the eye movement of the wearer in the field of view of the wearer,
On the computer,
A saccade-induced image display control step for displaying, in the field of view, a saccade-induced image that induces saccade movement on the eyeball of the wearer by the display unit;
Based on the eye movement detected by the eye movement detection unit in a state where saccade movement is induced in the eye of the wearer by the saccade induction image, the saccade movement state of the eye of the wearer is determined. A saccade motion state determination step;
Based on the comparison between the saccade motion state determined in the saccade motion state determination step and the threshold value stored in the threshold storage unit that stores the threshold value of the saccade motion state, the degree of fatigue of the eyeball of the wearer is calculated. A step of determining fatigue level;
A control program for executing a determination information output step of outputting determination information, which is information of a determination result of the fatigue level in the fatigue level determination step, to the outside. In a wearer's field of view, a control method for a head-mounted display comprising: a display unit capable of displaying an image superimposed on a real image; and an eye movement detection unit for detecting the eye movement of the wearer,
A saccade-induced image display control step for displaying, in the field of view, a saccade-induced image that induces saccade movement on the eyeball of the wearer by the display unit;
Based on the eye movement detected by the eye movement detection unit in a state where saccade movement is induced in the eye of the wearer by the saccade induction image, the saccade movement state of the eye of the wearer is determined. A saccade motion state determination step;
Based on the comparison between the saccade motion state determined in the saccade motion state determination step and the threshold value stored in the threshold storage unit that stores the threshold value of the saccade motion state, the degree of fatigue of the eyeball of the wearer is calculated. A step of determining fatigue level;
A control method comprising: a determination information output step of outputting determination information, which is information of the determination result of the fatigue level in the fatigue level determination step, to the outside.

Images