JP6594953B2 - Gaze movement detection method, program, information processing apparatus, and eyewear - Google Patents

Description

  The present invention relates to a line-of-sight movement detection method, a program, an information processing apparatus, and eyewear.

  There is known eyewear in which a plurality of electrodes are provided in a frame portion and an electrooculogram signal is acquired from each electrode (for example, see Patent Document 1). An apparatus for detecting a user's line-of-sight direction from an electrooculogram signal is known (see, for example, Patent Document 2).

US Patent Application Publication No. 2004/0070729 JP 2011-126992 A

  However, when detecting eye movement based on an electrooculogram signal, the movement of returning the eye line after moving the eye line is also detected as eye movement, so that the eye movement cannot be detected appropriately.

  Therefore, an object of the present invention is to be able to appropriately detect line-of-sight movement.

  In the line-of-sight movement method according to one aspect of the present invention, an acquisition step in which a computer acquires an electro-oculogram related signal related to an electro-oculogram detected by each electrode in contact with the periphery of the eye of the subject; A first detection step for detecting line-of-sight movement based on whether or not a threshold value has been exceeded; a determination step for determining whether or not an elapsed time from a first time point at which the line-of-sight movement is detected is within a predetermined time; Deciding which one of the first threshold and the second threshold having an absolute value larger than the first threshold to be used as the threshold based on the determination result of the elapsed time, and the electrooculogram after the first time point A second detection step of detecting line-of-sight movement based on whether or not the related signal exceeds the determined threshold value.

  According to the present invention, it is possible to appropriately detect line-of-sight movement.

It is a perspective view which shows an example from the front of the glasses in an Example. It is a perspective view which shows an example from the back of the spectacles in an Example. It is a block diagram which shows an example of the processing apparatus in an Example. It is a figure which shows roughly the contact position of the electrode with respect to a user. It is a figure which shows an example of a structure of the amplification part in an Example. It is a figure for demonstrating the reason for providing a buffer amplifier. It is a figure which shows the other example of a structure of the amplification part in an Example. It is a block diagram which shows an example of a structure of the external device in an Example. It is a figure which shows an example of a 1st electrooculogram related signal. It is a figure which shows an example of a 2nd electrooculogram related signal. It is a figure which shows an example of a 3rd ocular potential related signal. It is a figure which shows an example of a 4th electrooculogram related signal. It is a flowchart which shows an example of the process regarding the detection process of a gaze movement in an Example.

  Embodiments of the present invention will be described below with reference to the drawings. However, the embodiment described below is merely an example, and there is no intention to exclude various modifications and technical applications that are not explicitly described below. That is, the present invention can be implemented with various modifications without departing from the spirit of the present invention. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. The drawings are schematic and do not necessarily match actual dimensions and ratios. In some cases, the dimensional relationships and ratios may be different between the drawings.

[Example]
FIG. 1 is a perspective view illustrating an example from the front of the glasses 100 according to the embodiment. FIG. 2 is a perspective view illustrating an example from the rear of the glasses 100 in the embodiment. The glasses 100 include a lens 110 and a frame 120. Glasses 100 and frame 120 are examples of eyewear.

  The frame 120 supports the pair of lenses 110. The frame 120 includes a rim 122, an eyebrow portion (for example, a bridge) 124, an armor 126, a hinge 128, a temple 130, a modern 132, a pair of nose pads 140, a first electrode 152, and a second electrode. 154, a third electrode 156, an electric wire (not shown), a processing device 200, and an amplification unit 250. Depending on the type of glasses 100, there may be no bridge portion of the frame by using a single lens. In this case, the portion between the eyebrows of the single lens is defined as the portion between the eyebrows.

  The pair of nose pads 140 includes a right nose pad 142 and a left nose pad 144. The rim 122, the armor 126, the hinge 128, the temple 130, and the modern 132 are provided in a pair on the left and right.

  The rim 122 holds the lens 110. The armor 126 is provided outside the rim 122 and holds the temple 130 rotatably with a hinge 128. The temple 130 presses the upper part of the user's ear to pinch this part. The modern 132 is provided at the tip of the temple 130. The modern 132 contacts the upper part of the user's ear. The modern 132 is not necessarily provided in the glasses 100.

  The first electrode 152 and the second electrode 154 are provided on the respective surfaces of the pair of nose pads 140 and detect the electrooculogram. For example, the first electrode 152 is provided on the right nose pad 142, and the second electrode 154 is provided on the left nose pad 144.

  The first electrode 152 detects the electrooculogram of the user's right eye. The second electrode 154 detects the electrooculogram of the user's left eye. As described above, the electrode for detecting the electrooculogram is provided on the surface of the nose pad that inevitably contacts the skin of the user. Thereby, the burden given to a user's skin can be reduced compared with making a pair of electrodes contact the circumference | surroundings of a user's eye.

  The third electrode 156 is provided on the surface of the eyebrow portion 124 and detects an electrooculogram. A ground electrode (not shown) may be provided on the surface of the modern 132. When the glasses 100 do not have the modern 132, the ground electrode is provided at the tip of the temple 130. In the embodiment, the potential detected by the first electrode 152, the second electrode 154, and the third electrode 156 may be based on the potential detected by the ground electrode.

  The processing apparatus 200 may be provided in the temple 130, for example. Thus, the design when the glasses 100 are viewed from the front is not impaired. The installation position of the processing apparatus 200 is not necessarily the temple 130, but may be positioned in consideration of the balance when the glasses 100 are worn. The processing device 200 is connected to the amplifying unit 250 via an electric wire. Note that the processing device 200 and the amplifying unit 250 may be connected via wireless.

  The amplifying unit 250 is provided in the vicinity of the first electrode 152, the second electrode 154, and the third electrode 156, and is connected to each electrode to be amplified via an electric wire. The amplifying unit 250 acquires an electrooculogram signal indicating the electrooculogram detected by each electrode. For example, the amplification unit 250 amplifies an electrooculogram signal indicating an electrooculogram detected by the first electrode 152, the second electrode 154, and the third electrode 156.

  In addition, if the amplification unit 250 includes a processing unit that calculates an electrooculogram signal, the amplification unit 250 may perform addition / subtraction processing on each electrooculogram signal before amplification or after amplification. For example, the amplifying unit 250 may obtain a reference ocular potential signal indicating the potential of the first electrode 152 with respect to the third electrode 156. In addition, the amplifying unit 250 may obtain a reference ocular potential signal indicating the potential of the second electrode 154 with respect to the third electrode 156. The signal amplified or processed by the amplification unit 250 is output to the processing device 200.

  The external device 300 is an information processing device having a communication function. For example, the external device 300 is a mobile communication terminal such as a mobile phone and a smartphone possessed by the user, a personal computer, or the like. The external device 300 executes processing based on the electrooculogram signal received from the transmission unit 220 illustrated in FIG. For example, the external device 300 detects blinks and eye movements from the received electrooculogram signal. As a response when detecting blinks, the external device 300 issues a warning for preventing a doze when detecting that the number of blinks of the user has increased. Details of the external device 300 will be described later. Further, the external device 300 may be able to operate an application based on the detected line-of-sight movement.

<Configuration of processing device>
FIG. 3 is a block diagram illustrating an example of the processing apparatus 200 in the embodiment. As illustrated in FIG. 3, the processing device 200 includes a processing unit 210, a transmission unit 220, and a power supply unit 230. The first electrode 152, the second electrode 154, and the third electrode 156 are connected to the processing unit 210 via, for example, the amplification unit 250. Each component of the processing apparatus 200 may be provided by being distributed in a pair of temples.

  The processing unit 210 acquires the electrooculogram signal amplified from the amplification unit 250 and processes it. For example, the processing unit 210 may process a reference electrooculogram signal indicating the potential of the first electrode 152 with respect to the third electrode 156. In addition, although the reference | standard electrooculogram signal attached | subjected "reference | standard" for convenience of explanation, it is contained in an electrooculogram signal as a concept. The processing unit 210 may process a reference electrooculogram signal indicating the potential of the second electrode 154 with respect to the third electrode 156.

  At this time, the processing unit 210 performs processing so as to obtain an electrooculogram signal indicating the vertical and / or horizontal movement of the eye based on the electrooculogram detected from each electrode in the right eye and the left eye. May be. For example, the processing unit 210 subtracts the potential of the third electrode 156 from the potential of the second electrode 154 to generate an ocular potential signal, or the potential of the second electrode 154 and the third electrode 156 from the potential of the first electrode 152. An electrooculogram signal may be generated by subtracting the average of the potential.

  In addition, when the acquired electrooculogram signal is not digitized, the processing unit 210 performs the digitization process or increases or decreases the electrooculogram signal when the electrooculogram signal amplified from each electrode is acquired. Process. In addition, the processing unit 210 may transmit the electrooculogram signal acquired from the amplification unit 250 to the transmission unit 220 as it is.

  The transmission unit 220 transmits the electrooculogram signal processed by the processing unit 210 to the external device 300. For example, the transmission unit 220 transmits an electrooculogram signal to the external device 300 by wireless communication such as Bluetooth (registered trademark) and wireless LAN, or wired communication. The power supply unit 230 supplies power to the processing unit 210, the transmission unit 220, and the amplification unit 250.

  FIG. 4 is a diagram schematically showing the contact position of the electrode with respect to the user. The first contact position 452 represents the contact position of the first electrode 152. The second contact position 454 represents the contact position of the second electrode 154. The third contact position 456 represents the contact position of the third electrode 156. A horizontal center line 460 represents a horizontal center line connecting the center of the right eye 402 and the center of the left eye 404. The vertical center line 462 represents a center line orthogonal to the horizontal center line 460 at the center of the right eye 402 and the left eye 404.

  It is desirable that the first contact position 452 and the second contact position 454 are located below the horizontal center line 460. Further, it is desirable that the first contact position 452 and the second contact position 454 are arranged so that a line connecting the centers of the first contact position 452 and the second contact position 454 is parallel to the horizontal center line 460.

  Further, the first contact position 452 and the second contact position 454 are desirably arranged so that the distance from the first contact position 452 to the right eye 402 and the distance between the second contact position 454 and the left eye 404 are equal. . Further, it is desirable that the first contact position 452 and the second contact position 454 are separated from each other by a certain distance or more.

  The third contact position 456 is preferably located on the vertical center line 462. The third contact position 456 is preferably located above the horizontal center line 460 and away from the first contact position 452 and the second contact position 454. In addition, for example, the distance between the third contact position 456 and the right eye 402 is separated from the distance between the right eye 402 and the first contact position 452, and the distance from the left eye 404 is the second contact with the left eye 404. The distance from the position 454 may be greater than the distance.

  The eyeball is positively charged on the corneal side and negatively charged on the retinal side. Therefore, when the line of sight moves upward, the potential of the first electrode 152 with respect to the third electrode 156 and the potential of the second electrode 154 with respect to the third electrode 156 become negative. When the line of sight moves downward, the potential of the first electrode 152 with respect to the third electrode 156 and the potential of the second electrode 154 with respect to the third electrode 156 become positive.

  When the line of sight moves to the right, the potential of the first electrode 152 with respect to the third electrode 156 becomes negative, and the potential of the second electrode 154 with respect to the third electrode 156 becomes positive. When the line of sight moves to the left, the potential of the first electrode 152 with respect to the third electrode 156 becomes positive, and the potential of the second electrode 154 with respect to the third electrode 156 becomes negative.

  By detecting the potential of the first electrode 152 with respect to the third electrode 156 and the potential of the second electrode 154 with reference to the third electrode 156, the influence of noise can be suitably reduced. In order to make the third contact position 456 as far as possible from the first contact position 452 and the second contact position 454, the inter-brow portion 124 may be disposed at or near the upper end of the rim 122. The third electrode 156 may be provided above the center of the eyebrow portion 124. In this case, it is desirable to adopt the eyebrow portion 124 having a wide vertical width as the arrangement position of the third electrode 156.

  Instead of detecting the potential of the first electrode 152 based on the third electrode 156, the processing unit 210 detects the third electrode based on the reference electrode from the potential of the first electrode 152 based on the reference electrode. The potential of 156 may be reduced. Similarly, instead of detecting the potential of the second electrode 154 with respect to the third electrode 156, the processing unit 210 detects the potential of the second electrode 154 with respect to the reference electrode as a reference. The potential of the three electrodes 156 may be reduced.

  A ground electrode may be used as the reference electrode. Further, a reference electrode may be separately provided in the glasses 100 at a position away from the first electrode 152, the second electrode 154, and the third electrode 156. For example, the reference electrode may be provided on the modern 132 on the right side. Further, the reference electrode may be provided at a portion of the right temple 130 that is in contact with the user's skin.

  The process of subtracting the potential of the third electrode 156 from the potential of the first electrode 152 relative to the reference electrode and the process of subtracting the potential of the third electrode 156 from the potential of the second electrode 154 relative to the reference electrode are as follows: The processing unit 210 may execute, or the amplification unit 250 or the external device 300 may execute. In this case, the signal indicating the potential to be processed is amplified by the amplification unit 250.

<Configuration of amplification unit>
Next, the configuration of the amplification unit 250 will be described. FIG. 5 is a diagram illustrating an example of a configuration of the amplification unit 250 in the embodiment. As illustrated in FIG. 5, the amplification unit 250 includes a first amplifier 260 and a second amplifier 270. The first amplifier 260 is an amplifier that is positioned in front of the second amplifier 270 and functions as a buffer amplifier. Hereinafter, the first amplifier 260 is also referred to as a buffer amplifier 260. The second amplifier 270 is an amplifier that functions as a main amplifier. Hereinafter, the second amplifier 270 is also referred to as a main amplifier 270. The signal amplified by the main amplifier 270 is output to the processing device 200 by wire or wireless.

  As for the installation position of the amplification part 250, it is desirable that it is the part 124 between eyebrows. Note that the amplification unit 250 may be provided so as to be embedded in the eyebrow portion 124. As described above, it is desirable that the electrodes be separated as much as possible. However, since the installation positions of the electrodes depend on the shape of the frame 120, there is a limit even if they are separated.

  For this reason, the potential difference between the electrodes may not be sufficiently large, and if noise is mixed in an electrooculogram signal indicating a small potential detected at each electrode, a sufficiently accurate potential can be detected. It becomes difficult.

  Therefore, in the embodiment, the amplification unit 250 is provided in the vicinity of the first electrode 152, the second electrode 154, and the third electrode 156 for the purpose of amplifying the detected electrooculogram signal before noise is mixed therein. It is done. For example, the amplifying unit 250 is preferably provided in the portion between the eyebrows 124 where the space is present in the frame 120 near each electrode. Thereby, while the electrooculogram signal detected by each electrode passes through the electric wire, it is possible to reduce the risk of noise being mixed and reducing the accuracy of the electrooculogram signal.

  Next, the reason why the buffer amplifier 260 is provided at a position preceding the main amplifier 270 will be described with reference to FIG. FIG. 6 is a diagram for explaining the reason why the buffer amplifier 260 is provided. The example shown in FIG. 6 uses the third electrode 156, but the same applies to the first electrode 152 and the second electrode 154.

Since the third electrode 156 touches human skin when wearing the glasses 100, it may be considered that a resistance _R0 exists between the third electrode 156 and the ground. At this time, the resistance R ₀ is, for example, several hundred kΩ. Further, the main amplifier 270, there is an internal resistance _{R 1.} In this case, the use of conventional amplifier as a main amplifier 270, the internal resistance _{R 1} is the number 10kΩ~ number 100 k.OMEGA.

Here, although ideally is that does not flow a current to the main amplifier 270, the internal resistance R ₁ is smaller than the resistance R _0, the current flows to the main amplifier 270 side. Then, the voltage Vi of the electrode and the voltage Vx of the main amplifier 270 are divided and observed. Therefore, a buffer amplifier 260 is provided at a position preceding the main amplifier 270 so that no current flows into the main amplifier 270 side.

  FIG. 7 is a diagram illustrating another example of the configuration of the amplifying unit in the embodiment. The amplifying unit shown in FIG. 7 is denoted by reference numeral 250A. The amplification unit 250A includes a buffer amplifier 260, a main amplifier 270, an A / D conversion unit 280, and a wireless communication unit 290. Since the buffer amplifier 260 and the main amplifier 270 have the same functions as those shown in FIG. 5, the A / D conversion unit 280 and the wireless communication unit 290 will be mainly described below.

  The A / D converter 280 converts the signal amplified by the main amplifier 270 from analog to digital. The A / D conversion unit 280 outputs the digitally converted signal to the wireless communication unit 290.

  The wireless communication unit 290 transmits the digital signal converted by the A / D conversion unit 280 to the processing device 200 using wireless communication. Therefore, the wireless communication unit 290 functions as a transmission unit. The wireless communication unit 290 uses, for example, wireless communication such as Bluetooth (registered trademark) and wireless LAN. The wireless communication unit 290 may directly transmit a digital signal to the external device 300.

  In the embodiment, an example in which one buffer amplifier 260 and one main amplifier 270 are provided has been described. In this case, the order of the electrooculogram signals from the electrodes may be determined and amplified. Further, a buffer amplifier 260 and a main amplifier 270 may be provided for each electrode.

<Configuration of external device>
Next, the configuration of the external device 300 will be described. FIG. 8 is a block diagram illustrating an example of the configuration of the external device 300 in the embodiment. As illustrated in FIG. 8, the external device 300 includes a communication unit 310, a storage unit 320, and a control unit 330.

  The communication unit 310 receives an electrooculogram signal by wireless communication such as Bluetooth (registered trademark) and wireless LAN, or wired communication. The communication unit 310 outputs the electrooculogram signal received from the transmission unit 220 of the processing device 200 to the control unit 330.

  The storage unit 320 is, for example, a RAM (Random Access Memory), and stores a maximum value and / or a minimum value of the electrooculogram signals of the right eye and the left eye. Further, the maximum value or the minimum value may be a maximum value or a minimum value for each predetermined period.

  The storage unit 320 may store a maximum value and / or a minimum value of the differential signal of the electrooculogram signal. In addition, the storage unit 320 stores a maximum value and / or a minimum value for a signal of the difference between adjacent extreme values of the electrooculogram signal, and a maximum value and / or a minimum value for a signal of the difference between adjacent extreme values of the difference signal. May be.

  For example, the storage unit 320 includes a maximum value FIFO buffer and a minimum value FIFO buffer. When the storage capacity of the FIFO buffer is full due to the maximum or minimum value data, the oldest data is erased and the latest data is stored, whereby the data stored in the storage area is updated. . The storage unit 320 may store a detection result of the detection unit 360 described later.

  In addition, the storage unit 320 stores a program that causes a computer to execute a line-of-sight movement detection process described later. This program may be installed in the external device 300 via the Internet or a recording medium such as an SD card, or may be preinstalled. In addition, the storage unit that stores the program may be different from the storage unit 320.

  The control unit 330 is a CPU (Central Processing Unit), for example, and controls each unit and performs various arithmetic processes. In the example illustrated in FIG. 8, the control unit 330 includes an acquisition unit 340, a determination unit 350, and a detection unit 360.

  The acquisition unit 340 acquires an electrooculogram related signal based on an electrooculogram detected by each electrode that contacts the periphery of the eye of the subject. The electrooculogram related signal includes, for example, an electrooculogram signal acquired from the communication unit 310, a differential signal of the electrooculogram signal, and an extreme value difference signal representing a difference between the electropotential signal or the adjacent extreme values of the differential signal. The acquired electrooculogram related signal is an electrooculogram related signal of the right eye and / or the left eye. The electrooculogram related signal may be processed on the eyewear side and transmitted to the external device 300.

  The difference signal is a signal that represents a difference between a predetermined electrooculogram signal and an electrooculogram signal that is a predetermined time before the electrooculogram signal. The predetermined time is, for example, 5 msec. By taking the signal difference, noise tolerance can be increased. Note that taking the difference between these signals is synonymous with performing differentiation.

  An extreme value difference signal is a signal representing the difference between adjacent extreme values (peak values) of an electrooculogram signal or a difference signal. By taking this extreme difference signal, it is possible to appropriately determine the movement of the line of sight without being affected by the signal level immediately before the eye moves.

  The acquisition unit 340 includes a signal processing unit 342 in order to generate a determination target signal of the determination unit 350 described later. When the acquired signal is an electrooculogram signal and the determination target signal is a difference signal or an extreme value difference signal, the signal processing unit 342 calculates a difference or obtains an extreme value based on the electrooculogram signal. In other words, a process for obtaining a determination target signal is performed. If the acquired signal is an electrooculogram signal and the determination target signal is an electrooculogram signal, the signal processing unit 342 performs no particular processing. The acquisition unit 340 outputs the electrooculogram related signal acquired as the determination target signal to the determination unit 350.

  The control unit 330 stores the local maximum value and the local minimum value of the electrooculogram related signal indicating the movement of the right eye and / or the left eye in the storage unit 320. In addition, the control unit 330 may store the maximum value and / or the minimum value of the electrooculogram related signal indicating the movement of the right eye and the left eye for each predetermined period in the storage unit 320.

  The predetermined period is, for example, 200 msec, but is not limited thereto. Further, the predetermined period may be temporally varied by allowing overlap by using a time window.

  The determination unit 350 performs a threshold determination process using the electrooculogram related signal and the threshold, performs a time determination process as to whether or not the elapsed time is equal to or longer than a predetermined time, and determines a threshold used for the determination. Therefore, the determination unit 350 includes a threshold determination unit 352, a time determination unit 354, and a determination unit 356.

  The threshold determination unit 352 compares the electrooculogram related signal with the threshold, and determines whether the electrooculogram related signal exceeds the threshold. Exceeding the threshold includes that the absolute value of the electrooculogram related signal exceeds the threshold, the positive electrooculogram related signal is greater than or equal to the positive threshold, and the negative electrooculogram related signal is less than or equal to the negative threshold. Including becoming. In addition, that the electrooculogram related signal exceeds the threshold includes that the electrooculogram related signal itself exceeds the threshold and that the extreme value of the electrooculogram related signal exceeds the threshold.

  The threshold value includes a first threshold value and a second threshold value. The first threshold is a threshold for use in eye movement determination, and the second threshold is a threshold for use in reverse eye movement determination after the eye movement once, and has an absolute value larger than the first threshold.

  The determination unit 356 determines whether the threshold determination unit 352 uses the first threshold or the second threshold. The threshold determination unit 352 uses the first threshold for determination processing by default.

  The time determination unit 354 determines whether or not the elapsed time from the first time point when the eye movement is detected is within a predetermined time when the eye movement is detected when the electrooculogram related signal exceeds the threshold. . The predetermined time may be a general time required for the line of sight to return to the front view, for example, and is 1.0 sec, for example. For the predetermined time, an appropriate value may be set by a preliminary experiment or the like. The time determination unit 354 starts counting elapsed time each time eye movement is detected, and performs a determination process.

  The determination unit 356 determines which of the first threshold value and the second threshold value is used as the threshold value based on the elapsed time determination result acquired from the time determination unit 354. The determination unit 356 outputs the determined threshold value to the threshold determination unit 352.

  For example, the determination unit 356 determines to use the second threshold if the elapsed time is within a predetermined time, and determines to use the first threshold if the predetermined time has elapsed. The elapsed time may be reset every time a movement of the line of sight is detected, or the counting of time may be stopped after the elapsed time has passed a predetermined time. This is because it is important whether or not the elapsed time is within a predetermined time, and therefore it is not necessary to count the elapsed time when the elapsed time exceeds the predetermined time.

  The detection unit 360 detects line-of-sight movement based on the determination result by the threshold determination unit 352. For example, when the threshold determination unit 352 determines that the electrooculogram related signal exceeds the first threshold or the second threshold, the detection unit 360 detects the movement of the line of sight. For example, when the electrooculogram related signal exceeds the second threshold value, the eye movement in the direction opposite to the direction of the eye movement immediately before is detected. This is because the second threshold value is basically a threshold value used for detecting a gaze movement in the reverse direction from the last gaze movement. Note that the second threshold value may be used for a line-of-sight movement in the same direction as the previous line-of-sight movement.

  With the above detection processing, it is possible to prevent a movement of the line of sight returning to the original (for example, movement of returning to the front view) from being detected as the line of sight movement once the line of sight has moved. That is, after the line of sight has moved, it usually moves so that the line of sight returns, but in the above-described line of sight movement detection algorithm, the time required for the line of sight to return is set as a predetermined time. The predetermined time is a time in which the line-of-sight movement indicating the return movement and the line-of-sight movement in another direction (for example, the reverse direction) can be mixed. Therefore, in the gaze detection algorithm, in order to determine the difference between the gaze movement indicating the movement returning to the original front view and the gaze movement in the reverse direction, in particular, a threshold value (second threshold value) larger than the normal first threshold value is determined. ) To be able to separate these line-of-sight movements. Note that after the predetermined time has elapsed, the first threshold value may be used in order to detect normal line-of-sight movement.

<Gaze movement detection>
Next, detection of eye movement will be described. For example, the detection unit 360 divides the electrooculogram from the acquired electrooculogram signal into a right electrocardiogram and a left electrooculogram. Detect that you are facing up.

  Further, the detection unit 360 has a line of sight when a positive potential is shown in the right electrogram and the left electrogram, a negative potential is shown in the right electrogram, and a positive potential is shown in the left electrogram. Is detected, it is detected that the line of sight is directed to the left when the line of sight is right, the right electrogram shows a positive potential and the left electrogram shows a negative potential.

  In addition, the detection unit 360 uses the electrooculogram signal generated by subtracting the potential of the third electrode 156 from the potential of the second electrode 154, and when the negative potential is indicated, the line of sight is the right, positive potential. May be detected when the line of sight is directed to the left.

  In addition, when the detection unit 360 indicates a positive potential using an electrooculogram signal generated by subtracting the average of the potentials of the second electrode 154 and the third electrode 156 from the potential of the first electrode 152. If a negative potential is indicated, it may be detected that the camera has turned downward. At this time, the detection unit 360 detects line-of-sight movement using the result of the determination unit 350 determining a positive potential or a negative potential using the first threshold value or the second threshold value described above.

<Specific example>
Next, a peak (extreme value) used for eye movement detection will be described using a specific example of an electrooculogram related signal. In the graphs shown in FIGS. 9 to 12, the vertical axis represents the electrooculogram related signal intensity (hereinafter also referred to as electrooculogram intensity), and the horizontal axis represents time.

  In the graphs shown in FIGS. 9 to 12, an electrooculogram signal generated by subtracting the potential of the third electrode 156 from the potential of the second electrode 154 is used as an example of the electrooculogram related signal. Therefore, in the examples illustrated in FIGS. 9 to 12, if the extreme value of the signal S <b> 1 when the signal S <b> 1 is equal to or greater than the threshold value is positive, the detection unit 360 detects the line-of-sight movement to the left and If so, the movement of the line of sight to the right is detected. The scale of the electrooculogram intensity in the case of an electrooculogram signal is, for example, measured value × 1.5 (V) ÷ 2048 ÷ 1000. As described above, in the examples illustrated in FIGS. 9 to 12, the extreme value of the electrooculogram related signal will be described as the threshold determination target. By using the extreme value, it is sufficient that the threshold determination is performed when the extreme value is detected, and the threshold determination may not always be performed.

  FIG. 9 is a diagram illustrating an example of the first electrooculogram related signal. The signal S1 shown in FIG. 9 is a first electrooculogram related signal indicating the left and right movements of both eyes, the threshold Th1 is the first threshold, and the threshold Th2 is the second threshold. The threshold value Th1 is 100, for example, and the threshold value Th2 is 300, for example.

  In the example illustrated in FIG. 9, it is assumed that the threshold determination unit 352 determines that the extreme value of the signal S1 at the time t1 exceeds the first threshold Th1. The time determination unit 354 counts the elapsed time after the time t1 of the extreme value. In the example shown in FIG. 9, when the time from the first time point t1 to the time point t2 at which the signal S1 reaches the extreme value exceeding the second threshold Th2 is T1, this time T1 is assumed to be within the predetermined time TP. .

  In this case, the time determination unit 354 determines that the counted elapsed time is within the predetermined time TP, and the determination unit 356 determines the second threshold Th2 as a threshold used for line-of-sight movement, and the threshold determination unit 352 Notice.

  When the second threshold is notified from the determination unit 356, the threshold determination unit 352 determines whether or not the extreme value of the signal S1 exceeds the second threshold Th2, and outputs the determination result to the detection unit 360.

  At this time, when the detection unit 360 recognizes from the determination result of the threshold determination unit 352 that the absolute value of the signal S1 exceeds the second threshold Th2, the eye movement at the time t1 is the eye movement at the time t2. Detect line-of-sight movement in the opposite direction.

  FIG. 10 is a diagram illustrating an example of the second electrooculogram related signal. A signal S2 shown in FIG. 10 is a second electro-oculogram related signal indicating the left and right movements of both eyes, and the threshold is the same as the threshold shown in FIG.

  In the example illustrated in FIG. 10, it is assumed that the threshold determination unit 352 determines that the extreme value of the signal S2 at the time t1 exceeds the first threshold Th1. The time determination unit 354 counts the elapsed time after the time t1 of the extreme value. If the time from time t1 to time t3 is T2, this time T2 is assumed to be within a predetermined time TP.

  In this case, the time determination unit 354 determines that the counted elapsed time is within the predetermined time TP, and the determination unit 356 determines the second threshold Th2 as a threshold used for line-of-sight movement, and the threshold determination unit 352 Notice.

  When the second threshold is notified from the determination unit 356, the threshold determination unit 352 determines whether or not the extreme value of the signal S2 exceeds the second threshold Th2, and outputs the determination result to the detection unit 360. In the example illustrated in FIG. 10, the extreme value of the signal S2 at the time point t3 exceeds the first threshold value Th1, but does not exceed the second threshold value Th2. Therefore, the detection unit 360 sets the extreme value at the time point t3 as the line-of-sight movement. Do not detect.

  FIG. 11 is a diagram illustrating an example of a third ocular potential related signal. A signal S3 shown in FIG. 11 is a third electrooculogram related signal indicating the left and right movements of both eyes, and the threshold is the same as the threshold shown in FIG.

  In the example illustrated in FIG. 11, it is assumed that the threshold determination unit 352 determines that the extreme value of the signal S3 at time t1 exceeds the first threshold Th1. The time determination unit 354 counts the elapsed time after the time t1 of the extreme value. If the time from time t1 to time t4 is T3, it is assumed that this time T3 has passed a predetermined time TP.

  In this case, the time determination unit 354 determines that the counted elapsed time has passed the predetermined time TP, and the determination unit 356 determines the first threshold Th1 as a threshold used for eye movement, and notifies the threshold determination unit 352. To do.

  When the determination unit 356 notifies the first threshold value, the threshold value determination unit 352 determines whether or not the extreme value of the signal S3 exceeds the first threshold value Th1, and outputs the determination result to the detection unit 360. In the example illustrated in FIG. 11, the extreme value of the signal S3 at the time point t4 exceeds the first threshold value, and thus the detection unit 360 detects the extreme value at the time point t4 as line-of-sight movement.

  FIG. 12 is a diagram illustrating an example of a fourth ocular potential related signal. A signal S4 shown in FIG. 12 is a fourth ocular potential related signal indicating the left and right movements of both eyes, and the threshold is the same as the threshold shown in FIG.

  In the example illustrated in FIG. 12, it is assumed that the threshold determination unit 352 determines that the extreme value of the signal S4 at the time t1 exceeds the first threshold Th1. The time determination unit 354 counts the elapsed time after the time t1 of the extreme value. When the time determination unit 354 determines that the counted elapsed time has passed the predetermined time TP, the determination unit 356 determines the first threshold Th1 as a threshold used for eye movement, and notifies the threshold determination unit 352.

  When the determination unit 356 is notified of the first threshold, the threshold determination unit 352 determines whether or not the extreme value of the signal S4 exceeds the first threshold Th1, and outputs the determination result to the detection unit 360. In the example illustrated in FIG. 12, the extreme value of the signal S4 after the time point t1 does not exceed the first threshold value, and thus the detection unit 360 does not detect line-of-sight movement.

  As described above, once the line-of-sight movement is detected, the second threshold value is used for the determination of the line-of-sight movement if it is within the predetermined time, and when the predetermined time has passed, the original first threshold value is used for the determination of the line-of-sight movement. . Thereby, it is possible to prevent the movement of the eye from returning to the original state after moving the line of sight to be detected by moving the line of sight. On the other hand, by setting the threshold value to be larger than the normal threshold value within a predetermined time, it is possible to appropriately detect the eye movement in the reverse direction.

  In the examples shown in FIGS. 9 to 12, the extreme time point is used as the start point of the elapsed time. However, a time point exceeding the threshold may be used as the start point of the elapsed time. An appropriate value may be set for the predetermined time by a preliminary experiment or the like. Further, the first threshold value and the second threshold value may be appropriately set according to the electrooculogram related signal to be determined.

<Operation>
Next, the operation of the external device 300 in the embodiment will be described. FIG. 13 is a flowchart illustrating an example of a process related to the gaze movement detection process in the embodiment. The flowchart shown in FIG. 13 shows a state in which the user wears the glasses 100 and the first electrode 152, the second electrode 154, the third electrode 156, and the ground electrode are in contact with the user's skin. Is started when the operation mode (normal mode), which is a mode for detecting the eye movement, is set.

  In step S <b> 102 illustrated in FIG. 13, the acquisition unit 340 starts acquiring an electrooculogram related signal from the glasses 100.

  In step S104, the threshold determination unit 352 determines whether or not the electrooculogram related signal exceeds the first threshold. If the electrooculogram related signal exceeds the first threshold (step S104-YES), the process proceeds to step S106, and if the electrooculogram related signal does not exceed the first threshold (step S104-NO), the process proceeds to step S104. Return.

  In step S106, the detection unit 360 detects eye movement when the electrooculogram related signal exceeds the first threshold. Whether the line of sight is directed up, down, left, or right can be determined by the type of electrooculogram related signal and whether the signal is positive or negative.

  In step S108, the time determination unit 354 determines whether or not the elapsed time from the first time point when the movement of the line of sight is detected is within a predetermined time. If the elapsed time is within the predetermined time (step S108-YES), the process proceeds to step S110. If the elapsed time has passed the predetermined time (step S108-NO), the process returns to step S104.

  In step S110, the threshold determination unit 352 determines whether or not the electrooculogram related signal exceeds the second threshold. If the electrooculogram related signal exceeds the second threshold (step S110-YES), the process proceeds to step S112. If the electrooculogram related signal does not exceed the second threshold (step S110-NO), the process proceeds to step S108. Return.

  In step S112, the detection unit 360 detects a gaze movement in a direction opposite to the previous gaze movement when the electrooculogram related signal exceeds the second threshold. The process returns to step S104. The process illustrated in FIG. 13 ends when the operation mode is turned off by the user.

  As described above, according to the embodiment, it is possible to appropriately detect the movement of the line of sight. Further, according to the embodiment, the process of not detecting the movement of the line of sight after the movement of the line of sight as the movement of the line of sight and the process of detecting the movement of the normal line of sight are realized by a simple mechanism by using threshold switching. be able to. Further, according to the embodiment, with regard to the visual line movement in the reverse direction after the visual line movement is once performed, it is possible to detect the visual line movement in the reverse direction after the visual line movement by setting a more appropriate threshold value.

  In the present embodiment, the case where the eyewear is glasses has been described. However, eyewear is not limited to this. The eyewear may be any device related to the eye, and may be a face wearing device or a head wearing device such as glasses, sunglasses, goggles and a head mounted display, and a frame thereof.

  In this embodiment, the example in which the glasses 100 include the third electrode 156 has been described. However, the glasses 100 are not limited to this. The glasses 100 may not include the third electrode 156. In this case, an electrooculogram indicated by the potential of the first electrode 152 relative to the reference electrode and an electrooculogram indicated by the potential of the second electrode 154 relative to the reference electrode may be transmitted to the external device 300. Here, a ground electrode may be provided at the position of the third electrode 156 to serve as a reference electrode. Further, a ground electrode provided in the left modern may be used as a reference electrode, or an electrode provided separately from the first electrode 152 and the second electrode 154 may be used as a reference electrode.

  In the present embodiment, the example in which the glasses 100 include the nose pad 140 integrated with the rim 122 has been described. However, the glasses 100 are not limited to this. The glasses 100 may include a klings provided on the rim 122 and a nose pad 140 attached to the krings. In this case, the electrode provided on the surface of the nose pad 140 is electrically connected to the electric wire embedded in the frame via the krings.

  In the present embodiment, the first electrode 152 and the second electrode 154 have been described as examples provided below the center of the nose pad 140. However, it is not limited to this. The nose pad 140 may include an extending portion that extends downward, and the first electrode 152 and the second electrode 154 may be provided in the extending portion. This allows the first electrode 152 and the second electrode 154 to be in contact below the eye position even for a user whose nose pad is located directly beside the eye due to individual differences in eye and nose positions. Can do.

  In the present embodiment, the example in which the third electrode 156 is provided on the surface of the eyebrow portion 124 has been described. However, it is not limited to this. The eyebrow portion 124 may include an extending portion that extends upward, and the third electrode 156 may be provided in the extending portion. Furthermore, a movable part that moves the extending part up and down between the extending part and the eyebrow part 124 may be provided so that the position of the third electrode 156 can be adjusted up and down. As a result, even if the user has a contact position of the third electrode 156 near the eye due to individual differences in the eye position, the contact position of the third electrode 156 can be separated from the eye by adjustment. . Further, in the present embodiment, the position of each electrode is not limited to the above-described position, and may be disposed at a position where an electrooculogram signal indicating the vertical and horizontal movements of the eye can be acquired.

  In the present embodiment, as an example of the external device 300, a mobile communication terminal such as a mobile phone and a smartphone that is separate from the processing device 200 has been described. However, it is not limited to this. The external device 300 may be a unit integrated with the processing device 200. In this case, the external device 300 is provided integrally with the eyewear. In this case, the detection result of the detection unit of the processing device 200 is transmitted to the external device 300 every time it is detected in real time, and the detection results are totaled every predetermined time (for example, one minute). May be transmitted to the external device 300.

  By transmitting the detection result in real time, detailed analysis can be performed at an early stage.

  By transmitting the counting result every predetermined time, the data is transmitted together, so that power consumption can be reduced.

  In the present embodiment, a shield cable may be used as the electric wire to prevent noise from being mixed.

  Further, in this embodiment, the configuration using three electrodes is illustrated in FIG. 1, but a configuration using four or more electrodes may be used. In this case, the glasses have an upper electrode, a lower electrode, a left electrode, and a right electrode. For example, the upper electrode and the lower electrode are provided on the rim 122 shown in FIG. 1, the left electrode is provided on the left temple 130, and the right electrode is provided on the right temple 130. There is no. Note that these electrodes are in contact with a part of the face. Further, the glasses may have two electrodes at positions of the first electrode 152, the second electrode 154, and the third electrode 156.

  In the example of four electrodes, the vertical direction of the eye can be detected by the voltage difference between the upper electrode and the lower electrode, and the horizontal direction of the eye can be detected by the voltage difference between the left electrode and the right electrode. .

  Further, in the present embodiment, the example in which the external apparatus 300 detects the movement of the line of sight has been described. However, the eyewear side detects the movement of the first threshold line of sight, and the result of the line of sight movement detection and the extreme value at that time And the time may be transmitted to an external device. In this case, the external device 300 performs an elapsed time determination process and a second threshold determination process, and cancels a line of sight movement that does not exceed the second threshold within a predetermined time from the line of sight movement detection. Also good.

  As mentioned above, although this invention was demonstrated using the Example, the technical scope of this invention is not limited to the range as described in the said Example. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above-described embodiments. It is apparent from the description of the scope of claims that embodiments with such changes or improvements can be included in the technical scope of the present invention.

100 glasses 120 frame 124 eyebrow portion 140 nose pad 152 first electrode 154 second electrode 156 third electrode 200 processing device 300 external device 320 storage unit 330 control unit 340 acquisition unit 342 signal processing unit 350 determination unit 352 threshold determination unit 354 time Determination unit 356 Determination unit 360 Detection unit

Claims (8)

Computer
An acquisition step of acquiring an electrooculogram related signal related to electrooculogram detected by each electrode in contact with the periphery of the eye of the subject;
A first detection step of detecting eye movement based on whether or not the electrooculogram related signal exceeds a first threshold;
A determination step of determining whether or not an elapsed time from the first time point when the line-of-sight movement is detected is within a predetermined time;
A determination step of determining which of the first threshold value and the second threshold value having an absolute value larger than the first threshold value is used as the threshold value based on the determination result of the elapsed time;
A second detection step of detecting eye movement based on whether or not the electrooculogram related signal after the first time point exceeds the determined threshold;
Eye movement detection method for executing The determination step includes
The eye movement detection method according to claim 1, wherein the elapsed time determination process is performed each time the eye movement is detected. The determining step includes
Determining to use the second threshold if the elapsed time is within the predetermined time, and determining to use the first threshold if the elapsed time has passed the predetermined time; The line-of-sight movement detection method according to claim 1. The second detection step includes
The gaze movement detection method according to any one of claims 1 to 3, comprising detecting gaze movement in a direction opposite to the gaze movement at the first time point based on a determination result of the second threshold value.   5. The eye movement detection method according to claim 1, wherein the electrooculogram related signal is a differential signal of an electrooculogram signal indicating the electrooculogram. On the computer,
An acquisition step of acquiring an electrooculogram related signal related to electrooculogram detected by each electrode in contact with the periphery of the eye of the subject;
A first detection step of detecting eye movement based on whether or not the electrooculogram related signal exceeds a first threshold;
A determination step of determining whether or not an elapsed time from the first time point when the line-of-sight movement is detected is within a predetermined time;
A determination step of determining which of the first threshold value and the second threshold value having an absolute value larger than the first threshold value is used as the threshold value based on the determination result of the elapsed time;
A second detection step of detecting eye movement based on whether or not the electrooculogram related signal after the first time point exceeds the determined threshold;
A program that executes An acquisition unit for acquiring an electrooculogram related signal related to an electrooculogram detected by each electrode contacting the periphery of the eye of the subject;
A time determination unit that determines whether or not an elapsed time from the first time point when the eye movement based on the electrooculogram related signal is detected is within a predetermined time;
A determination unit that determines which one of the first threshold value and the second threshold value whose absolute value is larger than the first threshold value is used as the threshold value based on the determination result of the elapsed time;
A threshold determination unit for determining whether or not the electrooculogram related signal after the first time point exceeds the determined threshold;
A detection unit that detects line-of-sight movement based on a determination result of the threshold determination unit;
An information processing apparatus comprising: Frame,
At least three electrodes provided on the frame and in contact with the subject's eye periphery;
An acquisition unit for acquiring an electrooculogram related signal related to an electrooculogram detected by each electrode contacting the periphery of the eye of the subject;
A time determination unit that determines whether or not an elapsed time from the first time point when the eye movement based on the electrooculogram related signal is detected is within a predetermined time;
A determination unit that determines which one of the first threshold value and the second threshold value whose absolute value is larger than the first threshold value is used as the threshold value based on the determination result of the elapsed time;
A threshold determination unit for determining whether or not the electrooculogram related signal after the first time point exceeds the determined threshold;
A detection unit that detects line-of-sight movement based on a determination result of the threshold determination unit;
Eyewear with.

Images