JP2015203957A - Information processor, information processing system, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a correspondence relation between an ocular potential and information without making a user to perform any special operation.SOLUTION: An information processor includes: an ocular potential acquisition part for acquiring the continuously detected ocular potential of a user; a lock control part for, when it is determined that a predetermined condition is satisfied by the ocular potential acquired by the ocular potential acquisition part, releasing the lock state of the information processor; and a correlation acquisition part for, when it is determined that the predetermined condition is satisfied by the ocular potential by the lock control part, acquiring a correlation between the ocular potential and the visual line direction of the user on the basis of the ocular potential and the visual line direction corresponding to the predetermined condition.

Description

  The present invention relates to an information processing apparatus, an information processing system, and a program.

An apparatus for measuring an electrooculogram or the like is known (for example, see Patent Documents 1-3).
Patent Document 1 Japanese Patent Application Laid-Open No. 2011-12087 Patent Document 2 Japanese Patent Application Laid-Open No. 9-034631 Japanese Patent Application Laid-Open No. 2013-55383

  If the correspondence between the electrooculogram and information such as the line-of-sight direction is inaccurate, information specified from the electrooculogram may be inaccurate. In order to specify information more accurately from the electrooculogram, it is necessary to cause the user to perform a special operation for acquiring the correspondence between the electrooculogram and the information.

  In the first aspect, the information processing apparatus determines that the electrooculogram acquisition unit that continuously detects the electrooculogram of the user and the electrooculogram acquired by the electrooculogram acquisition unit satisfy a predetermined condition. In the case where the lock control unit for releasing the lock state of the information processing device and the lock control unit determines that the electrooculogram satisfies a predetermined condition, the electrooculogram corresponds to the predetermined condition. A correlation acquisition unit that acquires a correlation between the electrooculogram and the user's line-of-sight direction based on the line-of-sight direction.

  A line-of-sight specifying unit that specifies the line-of-sight direction of the user after the lock state is released may be further provided based on the correlation acquired by the correlation acquisition unit and the electrooculogram detected after the lock state is released.

  The line-of-sight direction corresponding to the predetermined condition defines the positional relationship between the first position and the second position that the user should see in a predetermined order in order to release the locked state. Comparing the amount of change in electrooculogram between the first electrooculogram detected at the timing and the second electrooculogram detected at the second timing and the positional relationship between the first position and the second position; It may be determined whether or not the electrooculogram acquired by the electrooculogram acquisition unit satisfies a predetermined condition.

  The lock control unit may further include a display unit, and the lock control unit may display the marker at a position corresponding to each of the first position and the second position in the display area by the display unit.

  The electrooculogram acquisition unit may acquire the electrooculogram detected by the electrooculogram detection unit attached to the user.

  The electrooculogram acquisition unit may acquire the electrooculogram detected by the electrooculogram detection unit provided in the wearing tool worn by the user.

  The electrooculogram acquisition unit may acquire the electrooculogram detected by the electrooculogram detection unit provided in the eyewear worn by the user.

  In a second aspect, an information processing system includes the information processing apparatus and a wearing tool worn by a user, and the electrooculogram acquisition unit detects an eye detected by an electrooculogram detection unit provided in the wearing tool. Get the potential.

  In the third aspect, the program determines that the computer has an electrooculogram acquisition unit that acquires the electrooculogram of the user detected continuously, and the electrooculogram acquired by the electrooculogram acquisition unit satisfies a predetermined condition. In the case where the electrooculogram is determined to satisfy a predetermined condition by the lock control unit for releasing the lock state of the computer and the lock control unit, the electrooculogram and the reference line of sight corresponding to the predetermined condition It is made to function as a correlation acquisition part which acquires the correlation between electrooculogram and a user's gaze direction based on a direction.

  The summary of the invention does not enumerate all the features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

An example of the utilization form of electrooculogram information processing system 10 in one embodiment is shown roughly. The glasses 100 and the smartphone 40 are schematically shown. The functional block configuration of the smart phone 40 and the functional block configuration of the processing unit 180 are schematically shown. The positional relationship between the electrooculogram detection electrode of the glasses 100 and the user 20 is schematically shown. The change of the gaze direction of the user 20 is shown schematically. An example of the passcode input screen 42 displayed by the UI unit 370 is shown. An example of temporal development of the electrooculogram is schematically shown together with the control contents of the lock control unit 330.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 schematically illustrates an example of a usage pattern of an electrooculogram information processing system 10 according to an embodiment. The electrooculogram information processing system 10 includes glasses 100, a smartphone 40, and headphones 50.

  The user 20 is a user of each of the glasses 100, the smartphone 40, and the headphones 50. The user 20 is a wearer who wears the glasses 100. The smartphone 40 is an example of an information processing apparatus that processes information such as electrooculogram.

  The glasses 100 are worn on the face of the user 20. The glasses 100 have a function of communicating with the smartphone 40. The glasses 100 detect the electrooculogram of the user 20 via the electrode that contacts the user 20, and transmit the detected electrooculogram information to the smartphone 40. Further, the glasses 100 detect the acceleration of the glasses 100 and transmit the detected acceleration information to the smartphone 40. Further, the glasses 100 detect the angular velocity of the glasses 100 and transmit information on the detected angular velocity to the smartphone 40.

  The smartphone 40 analyzes the electrooculogram received from the glasses 100. The smartphone 40 analyzes the electrooculogram received from the glasses 100 together with at least one of acceleration and angular velocity received from the glasses 100. The smartphone 40 provides information to the user 20 based on analysis results such as electrooculogram, acceleration, and angular velocity.

  For example, the smartphone 40 identifies the state of the user 20 by analyzing the electrooculogram, acceleration, angular velocity, and the like. Specifically, the smartphone 40 analyzes the electrooculogram and identifies the user's 20 line-of-sight direction, the blink state, and the like. For example, the smartphone 40 determines the state of the user 20 based on the line-of-sight direction and the blink state. Specifically, the smartphone 40 determines whether the user 20 is tired or painful based on the line-of-sight direction and the blink state. The smartphone 40 issues a warning to the user 20 when determining that the user 20 is tired, painful, or the like. For example, the smartphone 40 generates a warning sound when it is determined that the user 20 is tired or painful.

  Moreover, the smart phone 40 specifies the instruction | indication from the user 20 based on analysis results, such as electrooculogram. For example, the smartphone 40 specifies an instruction from the user 20 to the smartphone 40 by a combination of changes in the line-of-sight direction of the user 20 and a combination of the line-of-sight direction and the blink state. The smartphone 40 controls the operation of the smartphone 40 based on an instruction from the user 20. For example, when the smartphone 40 is playing music in a predetermined playback order and determines that the user 20 has instructed to play the next song based on the analysis result such as electrooculogram, Start playing the next song in the current order. The music being played back by the smartphone 40 is output wirelessly or by wire to a headphone 50 attached to the head of the user 20 and provided as a sound from the headphone 50 to the ear of the user 20.

  In the electrooculogram information processing system 10, an authentication passcode to be input before the user 20 uses the function of the smartphone 40 is set in the smartphone 40. For example, when in the locked state, the smartphone 40 displays the passcode input screen 42 and requests the user 20 to input the passcode with a line of sight. The smartphone 40 releases the locked state when the correct passcode is input by the line-of-sight input.

  The smartphone 40 determines whether or not a correct passcode is input based on the electrooculogram detected by the glasses 100 when the passcode is input by the line-of-sight input. For example, when the passcode input screen 42 is displayed, the smartphone 40 determines that the line-of-sight direction has changed downward after the line-of-sight direction has changed to the right based on the electrooculogram of the user 20. It is determined that the correct passcode has been input, and the lock state is released.

  At this time, the smartphone 40 considers that the user 20 is looking at the display position “2” in the passcode input screen 42 at the timing when the electrooculogram representing the change in the gaze direction to the right is detected. In addition, the smartphone 40 considers that the user 20 is viewing the display position “3” in the passcode input screen 42 at the timing when the electrooculogram representing the downward change in the line-of-sight direction is detected. Thereby, the smartphone 40 obtains a correlation between the electrooculogram and the line-of-sight direction of the user 20. The smartphone 40 analyzes the electrooculogram detected after obtaining the correlation based on the correlation, and specifies the line-of-sight direction of the user 20. Thereby, the user's 20 gaze direction can be specified more correctly. According to the electrooculogram information processing system 10, the smartphone 40 can be unlocked and the correlation between the electrooculogram and the line-of-sight direction can be acquired simultaneously.

  In the description of the present embodiment, various directions may be specified using the coordinate axes of the orthogonal coordinate system shown in FIG. The z-axis plus direction is defined as a direction along the front of the user 20. The z-axis plus direction is an acceleration in a direction from the face of the user 20 toward the front portion of the glasses 100 of the glasses 100 attached to the face of the user 20. Further, the negative y-axis direction is defined as the vertical direction, that is, vertically downward. The x-axis, y-axis, and z-axis are right-handed orthogonal coordinate systems. For convenience of explanation, the z-axis plus direction may be referred to as the front direction. In addition, the y-axis plus direction may be referred to as upward or the like. Also, the y-axis minus direction may be referred to as downward or the like. Further, the x-axis plus direction may be referred to as the left side or the like. Also, the x-axis minus direction may be referred to as the right side or the like.

  FIG. 2 schematically shows the glasses 100 and the smartphone 40. The glasses 100 include a lens 110 and a frame 120. Glasses 100 and frame 120 are examples of eyewear.

  The frame 120 supports the pair of lenses 110. The frame 120 includes a rim 122, a bridge 124, an alloy 126, a temple 130, a modern 132, a right nose pad 141, a left nose pad 142, a first electrode 151, a second electrode 152, a third electrode 153, a ground electrode 154, and a processing unit. 180 and a power supply unit 190. The lens 110, the rim 122, the armor 126, the temple 130, and the modern 132 are each provided in a pair of left and right. Of the frame 120, the rim 122, the bridge 124, the right nose pad 141, the left nose pad 142, and the armor 126 are referred to as a front portion of the glasses 100.

  The rim 122 holds the lens 110. The armor 126 is provided outside the rim 122 and holds the temple 130 movably. The temple 130 presses the upper part of the user's 20 ear and pinches the pressed part. The modern 132 is provided at the tip of the temple 130. The modern 132 contacts the upper part of the ear of the user 20.

  The first electrode 151 is an example of an electrooculogram detection unit that detects electrooculogram. The first electrode 151 is provided on the surface of the right nose pad 141. The first electrode 151 is provided on the surface of the right nose pad 141 that faces the face of the user 20 when the user 20 wears the glasses 100. When the user 20 wears the glasses 100, the first electrode 151 contacts the skin of the user 20. For example, when the user 20 wears the glasses 100, the first electrode 151 contacts the right side of the user 20 nose. In the present embodiment, the first electrode 151 mainly detects the electrooculogram of the right eye of the user 20.

  The second electrode 152 is an example of an electrooculogram detection unit that detects an electrooculogram. The second electrode 152 is provided on the surface of the left nose pad 142. The second electrode 152 is provided on the surface of the left nose pad 142 that faces the face of the user 20 when the user 20 wears the glasses 100. When the user 20 wears the glasses 100, the second electrode 152 contacts the skin of the user 20. For example, when the user 20 wears the glasses 100, the second electrode 152 contacts the left side of the user 20 nose. In the present embodiment, the second electrode 152 mainly detects the electrooculogram of the left eye of the user 20.

  The third electrode 153 is an example of an electrooculogram detection unit that detects electrooculogram. The third electrode 153 is provided on the surface of the bridge 124. The third electrode 153 is provided on the surface of the bridge 124 that faces the face of the user 20 when the user 20 wears the glasses 100. When the user 20 wears the glasses 100, the third electrode 153 contacts the skin of the user 20. For example, when the user 20 wears the glasses 100, the third electrode 153 contacts the upper part of the eyebrow of the user 20. In the present embodiment, the electrooculogram detected by the third electrode 153 is used as a measurement reference for measuring the electrooculogram of the right eye and the electrooculogram of the left eye of the user 20.

  The ground electrode 154 is provided on the surface of the modern 132. The ground electrode 154 is provided on the surface of the modern 132 on the right side, for example. The ground electrode 154 is provided on the surface of the modern 132 that faces the face of the user 20 when the user 20 wears the glasses 100. When the user 20 wears the glasses 100, the ground electrode 154 contacts the user 20's skin. For example, when the user 20 wears the glasses 100, the ground electrode 154 contacts the upper part of the right ear of the user 20. In the present embodiment, the potential of the ground electrode 154 provides the ground potential of the electric circuit included in the glasses 100.

  The processing unit 180 is provided inside the left temple 130. The electrooculogram of the user 20 detected by the first electrode 151, the second electrode 152, and the third electrode 153 is input to the processing unit 180. The processing unit 180 processes the input electrooculogram and transmits the processed potential to the smartphone 40.

  The power supply unit 190 is provided inside the left temple 130. The power supply unit 190 includes a battery such as a secondary battery. The power supply unit 190 supplies the electrical energy stored in the battery included in the power supply unit 190 to the processing unit 180. Specifically, the power supply unit 190 generates DC power based on the potential of the ground electrode 154 from the electrical energy stored in the battery. The power supply unit 190 supplies DC power generated from the electrical energy stored in the battery to the processing unit 180.

  The power supply unit 190 is provided inside the temple 130 on the side where the ground electrode 154 is provided. The potential of the ground electrode 154 provides a negative potential of DC power supplied from the power supply unit 190 to the processing unit 180. The right modern 132 has a charging port for charging the power supply unit 190. The battery included in the power supply unit 190 is charged through a charging port provided in the modern 132 on the right side.

  FIG. 3 schematically shows a functional block configuration of the smartphone 40 and a functional block configuration of the processing unit 180. The processing unit 180 includes a processing unit 200, an angular velocity detection unit 260, an acceleration detection unit 270, a transmission / reception unit 280, and a substrate unit 290. The smartphone 40 includes a processing unit 300, a storage unit 360, a UI unit 370, a transmission / reception unit 380, and a power supply unit 390.

  In the processing unit 180, the processing unit 200, the angular velocity detection unit 260, the acceleration detection unit 270, and the transmission / reception unit 280 are mounted on the substrate unit 290. The processing unit 200 is realized by a processor such as an MPU. Each unit of the smartphone 40 is mainly controlled by the processing unit 300. The transmission / reception unit 280 has a function of performing wireless communication with the smartphone 40. The transmission / reception unit 280 is realized by a communication processor. For example, the transmission / reception unit 280 is realized by a communication processor having a short-range wireless communication function such as Bluetooth (registered trademark).

  An electric wire portion 160 is provided inside the frame 120 of the glasses 100. The electric wire part 160 electrically connects the first electrode 151, the second electrode 152, the third electrode 153, the ground electrode 154, the power supply unit 190, and the processing unit 180. The electric wire unit 160 electrically connects the first electrode 151 and the processing unit 180, outputs an electrooculogram detected by the first electrode 151 to the processing unit 180, and the second electrode 152 and the processing unit 180. Are electrically connected to each other, and the third electrode 153 and the processing unit 180 are electrically connected to the third electrode 153 by electrically connecting the electric wire that outputs the electrooculogram detected by the second electrode 152 to the processing unit 180. And an electric wire for supplying electric power from the power supply unit 190 to the processing unit 180.

  In the glasses 100, the processing unit 200 acquires the electrooculogram of the user 20 and processes the acquired electrooculogram. Specifically, the processing unit 200 acquires the first electrooculogram that is the electrooculogram detected by the first electrode 151, and processes the acquired first electrooculogram. In addition, the processing unit 200 acquires a second electrooculogram that is an electrooculogram detected by the second electrode 152 and processes the acquired second electrooculogram. In addition, the processing unit 200 acquires a third ocular potential that is an ocular potential detected by the third electrode 153, and processes the acquired third ocular potential.

  For example, the processing unit 200 processes the first ocular potential based on the third ocular potential. In the present embodiment, the first electrooculogram based on the third electrooculogram is referred to as V1. The processing unit 200 samples V1 at a predetermined cycle and generates time series data of V1. The processing unit 200 outputs the generated time series data of V1 to the transmission / reception unit 280.

  Further, the processing unit 200 processes the second ocular potential based on the third ocular potential. In the present embodiment, the second ocular potential based on the third ocular potential is referred to as V2. The processing unit 200 samples V2 at a predetermined cycle and generates time-series data of V2. The processing unit 200 outputs the generated V2 time-series data to the transmission / reception unit 280.

  The acceleration detection unit 270 detects the acceleration of the glasses 100. The acceleration detection unit 270 is, for example, a triaxial acceleration sensor. The acceleration detection unit 270 detects the acceleration of the center of gravity of the glasses 100. When the glasses 100 are worn on the user 20, the acceleration of the center of gravity of the glasses 100 corresponds to the acceleration of the head of the user 20. The acceleration detected by the acceleration detection unit 270 is input to the processing unit 200.

  The angular velocity detection unit 260 detects the angular velocity of the glasses 100. The angular velocity detection unit 260 is, for example, a triaxial angular velocity sensor. The processing unit 200 receives the angular velocity detected by the angular velocity detection unit 260.

  The processing unit 200 acquires the acceleration detected by the acceleration detection unit 270 and processes the acquired acceleration. The processing unit 200 samples the acceleration at a predetermined cycle, and generates time-series acceleration data. The processing unit 200 outputs the generated time series data of acceleration to the transmission / reception unit 280. The acceleration data output to the transmission / reception unit 280 includes time-series data of acceleration in each direction of the three axes.

  The processing unit 200 acquires the angular velocity detected by the angular velocity detection unit 260 and processes the acquired angular velocity. The processing unit 200 samples the angular velocity at a predetermined cycle, and generates time-series angular velocity data. The processing unit 200 outputs the generated time series data of the angular velocity to the transmission / reception unit 280. The angular velocity data output to the transmission / reception unit 280 includes time-series data of angular velocities in the directions of the three axes.

  The transmission / reception unit 280 transmits the V1 time-series data, the V2 time-series data, the acceleration time-series data, and the angular velocity time-series data acquired from the processing unit 200 to the transmission / reception unit 380 by radio signals. As described above, the transmission / reception unit 280 transmits the electrooculogram information, the acceleration information, and the angular velocity information that are continuously detected to the smartphone 40.

  Note that the processing in the processing unit 200 includes amplification processing for amplifying the input ocular potential, acceleration, and angular velocity signals, and digitization processing for digitizing the input ocular potential, acceleration, and angular velocity signals. The processing unit 200 may include an amplifier circuit that amplifies analog signals of input electrooculogram, acceleration, and angular velocity. The processing unit 200 may include an analog-to-digital conversion circuit that digitizes an analog signal of input electrooculogram, acceleration, and angular velocity or an analog signal amplified by an amplifier circuit.

  In the smartphone 40, the power supply unit 390 includes a battery such as a secondary battery. The power supply unit 390 supplies power to each unit of the smartphone 40 including the processing unit 300, the transmission / reception unit 380, and the UI unit 370.

  The UI unit 370 provides a user interface (UI) with the user 20. For example, the UI unit 370 includes a display unit such as a touch panel, operation keys, a sound generation device, and the like.

  Storage unit 360 is implemented by a storage medium. Examples of the recording medium include a volatile storage medium and a nonvolatile storage medium. The storage unit 360 stores various parameters necessary for the operation of the processing unit 300. The storage unit 360 stores various types of information generated by the processing unit 300.

  The transmission / reception unit 380 has a function of performing wireless communication with the glasses 100. The transmission / reception unit 380 is realized by a communication processor having a short-range wireless communication function such as Bluetooth (registered trademark). The transmission / reception unit 380 and the transmission / reception unit 280 perform wireless communication in accordance with the Bluetooth (registered trademark) standard. Note that communication between the transmission / reception unit 280 and the transmission / reception unit 380 is not limited to Bluetooth (registered trademark) communication. Communication between the transmission / reception unit 280 and the transmission / reception unit 380 can be realized by various types of wireless communication including, for example, a wireless LAN. Communication between the transmission / reception unit 280 and the transmission / reception unit 380 can be realized by various types of wired communication including USB.

  The transmission / reception unit 380 receives information indicating the electrooculogram transmitted from the glasses 100. In addition, the transmission / reception unit 380 receives information indicating the acceleration transmitted from the glasses 100. In addition, the transmission / reception unit 380 receives information indicating the angular velocity transmitted from the glasses 100. Specifically, the transmission / reception unit 380 receives the radio signal received from the transmission / reception unit 280, demodulates the received radio signal, and performs time series data of V1, time series data of V2, time series data of acceleration, and Receive data including time-series data of angular velocity is generated. The transmission / reception unit 380 outputs the generated reception data to the processing unit 300.

  The processing unit 300 includes an electrooculogram acquisition unit 310, an acceleration acquisition unit 320, an angular velocity acquisition unit 322, a lock control unit 330, a correlation acquisition unit 340, and an analysis unit 350.

  The electrooculogram acquisition unit 310 acquires the electrooculogram of the user 20 detected by the electrooculogram detection unit attached to the user 20. Specifically, the electrooculogram acquisition unit 310 acquires the electrooculogram of the user 20 detected by the first electrode 151, the second electrode 152, and the third electrode 153 provided in the glasses 100 worn by the user 20. To do. More specifically, the electrooculogram acquisition unit 310 acquires the electrooculogram detected by the glasses 100 by extracting the time series data of V1 and the time series data of V2 from the reception data output from the transmission / reception unit 380. To do.

  The acceleration acquisition unit 320 acquires the acceleration of the glasses 100 detected by the glasses 100. Specifically, the acceleration acquisition unit 320 acquires the acceleration detected by the glasses 100 based on the information received by the transmission / reception unit 380. More specifically, the acceleration acquisition unit 320 acquires the acceleration detected by the glasses 100 by extracting the acceleration data from the reception data output from the transmission / reception unit 380.

  The angular velocity acquisition unit 322 acquires the angular velocity of the glasses 100 detected by the glasses 100. Specifically, the angular velocity acquisition unit 322 acquires the angular velocity detected by the glasses 100 based on the information received by the transmission / reception unit 380. More specifically, the angular velocity acquisition unit 322 acquires the angular velocity detected by the glasses 100 by extracting the angular velocity data from the reception data output from the transmission / reception unit 380.

  When the lock control unit 330 determines that the electrooculogram acquired by the electrooculogram acquisition unit 310 satisfies a predetermined condition, the lock control unit 330 unlocks the information processing apparatus. The correlation acquisition unit 340, when the lock control unit 330 determines that the electrooculogram satisfies a predetermined condition, the electrooculogram acquired by the electrooculogram acquisition unit 310 and the gaze direction corresponding to the predetermined condition Based on the above, the correlation between the electrooculogram and the line-of-sight direction of the user 20 is acquired.

  As an example, the line-of-sight direction corresponding to a predetermined condition defines the positional relationship between the first position and the second position that the user 20 should see in a predetermined order in order to release the locked state. Then, the lock control unit 330 includes a positional relationship indicated by a change amount of the electrooculogram between the first electrooculogram detected at the first timing and the second electrooculogram detected at the second timing, and the first By comparing the positional relationship between the position and the second position, it is determined whether or not the electrooculogram acquired by the electrooculogram acquisition unit 310 satisfies the predetermined condition. For example, the lock control unit 330 includes the positional relationship indicated by the amount of change in electrooculogram between the first electrooculogram detected at the first timing and the second electrooculogram detected at the second timing, When the positional relationship between the position and the second position matches with a degree of coincidence higher than a predetermined value, it is determined that the electrooculogram acquired by the electrooculogram acquisition unit 310 satisfies the predetermined condition. For example, the lock control unit 330 has a gaze direction indicated by an electrooculogram change amount between the first electrooculogram detected at the first timing and the second electrooculogram detected at the second timing; When the direction of change from the first position to the second position substantially matches, it is determined that the electrooculogram acquired by the electrooculogram acquisition unit 310 satisfies the predetermined condition.

  The lock control unit 330 displays a marker at a position corresponding to each of the first position and the second position in the display area by the display unit. Specifically, the lock control unit 330 displays a passcode input screen 42, for example. The lock control unit 330 is configured such that the change direction of the line-of-sight direction of the user 20 specified based on the electrooculogram is expected from the change direction of the line-of-sight direction expected from the positional relationship of the display position of the passcode displayed on the passcode input screen 42. If they substantially match, it is determined that the electrooculogram acquired by the electrooculogram acquisition unit 310 satisfies a predetermined condition.

  The analysis unit 350 analyzes the electrooculogram acquired by the electrooculogram acquisition unit 310. Specifically, the analysis unit 350 uses the continuous electrooculogram information acquired by the electrooculogram acquisition unit 310 to specify the state of the user 20 at a plurality of timings. For example, the analysis unit 350 specifies the state of the user 20 at a plurality of timings using information on the continuous first and second electrooculograms acquired by the electrooculogram acquisition unit 310. Examples of the state of the user 20 include the user's 20 line-of-sight direction. The analysis unit 350 functions as a gaze direction identification unit that identifies the gaze direction of the user 20.

  For example, the analysis unit 350 acquires information indicating the correlation acquired by the correlation acquisition unit 340 from the correlation acquisition unit 340. The analysis unit 350 specifies the line-of-sight direction of the user 20 after the lock state is released based on the correlation acquired by the correlation acquisition unit 340 and the electrooculogram detected after the lock state is released. In this way, the analysis unit 350 identifies the user's line-of-sight direction using the correlation information of the electrooculogram acquired by the correlation acquisition unit 340 and the continuous electrooculogram information acquired by the electrooculogram acquisition unit 310. To do.

  In addition, the analysis unit 350 analyzes the identified line-of-sight direction of the user 20 and identifies the state of the user 20. Moreover, the analysis part 350 specifies the presence or absence of a blink based on a 1st ocular potential or a 2nd ocular potential. The analysis unit 350 also analyzes the line-of-sight direction and the blink state of the user 20 to identify the state of the user 20. For example, the analysis unit 350 determines whether the user 20 is tired or painful as the state of the user 20. The analysis unit 350 issues a warning to the user 20 through the UI unit 370 when it is determined that the user 20 is tired or painful. Thus, the analysis unit 350 identifies the state of the user 20 based on the correlation acquired by the correlation acquisition unit 340 and the electrooculogram detected after the lock state is released. The state of the user 20 can be identified properly.

  In addition, the analysis unit 350 identifies an instruction from the user 20 based on the identified line-of-sight direction of the user 20. For example, in the smartphone 40, a combination of changes in the line-of-sight direction and an instruction for the smartphone 40 are associated in advance. The analysis unit 350 specifies, as an instruction from the user 20, an instruction associated with a combination that matches the change in the line-of-sight direction of the user 20 specified within a predetermined time. The processing unit 300 controls the operation of each unit of the smartphone 40 based on an instruction from the user 20. For example, when the processing unit 300 is playing music in a predetermined playback order, the processing unit 300 specifies that the analysis unit 350 has instructed the user 20 to play the next song based on the line-of-sight direction of the user 20 Then, the reproduction of the music in the next order currently being reproduced in the reproduction order is started. As described above, the analysis unit 350 specifies the instruction of the user 20 based on the correlation acquired by the correlation acquisition unit 340 and the electrooculogram detected after the lock state is released. The instruction of the user 20 with respect to the smartphone 40 can be specified properly. Thereby, the user 20 can operate the smart phone 40 properly according to the user's 20 line of sight.

  The analysis unit 350 also analyzes at least one of the acceleration acquired by the acceleration acquisition unit 320 and the angular velocity acquired by the angular velocity acquisition unit 322 to identify the movement of the user 20. For example, the analysis unit 350 specifies the running form of the body part of the user 20 based on at least one of acceleration and angular velocity. The storage unit 360 causes the storage unit 360 to store the electrooculogram information analysis result described above and the information indicating the identified movement of the user 20 in association with each other. The UI unit 370 may present information representing the total running posture based on the running form and the line-of-sight direction stored in the storage unit 360 to the user 20 using a video or the like. For example, the UI unit 370 may present an icon such as an arrow indicating the viewing direction of the user 20 to the user 20 together with an icon representing the running form.

  According to the electrooculogram information processing system 10, the smartphone 40 can acquire the correlation between the electrooculogram and the visual line direction of the user 20 when the user 20 performs an operation of releasing the lock of the smartphone 40. Therefore, the line-of-sight direction of the user 20 can be more correctly specified based on the electrooculogram detected thereafter.

  FIG. 4 schematically shows the positional relationship between the electrooculogram detection electrode of the glasses 100 and the user 20. FIG. 4 shows a contact position where the electrooculogram detection electrode contacts the user 20 when the user 20 is wearing the glasses 100.

  The first contact position 451 represents a position where the first electrode 151 contacts the user 20. The second contact position 452 represents a position where the second electrode 152 contacts the user 20. The third contact position 453 represents a position where the third electrode 153 contacts the user 20.

  The first contact position 451 and the second contact position 452 are located below the center of the cornea 411 of the right-eye eyeball 401 and the center of the cornea 412 of the left-eye eyeball 402.

  The first contact position 451 and the second contact position 452 may be at positions where the distance between the first contact position 451 and the eyeball 401 and the distance between the second contact position 452 and the eyeball 402 are substantially equal. desirable. Further, it is desirable that the first contact position 451 and the second contact position 452 are separated from each other by a certain distance or more.

  The third contact position 453 is located above the center of the cornea 411 of the right eyeball 401 and the center of the cornea 412 of the left eyeball 402. The position of the third contact position 453 is a position where the distance between the third contact position 453 and the first contact position 451 and the distance between the third contact position 453 and the second contact position 452 are substantially equal. It may be. The third contact position 453 is such that the distance between the third contact position 453 and the eyeball 401 is separated from the distance between the eyeball 401 and the first contact position 451, and the third contact position 453 and the eyeball 402 are separated from each other. May be at a position that is separated from the distance between the eyeball 402 and the second contact position 452.

  In the eyeball, the cornea side is positively charged and the retina side is negatively charged. Therefore, when the line-of-sight direction of the user 20 changes upward, V1 that is the potential of the first electrode 151 with respect to the third electrode 153 and V2 that is the potential of the second electrode 152 with respect to the third electrode 153 are Both will decline. When the line-of-sight direction of the user 20 changes downward, both the potentials V1 and V2 rise. When the line-of-sight direction of the user 20 changes to the right, V1 decreases and V2 increases. When the line-of-sight direction of the user 20 changes to the left, V1 increases and V2 decreases. The analysis unit 350 identifies the direction in which the line-of-sight direction has changed based on the change in V1 and the change in V2.

  Note that the processing unit 200 measures the electrooculogram of the right eye and the electrooculogram of the left eye by detecting V1 and V2. Therefore, the influence of noise applied to the electrooculogram can be reduced.

  If the head of the user 20 is fixed, the line-of-sight direction of the user 20 is determined by the direction of the eyeball 401 and the direction of the eyeball 402. Since the line-of-sight direction of the user 20 also changes depending on the direction of the head of the user 20, the global line-of-sight direction of the user 20 is determined by the direction of the eyeball 401, the direction of the eyeball 401 and the head of the user 20. In the description of the present embodiment, the line-of-sight direction determined by the direction of the eyeball 401 and the direction of the eyeball 401 may be simply referred to as the line-of-sight direction.

  FIG. 5 schematically shows a change in the viewing direction of the user 20. Here, the direction of the eyeball 401 is taken up. In particular, the direction of the eyeball 401 in the xz plane will be taken up. FIG. 5 shows a case where the eyeball 401 is rotated in the xz plane from the state where the eyeball 401 is oriented in the direction of the angle θ1.

  When the position of the cornea 411 changes as the eyeball 401 rotates in the xz plane from the state in which the eyeball 401 is oriented in the direction of the angle θ1, V1 changes according to the position of the cornea 411. The analysis unit 350 identifies the angle change amount Δθ1 that is the change amount of the rotation angle of the eyeball 401 based on the change amount of V1. The analysis unit 350 identifies the angle of the eyeball 401 after the rotation based on the angle θ1 before the change and the angle change amount Δθ1. Specifically, analysis unit 350 determines that eyeball 401 is oriented at an angle of θ1 + Δθ1.

  In this figure, the rotation angle in the xy plane has been described, but the rotation angles in other planes can be specified in the same manner. As described above, the analysis unit 350 specifies the orientation of the eyeball 401 based on the temporal change of V1. Note that the analysis unit 350 identifies the orientation of the eyeball 402 based on the time change of V2 by the same processing as the processing related to the angle of the eyeball 401.

  The analysis unit 350 identifies the line-of-sight direction of the user 20 based on the orientation of the eyeball 401 and the orientation of the eyeball 402. For example, the analysis unit 350 may specify a direction in which a vector obtained by combining a vector of the direction of the eyeball 401 and a vector of the direction of the eyeball 402 faces as the line-of-sight direction of the user 20.

  Note that the orientation of the eyeball 401 and the orientation of the eyeball 402 are examples of indices representing the line-of-sight direction of the user 20. Further, the line-of-sight direction of the user 20 may be one line-of-sight direction determined from a vector of the direction of the eyeball 401, a vector of the direction of the eyeball 402, and the like. That is, the analysis unit 350 may specify the one line-of-sight direction from V1 and V2. In this case, the analysis unit 350 specifies one gaze direction by performing a predetermined calculation based on the change amount of V1 and the change amount of V2 without specifying the direction of the eyeball 401 and the direction of the eyeball 402. You can do it. For example, the analysis unit 350 may specify one gaze direction by performing a predetermined calculation that associates the variation amount of V1 and the variation amount of V2 with the variation amount of one gaze direction.

  Note that the direction of the eyeball 401 can be rephrased as the position of the cornea 411. Further, the direction of the eyeball 402 can be rephrased as the position of the cornea 412. That is, the analysis unit 350 may specify the position of the cornea of the user 20 based on the electrooculogram detected by the glasses 100. Further, the change in the direction of the eyeball 401 can be rephrased as the eyeball movement of the eyeball 401. Further, a change in the direction of the eyeball 402 can be rephrased as eyeball movement. That is, the analysis unit 350 may specify the eye movement of the user 20 based on the electrooculogram detected by the glasses 100.

  FIG. 6 shows an example of the passcode input screen 42 displayed by the UI unit 370. Here, it is assumed that “123” is set as the correct passcode. The passcode can be set by the user 20. Information indicating the passcode set by the user 20 is stored in the storage unit 360.

  The lock control unit 330 includes marks representing the passcode numbers “1”, “2”, and “3” set by the user 20, and numbers “4” and “5” other than the passcode numbers set by the user 20. , “6”, “7”, “8”, and “9” are displayed on the UI unit 370.

  On the passcode input screen 42, the user 20 can release the passcode by viewing the display position “1”, the display position “2”, and the display position “3” in this order. The lock control unit 330 monitors changes in V1 and V2, and specifies a change in the line-of-sight direction based on the change directions of V1 and V2, so that the user 20 can display the display position “1” and “2”. It is determined whether or not the position and the display position “3” have been viewed in order.

  FIG. 7 schematically shows an example of temporal development of the electrooculogram together with the contents of control by the lock control unit 330. Here, an example of temporal development of V1 and V2 acquired during passcode input is schematically shown.

  A graph 601 shows the time evolution of V1. Graph 602 shows the time evolution of V2. The horizontal axis of the graph 601 and the graph 602 represents time. The vertical axis of the graph 601 represents V1. The vertical axis of the graph 602 represents V2.

  At time t0, the UI unit 370 displays a passcode lock screen. The lock control unit 330 continuously monitors V1 and V2.

  At time t1, the lock control unit 330 determines that V1 has substantially increased and V2 has substantially decreased. Based on the change direction of V1 and the change direction of V2 at time t1, the lock control unit 330 determines that the line-of-sight direction of the user 20 has changed to the left at time t1. The lock control unit 330 determines that the user 20 is viewing the display position of the code to be viewed first at time t1.

  Subsequently, at time t2, the lock control unit 330 determines that V1 has substantially decreased and V2 has substantially increased. Based on the change direction of V1 and the change direction of V2 at time t2, the lock control unit 330 determines that the line-of-sight direction of the user 20 has changed to the right at time t2. Then, the lock control unit 330 determines that the user 20 is looking at the display position of the code to be viewed second at the time t2.

  Subsequently, at time t3, the lock control unit 330 determines that V1 has substantially increased and V2 has also substantially increased. Based on the change direction of V1 and the change direction of V2 at time t3, the lock control unit 330 determines that the line-of-sight direction of the user 20 has changed downward at time t2. Then, the lock control unit 330 determines that the user 20 is viewing the display position of the third code to be viewed at the time t3.

  On the passcode input screen 42, the positional relationship of the passcode “123” is “2” to the right of “1” and “3” below “2”. Therefore, when the user 20 inputs a correct passcode, the direction of the user's line of sight changes from the right to the bottom. As described above, since it changes to the right at time t2 and changes downward at time t3, the lock control unit 330 determines that a change in the line-of-sight direction that satisfies the positional relationship of “123” has been detected. It is determined that the correct passcode has been entered. Therefore, the lock control unit 330 releases the lock at time t4 after time t3. For example, the lock control unit 330 causes the UI unit 370 to display a menu screen that displays a menu that the user 20 instructs the smartphone 40.

  Here, the correlation acquisition unit 340 is in the direction of viewing the display position where the line of sight of the user 20 is “1” at the time t1, and is in the direction of viewing the display position of “2”. Then, it is determined that the line-of-sight direction of the user 20 at time t3 is in the direction of viewing the display position “3”. Then, the correlation acquisition unit 340 calculates the correlation between the change amount in the line-of-sight direction between the display position of “1” and the display position of “2” and the change amount of V1 and the change amount of V2. Obtained as the correlation. In addition, the correlation acquisition unit 340 calculates the correlation between the change amount in the line-of-sight direction between the display position “2” and the display position “3” and the change amount of V1 and the change amount of V2 in the vertical direction. Obtained as the correlation. Thereby, the correlation acquisition part 340 acquires the correlation of the direction orthogonal to each other.

  After unlocking, at time t5, analysis unit 350 determines that V1 has substantially increased and V2 has substantially decreased. Based on the change direction of V1 and the change direction of V2 at time t3, the lock control unit 330 determines that the line-of-sight direction of the user 20 has changed to the left at time t5. The analysis unit 350 also determines the amount of change in the gaze direction to the left based on the correlation in the left-right direction acquired by the correlation acquisition unit 340 and the amount of change in V1 and the amount of change in V2. Is calculated. Then, the analysis unit 350 determines that the current line-of-sight direction has changed by the amount of change. For example, if it can be determined that the line-of-sight direction has not changed since time t3, the analysis unit 350 determines the direction that has changed to the left by the amount of change from the direction facing the display position of the passcode “3”. The current line-of-sight direction may be specified.

  Note that when displaying the passcode input screen, the lock control unit 330 may randomly determine the display position of the mark representing each number. Here, in the lock control unit 330, the positional relationship in the display surface of the plurality of marks representing the passcode numbers set by the user 20 is not similar to the positional relationship in the display surface of any combination of all marks. Thus, the display position of each mark may be determined. For example, in the passcode input screen illustrated in FIG. 6, regarding the positional relationship of the marks “1”, “2”, and “3”, the combination of the same positional relationship is the same regardless of which combination is selected. I can't get it. The shape of the figure formed by the lines connecting the display positions of the marks “1”, “2”, and “3” in order is the shape of any figure formed by the lines connecting the display positions of the marks of other combinations in order. Also different. As described above, when the lock control unit 330 determines the display position of each mark, even when the user 20 cannot determine the exact line-of-sight direction, it can be determined whether or not the correct passcode is input from the change in the line-of-sight direction. There is.

  In the passcode input method described above, the code is identified by a number. As the information for identifying the code, any identifiable information other than numbers can be applied. If the display content represented by the mark is different, the user 20 can identify the code. As information for identifying the code, the shape, size, color, brightness, etc. of the mark can be applied.

  Note that the passcode input described above is not limited to the line-of-sight input. When the UI unit 370 includes a display unit having a touch panel function, the lock control unit 330 can specify a code input by the user 20 based on a touch position touched by the user 20. When inputting a code by a touch operation, the correlation acquisition unit 340 may determine that the user 20 is looking at the touched position, and may acquire a correlation between the touched position and the electrooculogram.

  Further, the method of releasing the lock by inputting the passcode described above is an example of a method of releasing the lock. As another example of the unlocking method, the lock control unit 330 moves the display position of the mark to be displayed in the display area of the UI unit 370 in time, and sets the display position of the mark to the line of sight of the user 20. Or you may make it follow by touch operation.

  For example, the lock control unit 330 may determine to release the lock state when it is determined that the change in the viewing direction of the user 20 matches the change in the display position of the mark to be followed. In this case, the correlation acquisition unit 340 may acquire a correlation between the electrooculogram and the line-of-sight direction based on the correlation between the change in the display position of the mark to be followed and the change in the line-of-sight direction of the user 20.

  In the electrooculogram information processing system 10 described above, the processing unit 200 performs processing for calculating V1 and V2. It replaces with this and the analysis part 350 of the smart phone 40 may perform the process which calculates V1 and V2. In this case, the processing unit 200 generates time series data of the first ocular potential, the second ocular potential, and the third ocular potential, and the transmission / reception unit 280 causes the first ocular potential, the second ocular potential, and the third ocular potential. Each of the time series data may be transmitted to the smartphone 40.

  Further, in the electrooculogram information processing system 10 described above, the analysis unit 350 specifies the state of the user 20 such as the visual line direction of the user 20 using V1 and V2. In addition, the analysis unit 350 may specify the state of the user 20 using an arbitrary linear combination of V1 and V2. For example, the analysis unit 350 may specify the state of the user 20 using V1 + V2 and V1−V2. Moreover, the analysis part 350 may specify the state of the user 20 using the time differentiation of V1 and the time differentiation of V2. In addition, the analysis unit 350 uses the first eye based on a predetermined reference potential such as the potential of the ground electrode 154 instead of the above-described V1 and V2 as the eye potential for specifying the line-of-sight direction. The potential and the second ocular potential based on the potential of the ground electrode 154 may be applied.

  Note that the smartphone 40 may have a function of a detection unit that detects acceleration. In addition, the smartphone 40 may have a function of a detection unit that detects the angular velocity.

  In the above description, the processing described as operations of the processing unit 300 and the transmission / reception unit 380 in the smartphone 40 is realized by a processor such as the processing unit 300 and the transmission / reception unit 380 controlling each hardware included in the smartphone 40 according to a program. The As described above, at least a part of the processing of the smartphone 40 described in relation to the smartphone 40 of the present embodiment includes a processor, a memory, and the like by the processor operating according to the program and controlling each hardware. This can be realized by the hardware and the program operating in cooperation. That is, the process can be realized by a so-called computer. The computer may load a program for controlling execution of the above-described processing, operate according to the read program, and execute the processing. The computer can load the program from a computer-readable recording medium storing the program. Similarly, the processing described as the operations of the processing unit 200 and the transmission / reception unit 280 in the glasses 100 can be realized by a so-called computer.

  The electrooculogram information processing system 10 is an example of an information processing system. The smartphone 40 may process various information without being limited to the electrooculogram. The smartphone 40 is an example of an information processing apparatus that processes information such as electrooculogram detected by the glasses 100. The information processing apparatus may be various electronic devices having a communication function. The information processing apparatus may be a portable electronic device such as a mobile phone, a portable information terminal, a portable music player, etc. possessed by the user 20.

  The eyeglasses 100 as an example of eyewear can be used for the purpose of correcting the refractive error of the eyes of the user 20, protecting the eyes of the user 20, dressing up, and the like. However, eyewear is not limited to glasses. The eyewear may be a face wearing device such as sunglasses, goggles, a head mounted display, or a head wearing device. The eyewear may be a frame of a face wearing device or a head wearing device or a part of the frame. Eyewear is an example of a wearing tool that can be worn by a user. The wearing tool is not limited to a wearing tool related to the eye such as eyewear. Various members such as a hat, a helmet, headphones, and a hearing aid can be applied as the wearing tool.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

10 Ocular Potential Information Processing System 100 Glasses 20 User 40 Smartphone 42 Passcode Input Screen 50 Headphone 110 Lens 120 Frame 122 Rim 124 Bridge 126 Yoroi 130 Temple 132 Modern 141 Right Nose Pad 142 Left Nose Pad 151 First Electrode 152 Second Electrode 153 Third electrode 154 Ground electrode 160 Electric wire unit 180 Processing unit 190 Power supply unit 200 Processing unit 260 Angular velocity detection unit 270 Acceleration detection unit 280 Transmission / reception unit 290 Substrate unit 300 Processing unit 310 Ocular potential acquisition unit 320 Acceleration acquisition unit 322 Angular velocity acquisition unit 330 Lock Control unit 340 Correlation acquisition unit 350 Analysis unit 360 Storage unit 370 UI unit 380 Transmission / reception unit 390 Power supply unit 401, 402 Eyeball 411, 412 Cornea 451 First contact position 452 Second contact Position 453 third contact position 601,602 graph

Claims (9)

An information processing apparatus,
An electrooculogram acquisition unit for acquiring the electrooculogram of the user detected continuously;
A lock control unit for releasing the lock state of the information processing device when it is determined that the electrooculogram acquired by the electrooculogram acquisition unit satisfies a predetermined condition;
When the lock control unit determines that the electrooculogram satisfies the predetermined condition, the electrooculogram and the electrooculogram based on the electrooculogram and the line-of-sight direction corresponding to the predetermined condition An information processing apparatus comprising: a correlation acquisition unit that acquires a correlation with a user's line-of-sight direction. Based on the correlation acquired by the correlation acquisition unit and the electrooculogram detected after the lock state is released, a line-of-sight specifying unit that specifies the user's line-of-sight direction after the lock state is released The information processing apparatus according to claim 1, further comprising: The line-of-sight direction corresponding to the predetermined condition defines the positional relationship between the first position and the second position that the user should see in a predetermined order to release the locked state,
The lock control unit includes a positional relationship indicated by an electrooculogram change amount between the first electrooculogram detected at a first timing and the second electrooculogram detected at a second timing; 3. The method according to claim 1, wherein the positional relationship between the first position and the second position is compared to determine whether the electrooculogram acquired by the electrooculogram acquisition unit satisfies the predetermined condition. The information processing apparatus described. A display unit;
The information processing apparatus according to claim 3, wherein the lock control unit displays a marker at a position corresponding to each of the first position and the second position in a display area by the display unit. The information processing apparatus according to any one of claims 1 to 4, wherein the electrooculogram acquisition unit acquires the electrooculogram detected by an electrooculogram detection unit attached to the user. The information processing apparatus according to claim 5, wherein the electrooculogram acquisition unit acquires the electrooculogram detected by the electrooculogram detection unit provided in a wearing tool worn by the user. The information processing apparatus according to claim 6, wherein the electrooculogram acquisition unit acquires the electrooculogram detected by the electrooculogram detection unit provided in eyewear worn by the user. An information processing apparatus according to any one of claims 1 to 4,
A wearing tool to be worn by the user,
The information processing system in which the electrooculogram acquisition unit acquires the electrooculogram detected by an electrooculogram detection unit provided in the wearing tool. Computer
An electrooculogram acquisition unit for acquiring the electrooculogram of the user detected continuously;
A lock control unit for releasing the locked state of the computer when it is determined that the electrooculogram acquired by the electrooculogram acquisition unit satisfies a predetermined condition;
When it is determined by the lock control unit that the electrooculogram satisfies a predetermined condition, the electrooculogram and the electrooculogram based on the electrooculogram and a reference line-of-sight direction corresponding to the predetermined condition The program for functioning as a correlation acquisition part which acquires the correlation between a user's gaze directions.

Images