JP2014124308A - Eye potential generation device, eye potential generation method, and viewer - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately measure the eye movement of the left and the right eyes with the measurement characteristics consistent between left and right eye potentials.SOLUTION: An eye potential generation device comprises: an electric potential acquisition part 125 which is disposed on a viewer and acquires the electric potential of electrodes described later, the viewer being used attached to the head of a user such that a line connecting an electrode disposed in a nose side of an eye and an electrode disposed in an ear side of the eye intersects a line connecting both eyes, for each of the left and the right eyes; an eyeblink detection part 140 for detecting an eyeblink section, i.e. a time section in which the user blinked the eyes; a correction value calculation part 170 for calculating a correction value of an electric potential by using a ratio between a left eye potential, i.e. an electric potential difference between the electrodes disposed in the left eye side of the user, and a right eye potential, i.e. an electric potential difference between the electrodes disposed in the right eye side of the user in an eyeblink section; and a correction processing part 180 which corrects, by using the correction value calculated by the correction value calculation part 170, the left eye potential or the right eye potential calculated from the electric potential acquired by the electric potential acquisition part 125.

Description

  The present invention relates to an electrooculogram generation device and an electrooculogram generation method that generate electrooculogram information of a user, and a viewer used together with the electrooculogram generation device.

  Patent Document 1 discloses a blink measuring unit that determines a fatigue state of an eyeball based on a viewer's gazing point position, an adjustment position, and a gaze coordinate value of a three-dimensional image, and measures blinks.

  Patent document 2 is disclosing the wearable camera which measures electrooculogram.

  Patent Document 3 discloses a method for estimating the position of an electrode using an eyeball potential model.

  Patent Document 4 discloses that the potential measurement is interrupted periodically during the potential measurement to measure the impedance of each electrode, and the impedance measurement frequency is increased when the impedance is an inappropriate value.

JP 2006-305325 A JP 2011-125893 A JP 2007-252879 A JP 2001-231768 A

  When the impedance of the electrode changes, the electrooculogram also changes. Also, the electrooculogram varies depending on the contact state of the electrodes even if the same eye movement is performed on the left and right. For this reason, calibration is performed to align the measurement characteristics of the left and right electrooculograms. However, in order to perform the calibration, it is necessary to request a special operation such as letting the user gaze at the calibration index. For this reason, during calibration, there is a problem that the electrooculogram cannot be measured for a task being executed by the user. For example, if the impedance of the electrode changes during the measurement of the electrooculogram of the user viewing the 3D video, the viewing of the 3D video must be interrupted. The electrooculogram cannot be measured.

  The present invention has been made in view of such a problem, and even when the contact impedance between the electrode and the skin changes, the user is required to perform a special operation of paying attention to the calibration index. An electrooculogram generation device that can accurately measure left and right eye movements by aligning the measurement characteristics of the left and right electrooculograms.

  The electrooculogram generating device according to one aspect of the present invention connects both eyes with a straight line connecting an electrode arranged on the nose side of the eye and an electrode arranged on the ear side of the eye for each of the left and right eyes. A potential acquisition unit that acquires a potential of each electrode arranged in a viewer used on the user's head so that the straight line intersects, and a blink that is a time interval in which the user has blinked A blink detection unit that detects a section, a left eye potential that is a potential difference between electrodes disposed on the left eye side of the user in the blink section, and an electrode disposed on the right eye side of the user From the potential acquired by the potential acquisition unit using the correction value calculation unit that calculates the correction value of the potential using the ratio with the right eye potential that is a potential difference, and the correction value calculated by the correction value calculation unit A correction processing unit for correcting the calculated left eye potential or right eye potential. Obtain.

  The potential of the electrodes arranged as described above includes a potential generated at the time of blinking. Further, the potential difference generated at the time of blinking is ideally the same for the left and right eyes. For this reason, it is possible to correct the potential by calculating a potential correction value using the potential difference generated at the time of blinking. Therefore, even if the contact impedance between the electrode and the skin changes, the measurement characteristics of the left and right electrooculograms are aligned without requiring the user to perform a special operation to watch the calibration index. Accurate right and left eye movements can be measured.

  Note that these comprehensive or specific modes may be realized by a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, and the system, method, integrated circuit, and computer program. And any combination of recording media.

  According to the present invention, the right and left electrooculogram measurement characteristics can be obtained without requiring the user to perform a special operation of gazing at the calibration index even when the contact impedance between the electrode and the skin varies. The right and left eye movements can be measured accurately.

FIG. 1 is a diagram showing a time waveform of electrooculogram at the time of blinking on the left and right, which is very close to the ideal state. FIG. 2 is a diagram illustrating a typical example of an electrode mounting position with respect to the right eye and an electrooculogram recorded by each electrode pair. FIG. 3 is a diagram showing the ratio between the potential difference of horizontal eye movement and the potential difference of blinks observed by the electrode pairs attached to the eyes and the eyes. FIG. 4 is a diagram illustrating an example of an electrooculogram during blinking. FIG. 5 is a diagram illustrating an example of the correction result of the electrooculogram. FIG. 6 is a block diagram illustrating an example of a configuration of a three-dimensional display system including the electrooculogram correction system according to the first embodiment. FIG. 7A is a schematic diagram illustrating an example of a contact position of an electrode in the first exemplary embodiment. FIG. 7B is a schematic diagram illustrating an example of electrode installation positions in the three-dimensional glasses according to Embodiment 1. FIG. 8 is a diagram illustrating an example of a configuration of content stored in the content information storage unit in the first embodiment. FIG. 9 is a diagram illustrating an example of data stored in the threshold storage unit in the first embodiment. FIG. 10 is a diagram illustrating an example of data stored in the measurement value storage unit according to the first embodiment. FIG. 11 is a flowchart illustrating an example of the operation of the electrooculogram correction system according to the first embodiment. FIG. 12 is a flowchart showing a part of the operation of the electrooculogram correction system according to the first embodiment. FIG. 13 is a block diagram showing a detailed configuration of the blink detection unit in the first embodiment. FIG. 14 is a flowchart showing a part of the operation of the electrooculogram correction system according to the first embodiment. FIG. 15 is a block diagram illustrating a configuration of an electrooculogram correction system according to the first modification of the first embodiment. FIG. 16 is a diagram illustrating an example of a configuration of data stored in the correction value calculation schedule storage unit in the first modification of the first embodiment. FIG. 17 is a flowchart illustrating an example of the operation of the electrooculogram correction system according to the first modification of the first embodiment. FIG. 18 is a block diagram illustrating a configuration of an electrooculogram correction system according to the second modification of the first embodiment. FIG. 19 is a flowchart illustrating an example of the operation of the electrooculogram correction system according to the second modification of the first embodiment. FIG. 20 is a block diagram illustrating an example of a configuration of a three-dimensional display system including the electrooculogram correction system according to the second embodiment. FIG. 21 is a flowchart illustrating an example of the operation of the electrooculogram correction system according to the second embodiment. FIG. 22 is a block diagram illustrating an example of a configuration of a three-dimensional display system including the electrooculogram correction system according to the third embodiment. FIG. 23 is a diagram illustrating an example of a configuration of data stored in the content information storage unit in the third embodiment. FIG. 24 is a flowchart showing a part of the operation of the electrooculogram correction system according to the third embodiment. FIG. 25 is a block diagram illustrating a detailed configuration of the blink estimation unit in the third embodiment. FIG. 26 is a diagram illustrating an example of a configuration of data stored in the compatible section storage unit in the third embodiment. FIG. 27 is a flowchart illustrating a part of the operation of the electrooculogram correction system according to the third embodiment. FIG. 28 is a block diagram illustrating another configuration of the three-dimensional display system including the electrooculogram correction system according to the third embodiment. FIG. 29A is a diagram illustrating an example of a configuration of data stored in a content information storage unit in a modification of the third embodiment. FIG. 29B is a diagram illustrating an example of coordinate axes of a three-dimensional coordinate position. FIG. 30 is a block diagram illustrating a detailed configuration of the blink estimation unit of the electrooculogram correction system according to the modification of the third embodiment. FIG. 31 is a flowchart illustrating a part of the operation of the electrooculogram correction system according to the modification of the third embodiment. FIG. 32 is a block diagram showing a configuration of an electrooculogram correction system which is an example of an electrooculogram generation apparatus including components essential to the present invention.

(Knowledge that became the basis of the present invention)
Various techniques for extracting information by measuring movements of both eyes have been proposed. These include techniques for measuring eye movement during viewing of stereoscopic images and measuring fatigue, and techniques for detecting line of sight. There are two types of eye movement measurement, one using an image of an eyeball using a camera and the other using an electrooculogram measured from an electrode attached around the eyeball. Conditions that affect the potential measured as ocular potential include the position of the electrode and the contact impedance of the electrode and the skin. Patent Document 3 discloses a method for estimating an electrode position. However, since there is a difference in potential due to a difference in impedance for each electrode, the potential when the user is gazing at a calibration index in which eye movement is defined is required to estimate the electrode position. Considering the case of performing fatigue measurement at the time of stereoscopic video viewing, it is difficult to force the user to watch the calibration index when viewing the video. When the electrode is fixed at a certain position on a brace that is worn at a specific position of the head, such as glasses, the electrode is mounted at a position within a certain range of the head. Therefore, the influence of the electrode position on the measured potential is extremely small. However, when the electrooculogram is measured with the electrodes mounted on the three-dimensional glasses and fatigue is determined from the movement state of the eyeball, the movement state of the left and right eyeballs needs to be comparable. That is, it is desirable that the movement of the eyeball at the same angle or the same distance is measured as the same potential with the left and right electrodes. However, the contact state between the skin and the electrode varies from electrode to electrode, and the impedance varies from electrode to electrode. Therefore, even if the left and right eyeballs move at the same angle or the same distance, the measured potential may be different. Therefore, even when the electrode position is determined, calibration is required, and it is difficult to use an electrooculogram for fatigue measurement during video viewing.

  Even when an electrooculogram measurement electrode is mounted on the three-dimensional glasses, the impedance between the skin and the electrode varies from electrode to electrode, and the impedance of a single electrode also varies over time. The electrical state between the skin and the electrode is unstable for about 30 minutes immediately after the electrode is mounted, and the impedance tends to be high. After the electrodes are mounted, the impedance decreases and stabilizes with time, but the impedance difference between the electrodes does not necessarily disappear. When the electrode is temporarily detached from or displaced from the skin, the impedance varies greatly from electrode to electrode, and the impedance difference between the electrodes also varies. Even if the potential is the same, if the electrode impedance is large, the absolute value of the measured potential is small, and if the electrode impedance is small, the absolute value of the measured potential is large. If the impedance differs for each electrode, the potentials measured for each electrode cannot be compared. Patent Document 4 discloses that the potential measurement is periodically interrupted to measure the impedance. When the electrode is mounted on the glasses, the glasses are displaced due to re-wearing of the glasses or a change in posture, and the contact state of the electrodes changes. This causes impedance fluctuations. In the method of Patent Document 4, the measurement of electrooculogram is interrupted every time an impedance change occurs. As described above, the conventional method has a problem that the state in which the eyeball fatigue cannot be determined frequently due to the impedance variation occurs.

  The present invention has been made in view of such a problem, and even when the contact impedance between the electrode and the skin changes, the user is required to perform a special operation of paying attention to the calibration index. An electrooculogram generation device that can accurately measure left and right eye movements by aligning the measurement characteristics of the left and right electrooculograms.

  Before describing the detailed embodiment, the electrical characteristics related to blinking will be described.

  It is said that the electrooculogram in the vertical direction that is measured with the eyeball sandwiched between the eyelids due to blinking increases the contact area between the eyelid on the cornea side and the eyeball when the eyelid is closed. In the eyeball, the cornea on the surface is positively charged and the retina in the back is negatively charged. When the eyelid comes into contact with the cornea during blinking, a cornea-retinal potential is generated in the eyelid. The potential recorded by the electrodes mounted with the eyeball vertically sandwiched in the vertical direction is the potential at the moment of blinking recorded in the electrooculogram (for example, non-patent literature: supervised by Kitaoji Shobo, Hiroshi Miyata, “Physiological Psychology” Science Fundamentals 1 ”, Chapter 16 Blink Activity, 3EOG Method, p271-p272, FIG. 16-6). There are several types of blinks such as unconscious or conscious, and the time pattern or amplitude of the electrooculogram is different. For example, an unconscious blink has a small eyelid movement, and the potential amplitude is slightly smaller than a subjective blink. However, the difference in potential between blinks between the left and right eyeballs is small. FIG. 1A and FIG. 1B are graphs showing the blinking potential recorded in the vertical eye potential recorded when the impedance of the electrode is low and stable. The horizontal axis is time, and one scale is 0.1 second. The vertical axis represents potential and the unit is microvolt. The left and right electrodes were placed above the eyebrows and below the orbit at the pupil position when viewed from the front. The potential of the graph is the potential recorded simultaneously on the left and right, the graph of FIG. 1A shows the right eye potential, and the graph of FIG. 1B shows the left eye potential. Each graph shows the potential when the blink is performed twice. When the potentials at the second blink are compared, the left and right potentials are almost the same. Comparing the difference between the minimum and maximum potentials (peak-to-peak amplitude) on the left and right, the left-right difference at the first blink is 1.7%, and the left-right difference at the second blink is 3 %. For biological signals with a lot of noise, these values are small and can be ignored.

  As shown in FIG. 2 (d), blinks recorded when the electrodes are mounted at positions where the eyeball is sandwiched in the horizontal direction are very small. In FIG. 2A, the positions where the electrodes near the eye are attached are indicated by numerals “1” to “5”. Here, the electrode at the attachment position i is referred to as an electrode i. FIG. 2B shows time-series data of the potential difference between the electrode 1 and the electrode 2 shown in FIG. 2A (hereinafter, the potential difference is also referred to as a potential). FIG. 2C shows time-series data of the potential difference between the electrode 3 and the electrode 4 shown in FIG. FIG. 2D shows time-series data of the potential difference between the electrode 5 and the electrode 4 shown in FIG. That is, when the electrodes are attached in the vertical direction, the potential changes greatly during blinking as shown in FIG. 2B, whereas when the electrodes are attached in the horizontal direction, the potential changes as shown in FIG. As shown in d), the potential at the time of blinking is small. In addition, when the electrodes are attached in an oblique direction, as shown in FIG. 2 (c), the electric potential at the blinking time is larger than in the case where the electrodes are attached in the horizontal direction.

  FIG. 3 is a diagram showing an outline of the first experiment conducted by the inventors of the present application. FIG. 3A and FIG. 3B are schematic views showing the attachment positions of the electrooculogram measurement electrodes. The numbers in the figure are the angles formed by the electrode pair consisting of the nose side (eye side) electrode and the ear side (eye corner side) electrode with respect to the horizontal direction. The sign of the angle is positive in the direction in which the eyeball is also sandwiched up and down. Therefore, in FIG. 3A, the position of the nose side (upper eye side) electrode is high, so the upper side of the eye has a negative angle and the lower side of the eye has a positive angle. In FIG. 3B, the position of the nose side (eye side) electrode is low, so the lower side of the eye has a negative angle and the upper side of the eye has a positive angle. FIG. 3 (c) shows the ratio of the potential difference between the electrodes associated with the blink to the potential difference between the electrodes associated with the impulsive eye movement (saccade), measured for each pair of the nose electrode and the ear electrode. Show. The horizontal axis indicates the angle formed by the nose side electrode and the ear side electrode with respect to the horizontal direction (straight line connecting both eyes), and the vertical axis indicates the above-described potential difference (potential) ratio. The impulsive eye movement (saccade) assumed here is an eye movement when the horizontal viewing angle is 10 degrees. The reason why the viewing angle in the horizontal direction is set to 10 degrees is that the angle of convergence and movement of the eyeball that occurs per predetermined time is at most 10 degrees.

  We examined the arrangement of electrode pairs that can record blinks with a potential that can be clearly distinguished from the potential associated with eye movement in the horizontal direction with a viewing angle of 10 degrees, which is often used for EOG calibration.

  As a result, in the case where the ear side (eye corner side) electrode is arranged in a direction in which the eyes are sandwiched horizontally and vertically from the nose side (eye side) electrode, the angle formed by the electrode pair with respect to the horizontal direction is 10 More than degree. In addition, when the ear side (eye corner side) electrode is arranged in a direction in which the eyes are not sandwiched vertically from the nose side (eye side) electrode, the angle formed by the electrode pair with respect to the horizontal direction is 40 degrees or more. And By arranging the electrode pairs in this way, the above-described potential ratio becomes 1.5 or more. For this reason, it has been shown that the potential associated with blinking can be used separately from the potential associated with eye movement in the horizontal direction.

  Theoretically, the electrodes are mounted on the user so that the relative position to the eyeball and the angle with respect to the horizontal direction are the same on the left and right, and if the impedance of each electrode is the same, the pair of electrodes mounted at a position sandwiching the eyeball diagonally The potential at the time of blinking recorded in is the same on the left and right. Actually, when the electrodes are individually mounted, it is difficult to mount the electrodes so that the relative position and angle with the eyeball are the same on the left and right. In order to mount the electrode so that the relative position and angle with the eyeball are the same on the left and right, a jig that holds the electrode at a symmetrical position is required. Glasses, goggles, or a head-mounted display is such a jig. By fixing the electrodes to the glasses or the like, the electrodes for measuring the electrooculogram are mounted at positions where the relative position and the angle with the eyeball are symmetrical with each other. The difference between the left and right potentials that occurs even though the electrodes are mounted at positions that are symmetrical with respect to the right and left of the eyeball is caused by the contact impedance of the left and right electrodes. If there is a difference between the left and right electrode mounting positions, the potential associated with the movement in a specific direction among the eye movements is recorded larger or smaller depending on the direction of displacement. For this reason, accurate electrode position information is required for potential correction. On the other hand, the potential shift due to the difference in impedance does not depend on the eye movement direction.

  The inventor of the present application discovered that the blink is recorded as a potential equal to the left and right ocular potentials and that the potential shift due to the difference in the contact impedance of the electrodes does not depend on the direction of the eye movement. did. In other words, by correcting the potential generated during eye movement in the horizontal direction using the eye potential during blinking, which shows a large value when the electrode pair is mounted in the vertical direction, the left-right balance of eye movement from the left and right eye potentials is corrected. Was found by a second experiment to be able to be observed with high accuracy.

  FIG. 4A is a diagram schematically showing the arrangement of the electrodes mounted in the second experiment. The electrodes were placed on the nasal bridge on the midline and under the eyes of the left and right eyes, that is, on the ear side with the eyeball sandwiched between the electrodes of the nasal bridge and on the chin side of the nasal bridge electrode position. FIG. 4B is a time waveform of the left and right electrooculograms recorded with the electrode arrangement of FIG. The horizontal axis represents time, and the vertical axis represents the potential difference between the nasal bridge electrode and the ear-side electrode. A solid line indicates the electrooculogram (potential) of the right eye, and a broken line indicates the electrooculogram (potential) of the left eye. A white arrow indicates the potential at the time of blinking. In FIG. 4B, the amplitude of the electrooculogram of the left eye is large. Many of the left and right differences seen in the electrooculogram at the time of blinking, which should be the same, are due to the difference in contact impedance between the left and right electrodes. A correction value for correcting the electrooculogram of the left eye was obtained using the electrooculogram associated with nine blinks excluding abnormal values from the electrooculogram associated with the blink of FIG. 4B. Here, the abnormal value is generated, for example, when blinking and eyeball movement occur simultaneously. For each of the left and right electrooculograms, the difference between the negative peak and the positive peak (peak-to-peak amplitude) for each electrooculogram at the time of blinking is set as the amplitude at each blink, The average of the amplitudes at the time of nine blinks was obtained. The left / right ratio of the average amplitude was used as the correction value.

  Fig.5 (a) is a schematic diagram which shows arrangement | positioning of the electrode similar to Fig.4 (a). FIG. 5B is a time waveform of the electrooculogram when the eye movement is performed in which the points presented in the left-right direction and the horizontal direction are viewed alternately. The horizontal axis represents time, and the vertical axis represents the potential difference between the nasal bridge electrode and the ear-side electrode. A thin solid line indicates the electrooculogram (potential) of the right eye. The broken line indicates the electrooculogram (potential) of the left eye before correction. A thick solid line indicates the corrected electrooculogram (potential) of the left eye. What is shown as a negative potential is the potential when saccade is performed to move the line of sight to a point presented 8 degrees to the right from the gazing point, and is shown as a positive potential. What is shown is the electric potential at the time of performing saccadic eye movement (saccade) that moves the line of sight to a point presented to the left by an angle of view of 8 degrees. The electro-oculogram shown in FIG. 5B is an electro-oculogram recorded with the subject (user) and the electrode as they are after the blinking potential shown in FIG. 4B is recorded. The electrooculogram on the left eye side has a larger amplitude than that on the right eye side. By correcting the electrooculogram on the left eye side, the electrooculogram of the eye movement of 8 degrees to the right, which was largely shifted from the left and right, was corrected. Although there is a portion where the difference is increased by correction for the leftward eye movement, the difference between the left and right is reduced as a whole.

  In the present disclosure, the potential associated with the blink that is naturally performed by the user viewing the 3D image is used, and the impedance difference between the left and right electrodes is calculated based on the difference between the potentials of the left and right electrodes measured during the blink. The difference in the derived measurement potential is corrected. Thus, the left and right electrooculograms are measured and compared without forcing the user to pay attention to the calibration index. Thereby, visual fatigue can be detected from the time change of the electrooculogram. Furthermore, it is known that the electrical resistance of the skin is high immediately after the electrode is attached, gradually decreases, and stabilizes after 30 minutes. By interrupting the viewing of the video for the user who is viewing and forcing the calibration operation frequently, there is no need to correct the correction value. The correction value can be corrected sequentially using the potential associated with the blink that the user who is watching the 3D image naturally performs, and the user can view the video to correct the correction value due to the time variation of the impedance. There is no need to interrupt the action. Similarly, with regard to the discontinuous change in impedance that occurs when the user reapplies or reapplys the 3D glasses, the instantaneous change that occurs after the reapply or reappearance of the spectacles occurs. Correction values can be corrected by using eyes.

  The electrooculogram generating device according to one aspect of the present invention connects both eyes with a straight line connecting an electrode arranged on the nose side of the eye and an electrode arranged on the ear side of the eye for each of the left and right eyes. A potential acquisition unit that acquires a potential of each electrode arranged in a viewer used on the user's head so that the straight line intersects, and a blink that is a time interval in which the user has blinked A blink detection unit that detects a section, a left eye potential that is a potential difference between electrodes disposed on the left eye side of the user in the blink section, and an electrode disposed on the right eye side of the user From the potential acquired by the potential acquisition unit using the correction value calculation unit that calculates the correction value of the potential using the ratio with the right eye potential that is a potential difference, and the correction value calculated by the correction value calculation unit A correction processing unit for correcting the calculated left eye potential or right eye potential. Obtain.

  The potential of the electrodes arranged as described above includes a potential generated at the time of blinking. Further, the potential difference generated at the time of blinking is ideally the same for the left and right eyes. For this reason, it is possible to correct the potential by calculating a potential correction value using the potential difference generated at the time of blinking. Therefore, even if the contact impedance between the electrode and the skin changes, the measurement characteristics of the left and right electrooculograms are aligned without requiring the user to perform a special operation to watch the calibration index. Accurate right and left eye movements can be measured. In addition, even for changes in electrode impedance due to changes in the skin condition and contact state of the electrode due to viewer reappearance, etc., correct correction is always performed by calculating a correction value each time the user blinks. The potential of the left and right electrodes can be corrected by the value. That is, even if there is a change in impedance, the left and right electrooculogram balance can always be maintained. The viewer includes glasses, goggles, and a head mounted display.

  For example, the blink detection unit may detect the blink interval by performing threshold processing on the potential of at least one electrode arranged in the viewer and the change rate of the potential.

  Thereby, a portion where the absolute value of the potential and the change rate of the potential is large can be detected as a blink interval.

  The correction value calculation unit calculates a ratio of peak values of the left eye potential and the right eye potential for each of a plurality of blink intervals, and averages the ratio of the peak values of the plurality of blink intervals. A value may be calculated as the correction value.

  By calculating the average value of the ratios of peak values as a correction value, an accurate correction value can be calculated even when there are variations in peak values.

  The electrooculogram generation apparatus described above further includes an electrode movement detection unit that detects movement of the electrode from a potential change for each electrode, and the correction value calculation unit includes the electrode movement detection unit that moves the electrode. Is detected, the correction value is recalculated using the ratio of the left eye potential and the right eye potential in the blink interval after the movement of the electrode, and the correction processing unit is recalculated. The left eye potential or the right eye potential acquired by the potential acquisition unit may be corrected using a correction value.

  When the electrode moves due to the user's viewer being turned on again, the contact impedance between the electrode and the skin varies. For this reason, the correction value can be recalculated in such a case.

  Further, the electrooculogram generation device described above further includes an abnormality determination unit that determines whether or not the user's eyeball moves unexpectedly from the potential change for each electrode, and the correction value calculation unit includes: When the abnormality determining unit determines that the user's eyeball is moving unexpectedly, using the ratio of the left eye potential and the right eye potential in the blink interval after the movement of the electrode, The correction value may be recalculated, and the correction processing unit may correct the left eye potential or the right eye potential acquired by the potential acquisition unit using the recalculated correction value.

  Thereby, when an abnormality in the movement of the eyeball in the horizontal direction is detected, it is possible to avoid using the potential difference in the blink interval until the abnormality is detected, assuming that an error has occurred. Thereby, an accurate correction value can be recalculated.

  In addition, the electrooculogram generation apparatus described above further estimates a time interval in which the horizontal movement of the eyeball during blinking is less than a predetermined amount based on display position information of an object included in the video content viewed by the user And the correction value calculation unit includes the left eye in a blink interval within a time interval in which the eye movement is estimated to be less than a predetermined amount when the blink is estimated by the blink estimation unit. The correction value may be calculated using the ratio of the potential and the right eye potential.

  As a result, the correction value can be calculated using the potential difference in the blink interval where there is little movement of the eyeball in the horizontal direction. For this reason, the correction value can be calculated with high accuracy.

  For example, the electrooculogram generation device described above further includes a content information storage unit that stores display position information of the object included in the video content, and the blink estimation unit stores the content information storage unit in the content information storage unit. Based on the displayed display position information, a time interval in which the horizontal movement of the eyeball during blinking is less than a predetermined amount may be estimated.

  The electrooculogram generation apparatus further estimates a time interval in which the horizontal movement of the eyeball during blinking is less than a predetermined amount based on the depth information of the object included in the video content viewed by the user. A blink estimation unit, wherein the correction value calculation unit is the left eye potential in a blink interval within a time interval in which the eye movement is estimated to be less than a predetermined amount when the blink is estimated by the blink estimation unit. The correction value may be calculated using the right eye potential ratio.

  As a result, the correction value can be calculated using the potential difference in the blink interval where there is little movement of the eyeball in the horizontal direction. In particular, since the movement of the eyeball in the horizontal direction is determined based on the depth information of the object, the correction value can be calculated using the potential difference in the blinking section where the eyeball has less convergence motion. For this reason, the correction value can be calculated with high accuracy.

  For example, the blink estimation unit determines a time interval in which the depth of the object is greater than a predetermined threshold based on the depth information of the object included in the video content in the horizontal direction of the eyeball at the time of blink. It may be estimated as a time interval in which the movement is less than the predetermined amount.

  In addition, the electrooculogram generation apparatus described above further includes a blink estimation unit that detects a time interval in which the amount of change in luminance of the video content viewed by the user is equal to or greater than a predetermined threshold, and the correction value calculation unit includes: The correction value may be calculated using the ratio of the left eye potential and the right eye potential in the blink interval within the time interval detected by the blink estimation unit.

  When the amount of change in luminance of the video content is large, it is considered that a scene change of the video content has occurred. In such a case, the user is considered to blink. For this reason, it is possible to calculate the correction value with high accuracy by calculating the correction value using the potential difference in the blink interval within the time interval in which the luminance of the video content has greatly changed.

  The electrode on the left eye side of the user and the electrode on the right eye side of the user may be arranged so as to be line symmetric with respect to the median line of the user's head.

  Thereby, an electrode can be arrange | positioned left-right symmetrically. Therefore, the correction value can be calculated with high accuracy.

  For each of the left and right eyes, the angle formed by the straight line connecting the electrode arranged on the nose side of the eye and the electrode arranged on the ear side of the eye and the straight line connecting both eyes is It may be 10 degrees or more when arranged at a position where the eyes are sandwiched, and 40 degrees or more when both electrodes are disposed at a position where the eyes are not sandwiched.

  With such an electrode arrangement, it is possible to distinguish between a potential difference due to blinking and a potential difference due to horizontal movement of the eyeball. Therefore, the correction value can be calculated with high accuracy.

  A viewer according to another aspect of the present invention is a viewer that is used together with an electrooculogram generation device that corrects the potential of an electrode arranged around a user's eye using the potential of the electrode at the time of blinking of the user. Then, for each of the left and right eyes of the user, a straight line connecting the electrode arranged on the nose side of the eye and the electrode arranged on the ear side of the eye intersects with a straight line connecting both eyes. And an electrode installed in the viewer.

  The potential of the electrodes arranged as described above includes a potential generated at the time of blinking. Further, the potential difference generated at the time of blinking is ideally the same for the left and right eyes. For this reason, it is possible to correct the potential by calculating a potential correction value using the potential difference generated at the time of blinking. Therefore, the electrooculogram generation device does not require the user to perform a special operation of gazing at the calibration index even when the contact impedance between the electrode and the skin changes, and the left and right ocular potentials. The right and left eye movements can be accurately measured with the same measurement characteristics.

  The electrode on the left eye side of the user and the electrode on the right eye side of the user may be arranged so as to be line symmetric with respect to the median line of the user's head.

  Thereby, an electrode can be arrange | positioned left-right symmetrically. Therefore, the electrooculogram generating apparatus can correct the acquired potential to an accurate potential.

  For each of the left and right eyes, the angle formed by the straight line connecting the electrode arranged on the nose side of the eye and the electrode arranged on the ear side of the eye and the straight line connecting both eyes is It may be 10 degrees or more when arranged at a position where the eyes are sandwiched, and 40 degrees or more when both electrodes are disposed at a position where the eyes are not sandwiched.

  With such an electrode arrangement, it is possible to distinguish between a potential difference due to blinking and a potential difference due to horizontal movement of the eyeball. Therefore, the electrooculogram generating apparatus can correct the acquired potential to an accurate potential.

  Note that these comprehensive or specific modes may be realized by a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, and the system, method, integrated circuit, and computer program. Alternatively, it may be realized by any combination of recording media.

  Hereinafter, embodiments will be specifically described with reference to the drawings.

  It should be noted that each of the embodiments described below shows a comprehensive or specific example. The numerical values, shapes, materials, constituent elements, arrangement positions and connecting forms of the constituent elements, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present invention. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept are described as optional constituent elements.

(Embodiment 1)
FIG. 6 is a configuration diagram of the three-dimensional display system 1 using the electrooculogram correction system 100 according to the first embodiment. The electrooculogram correction system 100 shows an example of an electrooculogram generation device.

  The three-dimensional display system 1 includes a three-dimensional display device 10 and three-dimensional glasses 20 that are an example of a viewer. The electrooculogram correction system 100 includes a plurality of electrodes 110, an electrooculogram measurement unit 120, a threshold storage unit 130, a blink detection unit 140, a measurement value storage unit 150, a blink potential extraction unit 160, and a correction value. A calculation unit 170 and a correction processing unit 180 are provided. In the present embodiment, the electrode 110 and the electrooculogram measurement unit 120 of the electrooculogram correction system 100 are mounted on the three-dimensional glasses 20, and the threshold storage unit 130, the blink detection unit 140, the measurement value storage unit 150, The blink potential extraction unit 160, the correction value calculation unit 170, and the correction processing unit 180 are mounted on the three-dimensional display device 10.

<Three-dimensional glasses>
The three-dimensional glasses 20 include a control signal receiving unit 22, a shutter control unit 23, an electrode 110, an electrooculogram measurement unit 120, and a transmission / reception unit 21.

  The control signal receiving unit 22 receives a control signal transmitted from the 3D display device 10 for synchronizing the screen display and the 3D glasses.

  The shutter control unit 23 performs control for opening and closing a right-eye or left-eye shutter (not shown) in synchronization with the right-eye or left-eye image displayed on the screen.

  The transmission / reception unit 21 performs information communication with the three-dimensional display device 10. The transmission / reception unit 21 transmits the electrooculogram data measured by the electrooculogram measurement unit 120 to the three-dimensional display device 10. Transmission is performed, for example, every 0.1 second, and information indicating the measurement time of the electrooculogram and the potential acquired by each electrode 110 at each measurement time is transmitted.

  Each electrode 110 is placed in contact with the user's skin and acquires the potential of the skin in contact.

  The electrooculogram measurement unit 120 amplifies and quantizes the potential acquired at each electrode 110. The sampling frequency at the time of quantization is 200 Hz, for example.

  FIG. 7A shows an example of a contact position on the user face of the electrode 110 in the present embodiment, and FIG. 7B shows an example of the three-dimensional glasses 20 provided with the electrode 110 of the present embodiment. The electrode 110 is disposed obliquely with respect to the user's left and right eyes with respect to the user's head vertical axis and horizontal axis across one eye. That is, for each of the left and right eyes, the electrode 110 is arranged so that the straight line connecting the electrode arranged on the nose side of the eye and the electrode arranged on the ear side of the eye intersects with the straight line connecting both eyes. ing. For example, the electrode 110a and the electrode 110b contact the right eye 112R by placing the electrode 110a in contact with the upper right side of the right eye 112R and the electrode 110b in contact with the lower left side of the right eye 112R. It is sandwiched diagonally. Also, the electrode 110d and the electrode 110d are arranged so that the electrode 110d contacts the upper left side of the left eye 112L and the electrode 110c contacts the lower right side of the left eye 112L. Is sandwiched diagonally. Furthermore, the electrodes 110 are arranged so that the positions of the electrode pair mounted with the right eye 112R sandwiched between and the electrode pair mounted across the left eye 112L are symmetrical with respect to the midline 111 of the user's head. Has been. For example, the electrode 110a that is in contact with the right upper side of the right eye 112R and the electrode 110d that is in contact with the upper left side of the left eye 112L are symmetrical with respect to the midline 111, and are located on the lower left side of the right eye 112R. The electrode 110b and the lower right electrode 110c of the left eye 112L are in a line-symmetric position with respect to the midline 111. The electrodes 110a to 110d are supported on the three-dimensional glasses 20 as shown in FIG. 7B, for example. For example, the electrode 110 b and the electrode 110 c are installed on the nose pad 231, and the electrode 110 a and the electrode 110 d are installed inside the temple 232. Further, in the three-dimensional glasses 20, an electrode 110 </ b> R that is a reference electrode and an electrode 110 </ b> E that is a body ground are installed inside the ear hook 230. Note that the installation position and number of the electrodes 110 are not limited to this. The electrooculogram measurement unit 120 uses these electrodes 110 to acquire the viewer's electrooculogram. In the example of FIG. 7A, the electrooculogram measurement unit 120 uses the electrodes 110a and 110b to acquire the eye movement of the user's right eye 112R and the electrooculogram associated with the blink. The electrooculogram measurement unit 120 uses the electrode 110c and the electrode 110d to acquire an eye movement associated with the user's left eye 112L and an eye potential associated with blinking.

  Note that the electrode 110b and the electrode 110c illustrated in FIGS. 7A and 7B may be configured as a single electrode. That is, as shown in the schematic diagram of FIG. 4A, by arranging one electrode on the nose bridge on the midline, the electrode 110b and the electrode 110c are caused to play a role in this electrode.

<3D display device>
A three-dimensional display device 10 shown in FIG. 6 includes a content information storage unit 11, a screen control unit 12, a display screen 13, a fatigue determination unit 14, a control signal transmission unit 15, a transmission / reception unit 16, and a threshold storage unit. 130, a blink detection unit 140, a measurement value storage unit 150, a blink potential extraction unit 160, a correction value calculation unit 170, and a correction processing unit 180.

  The content information accumulation unit 11 accumulates moving image content (hereinafter referred to as “content”) including stereoscopic video.

  FIG. 8 shows an example of content stored in the content information storage unit 11. The content information storage unit 11 stores, for example, a plurality of contents (for example, contents A to C). Each content includes content identification information, a time from the start of the content, a right-eye image at each time, Image for the left eye.

  The screen control unit 12 performs a control for displaying each content stored in the content information storage unit 11 on the display screen 13 and a command for controlling the synchronization between the display screen 13 and the three-dimensional glasses 20. Generate a signal.

  The display screen 13 displays content.

  The control signal transmission unit 15 transmits a control signal for synchronizing the screen display and the three-dimensional glasses 20 to the three-dimensional glasses 20 based on the command signal from the screen control unit 12.

  The threshold storage unit 130 stores a threshold for detecting blinks from the electrooculogram.

  The blink detection unit 140 calculates the left eye potential and the right eye potential from the potential of each electrode 110 acquired from the transmission / reception unit 16. In other words, the blink detection unit 140 calculates, for each eye, the potential difference between the potential of the electrode 110R that is the reference electrode for each of the potentials of the two electrodes 110 that are placed across the eye. The blink detection unit 140 calculates the potential of the eye by calculating the difference between the two obtained potential differences. For example, for the electrodes 110a and 110b, which are electrodes placed with the right eye shown in FIGS. 7A and 7B, the blink detection unit 140 determines that the electrodes 110b and 110R are based on the potential difference between the electrodes 110a and 110R. The potential obtained by subtracting the potential difference between the right eye potential and the right eye potential. Similarly for the left eye, the potential obtained by subtracting the potential difference between the electrode 110c and the electrode 110R from the potential difference between the electrode 110d and the electrode 110R is defined as the left eye potential. For example, the blink detection unit 140 cuts out the blink interval based on the threshold value stored in the threshold storage unit 130 from the right eye potential and the left eye potential thus obtained.

  The measurement value storage unit 150 stores the potential of the blink interval cut out by the blink detection unit 140 together with time information.

  The blink potential extraction unit 160 extracts a blink segment used for correction value calculation from a plurality of blink segments stored in the measurement value storage unit 150.

  The correction value calculation unit 170 calculates a correction value for correcting at least one of the left and right electrooculograms from the blink interval extracted by the blink potential extraction unit 160.

  The correction processing unit 180 calculates the right eye potential and the left eye potential for the electrooculogram acquired from the transmission / reception unit 16 in the same manner as the blink detection unit 140. The correction processing unit 180 corrects these right eye potential and left eye potential using the correction value calculated by the correction value calculation unit 170.

  FIG. 9 shows an example of data stored in the threshold storage unit 130. The threshold value storage unit 130, for example, a blink determination threshold value for a positive potential, a blink determination threshold value for a negative potential, a threshold value for a positive potential change rate, a threshold value for a negative potential change rate, The threshold value with respect to the time width between the time when the potential threshold is exceeded and the time when the negative potential is exceeded is stored.

  FIG. 10 shows an example of data stored in the measurement value storage unit 150. For example, the measurement value storage unit 150 is the data of the left or right eye for each blink segment cut by the blink detection unit 140, indicating a cut-out number indicating a cut-out order, a cut-out start time, a cut-out end time, or the like. , The time when the cut-out start time is 0 second, and the potential information measured every time are stored. The potential is measured, for example, every 0.005 seconds. That is, the sampling frequency when the electrooculogram measuring unit 120 digitizes the potential of each electrode is 200 Hz. However, this sampling frequency is an example, and it goes without saying that other sampling frequencies may be used.

  The transmission / reception unit 16 performs information communication with the three-dimensional glasses 20. From the three-dimensional glasses 20, the electrooculogram measurement information is received. The transmission / reception unit 16 functions as the electrooculogram measurement unit 120 illustrated in FIG. The ocular potential measurement information is, for example, information on the difference between the potentials of the electrodes 110a to 110d at each measurement time and the potential of the reference electrode 110R.

<Operation of the electrooculogram correction system 100>
FIG. 11 is a flowchart showing the operation of the electrooculogram correction system 100 of the present embodiment. The operation of the electrooculogram correction system 100 according to the first embodiment will be described with reference to FIGS.

  First, a start signal is input to the electrooculogram correction system 100 from an input unit or an input unit (not shown), and the electrooculogram correction system 100 starts operation (step S1000). The correction processing unit 180 determines whether there is an available correction value output from the correction value calculation unit 170 (step S2000). If it is determined in step S2000 that there is no correction value available, that is, if no in step S2000, the process proceeds to step S3000. If it is determined in step S2000 that there is a correction value available, that is, if yes in step S2000, the process proceeds to step S4000.

  In step S3000, the correction value calculation unit 170 of the electrooculogram correction system 100 calculates a correction value and outputs it to the correction processing unit 180 (step S3000). In step S4000, the correction processing unit 180 corrects and outputs the electrooculogram data measured by the electrooculogram measurement unit 120 according to the correction value (step S4000). The correction processing unit 180 determines whether or not an end signal is input from an input unit or an input unit (not shown) (step S5000). If there is an end signal input in step S5000, that is, if yes in step S5000, the electrooculogram correction system 100 ends the operation (step S6000). If no end signal is input in step S5000, that is, if no in step S5000, the process returns to step S2000.

  FIG. 12 is a flowchart illustrating a part of the operation of the electrooculogram correction system 100 according to the first embodiment.

  Details of the operation in step S3000 will be described with reference to FIGS.

  The electrooculogram measurement unit 120 acquires the user's electrooculogram from the electrode 110 (step S3010). The electrooculogram measurement unit 120 amplifies the potential acquired in step S3010 and digitizes it by analog-digital conversion (step S3020). The blink detection unit 140 determines an interval (hereinafter referred to as “blink interval”) determined as blink according to the threshold stored in the threshold storage unit 130 from the potential of each electrode digitized in step S3020. Time series data of the potential of the blink interval is detected (step S3030). If a blink interval is detected in step S3030, that is, if yes in step S3030, the process proceeds to step S3040. If no blink segment is detected in step S3030, that is, if no in step S3030, the process returns to step S3010. In step S3040, the blink detection unit 140 cuts out the time series data of the potential of the blink interval detected in step S3030 from the time series data of the potential measured by the electrooculogram measurement unit 120 (step S3040). The blink detection part 140 memorize | stores the electrical potential data of the blink area cut out by step S3040 in the measured value memory | storage part 150 with time information (step S3050). The blink potential extraction unit 160 determines whether or not the number of blink segments to which the potential data stored in the measurement value storage unit 150 belongs is greater than the number necessary for correction value calculation (step S3060). The required number of blink intervals is, for example, ten. If it is determined in step S3060 that the measurement value storage unit 150 stores more potential data than the necessary number of blinks, that is, if the answer is yes in step S3060, the process proceeds to step S3070. If it is determined in step S3060 that the measurement value storage unit 150 does not store more than the necessary number of potential data in the blink interval, that is, if no in step S3060, the process returns to step S3010. The blink potential extraction unit 160 extracts potential data of a blink interval that does not include an abnormal value among the latest 10 blink intervals for correction value calculation (step S3070). For example, the average and standard deviation of 10 blink intervals were obtained for the maximum potential in the blink interval and the minimum potential in the blink interval, and one or both of the maximum potential and the minimum potential were more than the standard deviation from the average. A blink interval having a value is defined as a blink interval including an abnormal value. For example, the potential at the fifth blink from the left in FIG. 4 is an abnormal value. At the same time as the blinking, the potential associated with the eye movement in the horizontal direction is added to the right eye potential, so that the peak value of the right eye potential is significantly different from the peak values of other potentials. The correction value calculation unit 170 calculates a correction value for correcting the left and right ocular potentials from the potential data of the plurality of blink intervals extracted in step S3070 (step S3080). That is, the correction value calculation unit 170 extracts, for example, a maximum value, that is, a positive peak potential and a minimum value, that is, a negative peak potential, from potential data included in each blink interval. The correction value calculation unit 170 obtains the ratio of the left and right potentials for each peak, and finds the average of all the ratios including the ratios to the positive peak potentials and the negative peak potentials in all the blink intervals. The correction value calculation unit 170 uses this average value as a correction value. The correction value may be obtained separately by the sign of the potential, or may be obtained from the potential of the peak-to-peak amplitude.

  FIG. 13 is a block diagram illustrating a detailed configuration of the blink detection unit 140 of the electrooculogram correction system 100 according to the first embodiment.

  The blink detection unit 140 includes a negative potential detection unit 141, a negative potential change rate determination unit 142, a positive potential detection unit 143, a positive potential change rate determination unit 144, a cut-out section storage unit 145, and a cut-out unit 146. Prepare.

  FIG. 14 is a flowchart showing a part of the operation of electrooculogram correction system 100 in the first embodiment.

  Based on FIG. 13 and FIG. 14, the blinking section detection process in step S3030 and the potential data cut-out process in the blinking section in step S3040 will be described.

  In step S3031, the negative potential detection unit 141 is measured by the electrooculogram measurement unit 120, and the potential measured as the potential difference between the electrode 110a and the electrode 110R based on the potential of each electrode received by the transmission / reception unit 16, the electrode 110b and the electrode The difference from the potential measured as the potential difference of 110R is obtained as the right eye potential. Further, the negative potential detection unit 141 obtains the difference between the potential measured as the potential difference between the electrode 110c and the electrode 110R and the potential measured as the potential difference between the electrode 110d and the electrode 110R as the left eye potential.

  In step S3032, the negative potential change rate determination unit 142 sets the upper limit value (−60 μV in the example of FIG. 9) for the negative potential, which is the negative potential threshold stored in the threshold storage unit 130, for the right eye potential. The part of the potential below is detected. Further, the negative potential change rate determination unit 142 detects a portion of the left eye potential that is lower than the upper limit value for the negative potential that is the negative potential threshold stored in the threshold storage unit 130. If the above portions are detected for each of the right eye potential and the left eye potential (yes in step S3032), the process proceeds to step S3033. If the above portions are not detected (no in step S3032), step is performed. Return to S3010.

  In this embodiment, the difference in potential between the electrodes is obtained by subtracting the potential of the head electrode from the potential of the electrode on the right and left sides. That is, the right eye potential is obtained by subtracting the electrode 110b potential from the electrode 110a potential, and the left eye potential is obtained by subtracting the electrode 110c potential from the electrode 110d potential. Note that the direction of subtraction when obtaining the potential difference between the electrodes is not limited to this. However, when the subtraction direction is viewed at the vertical position, the subtraction directions must be the same on the left and right. In the first embodiment, for the right eye, the potential of the lower electrode 110b is subtracted from the potential of the upper electrode 110a, and for the left eye, the lower electrode is subtracted from the potential of the upper electrode 110d. The potential of an electrode 110c was subtracted. For example, for the right eye, the potential of the electrode 110b, which is the upper electrode, may be subtracted from the potential of the electrode 110b, which is the lower electrode. It is necessary to obtain the left eye potential by subtracting.

  In step S <b> 3033, the negative potential change speed determination unit 142 determines the potential change of the first part of the detection portion as the potential change per second for each of the left eye potential and the right eye potential detected by the negative potential detection unit 141. Convert to a certain potential change rate. The negative potential change rate determination unit 142 compares the obtained potential change rate with the negative change rate threshold value (−500 μV / s in the example of FIG. 9) stored in the threshold value storage unit 130. When the potential change rate obtained from each of the left eye potential and the right eye potential is less than the negative change rate threshold, the negative potential change rate determination unit 142 sets the detection portion as a blink portion candidate, The process proceeds to S3034. When the potential change rate obtained from each of the left eye potential and the right eye potential is equal to or greater than the negative change rate threshold value (no in step S3033), the process returns to step S3010.

  In step S <b> 3034, the positive potential detection unit 143 starts from the start point of the portion determined as the blink candidate portion by the negative potential change rate determination unit 142, and the positive potential threshold excess point and negative potential stored in the threshold storage unit 130. The period up to the upper limit time (0.3 seconds in the example of FIG. 9) of the time difference when the threshold is exceeded is set as the scanning range.

  In step S3035, the positive potential detection unit 143 detects a portion of the potential that exceeds the lower limit (20 μV in the example of FIG. 9) with respect to the positive potential in the scanning range. If a potential part exceeding the lower limit is detected (yes in step S3035), the process proceeds to step S3036. If a part exceeding the lower limit is not detected (no in step S3035), the process returns to step S3010. .

  In step S <b> 3036, the positive potential change rate determination unit 144 obtains the potential change rate of the first part of the detected portion for each of the left and right potential data of the portion detected by the positive potential detection unit 143. The positive potential change rate determination unit 144 compares the obtained potential change rate with the positive change rate threshold value (200 μV / s in the example of FIG. 9) stored in the threshold value storage unit 130. When the change rate obtained from the left and right potentials exceeds the positive change rate threshold, the positive potential change rate determination unit 144 detects the positive potential detection unit 143 from the portion detected by the negative potential detection unit 141. Up to the part is a series of blink part candidates. Thereafter, the process proceeds to step S3041. If there is no potential change rate exceeding the positive change rate threshold value (No in step S3036), the process returns to step S3010.

  In step S3041, the cutout unit 146 determines a cutout section according to the cutout time width constant stored in the cutout section storage unit 145. The cut-out section storage unit 145 has, for example, 0.05 seconds as a time width that goes back from the start point of the negative potential of the blink candidate portion, and 0.07 seconds as a time width that extends from the end point of the positive potential, for example. , Remembered. The cutout unit 146 sets a time point that goes back 0.05 seconds from the start time point of the series of blink portion candidate parts as the head of the cutout section. In addition, a time point that is 0.07 seconds longer than the end time point of the blink candidate portion is set as the end of the cut-out section.

  In step S3042, the cutout unit 146 cuts out potential data included in the determined cutout section.

  The three-dimensional display system 1 including the electrooculogram correction system 100 that operates in this way displays a three-dimensional image according to the following procedure, and determines the fatigue level of the user in the three-dimensional image.

  With reference to FIG. 6, the screen control unit 12 reads the information of the three-dimensional video including the right-eye image and the left-eye image stored in the content information storage unit 11. The screen control unit 12 controls the display timing so as to alternately display the right eye image and the left eye image. The display screen 13 alternately displays the right-eye image and the left-eye image under the control of the screen control unit 12. On the other hand, the control signal transmission unit 15 transmits the control signal for the display timing of the right eye image and the left eye image output from the screen control unit 12 to the three-dimensional glasses 20. The control signal receiving unit 22 included in the 3D glasses 20 receives the control signals for the display timing of the right eye image and the left eye image transmitted from the control signal transmitting unit 15 of the 3D display device 10. The shutter control unit 23 opens and closes left and right shutters (not shown) according to the image displayed on the display screen 13 included in the three-dimensional display device 10 based on the display timing control signal received by the control signal receiving unit 22. To do. That is, when the display screen 13 displays the right eye image, the right eye shutter is opened and the left eye shutter is closed. When the display screen 13 displays a left-eye image, the left-eye shutter is opened and the right-eye shutter is closed. As a result, the user sees only the right-eye image with the right eye and sees only the left-eye image with the left eye.

  In this embodiment, an example of 3D display in which the shutter of the 3D glasses 20 is opened and closed in synchronization with the display switching of the left and right images on the screen of the 3D display device 10 has been described. This method may be other methods. For example, the control signal transmission unit 15 of the three-dimensional display device 10 and the control signal reception unit 22 and the shutter control unit 23 of the three-dimensional glasses 20 may be omitted. In this case, the screen control unit 12 acquires a video including the right-eye image and the left-eye image from the content information storage unit 11, and the right-eye image and the left-eye image for each horizontal line of the display screen. The video is processed so that the image is displayed alternately. The display screen 13 outputs the video processed by the screen control unit 12. The display screen 13 is provided with a right-eye polarizing filter and a left-eye polarizing filter for each horizontal line. Therefore, the right-eye image is displayed through the right-eye polarizing filter, and the left-eye image is displayed through the left-eye polarizing filter. The three-dimensional glasses 20 are provided with a right eye polarizing filter for the right eye and a left eye changing filter for the left eye. As a result, the right eye can see only the right eye horizontal line, that is, the right eye image, and the left eye can see only the left eye horizontal line, that is, the left eye image.

  Simultaneously with the start of viewing 3D video or wearing of the 3D glasses 20, the electrooculogram measuring unit 120 of the 3D glasses 20 starts measuring the electrooculogram. The electrooculogram measurement unit 120 outputs the time series data of the measured electrooculogram to the transmission / reception unit 21, and the transmission / reception unit 21 transmits the time series data of the electrooculogram to the three-dimensional display device 10. The transmission / reception unit 16 of the three-dimensional display device 10 receives the time series data of the electrooculogram and outputs it to the correction processing unit 180. The correction processing unit 180 performs the correction process in step S4000. The correction processing unit 180 outputs the electrooculogram data that can be compared between the left and right eyes by the correction to the fatigue determination unit 14. The fatigue determination unit 14 records the amplitudes of the opposite phase potentials of the left and right eyes for a certain period of time due to the eye movements in the opposite directions due to the convergence and diverging movement of the eyes. The recording time is, for example, 1 minute. The fatigue determination unit 14 determines fatigue when the left-right bias of the opposite-phase potential amplitude on the left and right is greater than a predetermined value. For example, fatigue is determined when the bias becomes larger by 30% or more than the average of the bias for 10 minutes after the start of viewing. The fatigue determination unit 14 outputs the degree of user fatigue to the screen control unit 12. The screen control unit 12 adjusts the depth of the display image based on the user fatigue state information output from the fatigue determination unit 14.

  As described above, in the electrode pair placed on each of the left and right eyes with the user's eyes diagonally symmetrical with respect to the median line, the right / left ratio of the blink potential acquired by the left and right electrode pairs is obtained. By correcting the electrooculogram acquired by the left and right electrode pairs using the left / right ratio as a correction value, it is possible to compare the electrooculogram regardless of the difference in impedance between the left and right electrodes. This makes it possible to determine eye strain using the balance between the left and right electrooculograms without forcing the user to gaze at the calibration index. In addition, even if the impedance of the electrode changes due to changes in the skin condition or changes in the contact state of the electrodes due to re-wearing glasses, the correct correction value is always corrected by calculating the correction value each time the user blinks. The left and right electrodes can be corrected by the value. Thereby, even when there is a change in impedance, it is possible to determine eye fatigue using the balance between the left and right electrooculograms.

  The blink interval may be detected by other methods. For example, a threshold is set for the rate of change of the potential and the duration of the rate of change, and a method of detecting and cutting out a section where the rate of change in a certain range continues for a certain period or a positive and negative peak value exceeding the threshold is detected and detected. There is a method of cutting out a certain time interval before and after.

  In the first embodiment, the potential at the time of blinking has been described as having a negative large potential first and then a positive potential as shown in FIG. 1. However, the sign of the potential is relative to the electrodes. It changes depending on the direction of subtraction when obtaining the difference between the electrodes. For the potential data in the state where the sign is reversed, the correction value may be obtained by an operation in which the sign is reversed as in the first embodiment.

(Modification 1 of Embodiment 1)
FIG. 15 is a block diagram illustrating a configuration of an electrooculogram correction system 200 according to the first modification of the first embodiment. The electrooculogram correction system 200 shows an example of an electrooculogram generation device.

  In FIG. 15, the transmitting / receiving unit 16 and the transmitting / receiving unit 21 of FIG. 6 are omitted, and the electrooculogram measuring unit 120 and the blink detection unit 140, and the electrooculogram measuring unit 120 and the correction processing unit 180 are directly connected. Similarly to the above, a transmission / reception unit may be provided. The electrooculogram correction system 200 of FIG. 15 includes a timing unit 210 and a correction value calculation schedule storage unit 220, and includes an correction value calculation unit 170A instead of the correction value calculation unit 170. The configuration is the same as 100. The same components as those in FIG. 6 are denoted by the same reference numerals, and description thereof is omitted.

  The electrooculogram correction system 200 includes an electrode 110, an electrooculogram measuring unit 120, a blink detection unit 140, a threshold storage unit 130, a measurement value storage unit 150, a blink potential extraction unit 160, and a correction value calculation unit. 170A, a correction processing unit 180, a timer unit 210, and a correction value calculation schedule storage unit 220 are provided.

  The timer unit 210 measures the elapsed time.

  The correction value calculation schedule storage unit 220 is a correction value based on the elapsed time from the electrode mounting, that is, the 3D glasses 20 in FIG. 6 or the 3D display device 10 in FIG. The schedule for calculating is stored. FIG. 16 shows an example of data stored in the correction value calculation schedule storage unit 220. The correction value calculation schedule storage unit 220 stores a correction value calculation ID for identifying a correction value calculation timing, and a schedule for performing a correction value calculation associated with the correction value calculation ID. The schedule is shown as an elapsed time from the start of viewing (from the wearing of the 3D glasses 20 or the start of display of 3D video by the 3D display device 10). In the example of FIG. 16, the correction value is calculated at a short time interval (for example, every 3 minutes) immediately after the start of viewing. With the elapsed time from the start of viewing, the interval for calculating the correction value becomes longer every 5 minutes and every 10 minutes. For example, after 45 minutes, the correction value is calculated every 10 minutes. Immediately after the electrode is mounted, the impedance is high and unstable. Therefore, the correction value is frequently calculated. Reduce. That is, correction value calculation is frequently performed immediately after electrode mounting with large fluctuations in impedance with respect to variations in contact impedance between electrodes, which is the cause of potential differences between electrodes. When more than 30 minutes have passed after the electrodes are mounted and the impedance variation is small, the frequency of calculating the correction value is reduced. Thereby, the electrooculogram can be corrected by a correction value in accordance with the impedance at the time of measuring the electrooculogram, and the three-dimensional display system 1 can perform fatigue determination with high accuracy.

  FIG. 17 is a flowchart showing the operation of the electrooculogram correction system 200 according to the first modification of the first embodiment. Steps S1000 and S3000 to S6000 in FIG. 17 are the same as those in FIG. The same parts as those in FIG. 11 are denoted by the same reference numerals, and description thereof will be omitted as appropriate. The operation of the electrooculogram correction system 200 according to the first modification of the first embodiment will be described with reference to FIGS. 15 and 17.

  First, a start signal is input to the electrooculogram correction system 200 from an input unit or an input unit (not shown), and the electrooculogram correction system 200 starts operation (step S1000). The correction value calculation unit 170A refers to the correction value calculation schedule storage unit 220, and determines whether or not the elapsed time from the viewing start time measured by the time measuring unit 210 to the present time is the correction value calculation timing (step S2100). ). If it is determined in step S2100 that the elapsed time from the viewing start to the present time measured by the timer 210 is the correction value calculation timing stored in the correction value calculation schedule, that is, if yes in step S2100, step Proceed to S3000. If it is determined in step S2100 that the elapsed time from the viewing start to the present time measured by the timer 210 is not the correction value calculation timing stored in the correction value calculation schedule, that is, if no in step S2100, the process proceeds to step S2200. move on. In step S2200, correction value calculation unit 170A determines whether or not a correction value that can be used to correct the current electrooculogram has been calculated. If it is determined in step S2200 that an available correction value has not been calculated, that is, if no in step S2200, the process returns to step S2100. If it is determined in step S2200 that a usable correction value has been calculated, that is, if yes in step S2200, the process proceeds to step S4000.

  In step S3000, the correction value calculation unit 170A calculates the correction value and outputs it to the correction processing unit 180 (step S3000). In step S4000, the correction processing unit 180 corrects and outputs the electrooculogram data measured by the electrooculogram measurement unit 120 according to the correction value (step S4000). The correction processing unit 180 determines whether or not an end signal is input from an input unit or an input unit (not shown) (step S5000). If there is an end signal input in step S5000, that is, if yes in step S5000, the electrooculogram correction system 100 ends the operation (step S6000). If no end signal is input in step S5000, that is, if no in step S5000, the process returns to step S2100.

  As described above, the electrooculogram correction system 200 includes the time measuring unit 210 and the correction value calculation schedule storage unit 220 that measure the elapsed time after the start of viewing. Thereby, with respect to the impedance of each electrode 110 changing with time, the correction value is recalculated according to a predetermined schedule, and the correction value corresponding to the impedance at the time of measuring the electrooculogram is calculated. Thereby, the electrooculogram is correctly corrected regardless of the time change of the impedance. That is, the three-dimensional display system including the electrooculogram correction system 200 according to the first modification of the first embodiment can always determine eye strain using a balance between left and right electrooculograms.

(Modification 2 of Embodiment 1)
FIG. 18 is a block diagram showing a configuration of an electrooculogram correction system 300 according to the second modification of the first embodiment. The electrooculogram correction system 300 shows an example of an electrooculogram generation device.

  In FIG. 18, the transmitting / receiving unit 16 and the transmitting / receiving unit 21 of FIG. 6 are omitted, and the electrooculogram measuring unit 120 and the blink detection unit 140, and the electrooculogram measuring unit 120 and the correction processing unit 180 are directly connected. Similarly to the above, a transmission / reception unit may be provided. The electrooculogram correction system 300 of FIG. 18 has the same configuration as the electrooculogram correction system 100 of FIG. 6 except that an electrode movement detection unit 310 is added and a correction value calculation unit 170B is provided instead of the correction value calculation unit 170. . The same components as those in FIG. 6 are denoted by the same reference numerals, and description thereof is omitted.

  The electrooculogram correction system 200 includes an electrode 110, an electrooculogram measuring unit 120, a blink detection unit 140, a threshold storage unit 130, a measurement value storage unit 150, a blink potential extraction unit 160, and a correction value calculation unit. 170B, the correction | amendment process part 180, and the electrode movement detection part 310 are provided.

  The electrode movement detection unit 310 is in a state where any one or more of the electrodes 110 have moved their mounting positions or are in contact with the skin due to a rapid change in the electrooculogram waveform of each electrode output from the electrooculogram measurement unit 120. Detect that there was a change.

  The correction value calculation unit 170B recalculates the correction value when the electrode movement detection unit 310 detects the movement of the mounting position of the electrode 110 or the change in the contact state.

  FIG. 19 is a flowchart showing an operation of the electrooculogram correction system 300 according to the second modification of the first embodiment. Steps S1000 and S3000 to S6000 in FIG. 19 are the same as those in FIGS. 11 and 17. Portions similar to those in FIGS. 11 and 17 are denoted by the same reference numerals, and description thereof will be omitted as appropriate. The operation of the electrooculogram correction system 300 according to the second modification of the first embodiment will be described with reference to FIGS.

  First, a start signal is input to the electrooculogram correction system 300 from an input unit or an input unit (not shown), and the electrooculogram correction system 300 starts operation (step S1000). The electrode movement detection unit 310 determines from the potential data measured by the electrooculogram measurement unit 120 whether or not the contact state of one or more of the electrodes 110 has changed (step S2300). If a change in the contact state of the electrode is detected in step S2300, that is, if yes in step S2300, the process proceeds to step S2400. If no change in the contact state of the electrode is detected in step S2300, that is, if no in step S2300, the process proceeds to step S2500. In step S2400, the electrode movement detection unit 310 erases the measurement value of the electrooculogram stored in the measurement value storage unit 150 before the change in the contact state of the electrodes is detected (step S2400). . After step S2400, the process proceeds to step S2500. In step S2500, the correction value calculation unit 170B obtains the correction value calculated after the most recent change in the contact state of the electrode, that is, after the change in the contact state of the electrode detected before the current and closest to the present. It is determined whether or not there is (step S2500). If it is determined in step S2500 that there is no correction value calculated after the most recent change in electrode contact state, that is, if no in step S2500, the process proceeds to step S2600. If it is determined in step S2500 that there is a correction value calculated after the most recent change in the contact state of the electrode, that is, if yes in step S2500, the process proceeds to step S4000. In step S2600, correction value calculation section 170B determines whether or not the measurement value storage section 150 stores measurement values for the number of blink intervals necessary for correction value calculation (step S2600). If it is determined in step S2600 that the measurement values for the number of blink intervals necessary for correction value calculation are stored, that is, if yes in step S2600, the process proceeds to step S3000. If it is determined in step S2600 that the measurement values for the number of blink intervals necessary for correction value calculation are not stored, that is, if no in step S2600, the process returns to step S2300.

  In step S3000, the correction value calculation unit 170B calculates the correction value and outputs it to the correction processing unit 180 (step S3000). In step S4000, the correction processing unit 180 corrects and outputs the electrooculogram data measured by the electrooculogram measurement unit 120 according to the correction value (step S4000). The correction processing unit 180 determines whether or not an end signal is input from an input unit or an input unit (not shown) (step S5000). If there is an end signal input in step S5000, that is, if yes in step S5000, the electrooculogram correction system 100 ends the operation (step S6000). If no end signal is input in step S5000, that is, if no in step S5000, the process returns to step S2100.

  As for the method for detecting the change in the contact state of the electrode, for example, as shown in Japanese Patent No. 4912621, the change in the contact state is detected by using the increase in the energy of the total frequency and the increase in the energy of the power supply frequency. The method to be used is used. In addition, the positive potential is the first threshold value using the first threshold value that is larger than the lower limit threshold value for the positive potential and the second threshold value that is smaller than the upper limit threshold value for the negative potential. It may be determined that the contact state of the electrode has changed when it becomes larger than the threshold value or when the negative potential becomes smaller than the second threshold value.

  As described above, when the change in the contact state of the electrode due to the movement of the electrode after the start of viewing is detected, the correction value is recalculated using the electrooculogram in the blink after the contact state is changed. For example, with respect to a change in the impedance of each electrode caused by a change in the contact state due to reapplying of the three-dimensional glasses 20 or the like, a blink after a change in the detected contact state, that is, a blink after a change in impedance The correction value is calculated using the potential. As a result, the electrooculogram is corrected correctly even when the impedance changes due to the user's operation on the three-dimensional glasses 20 such as re-applying the three-dimensional glasses 20. That is, the three-dimensional display system including the electrooculogram correction system 300 according to the second modification of the first embodiment always uses the balance between the right and left electrooculograms even when the user reapplies the three-dimensional glasses 20. Eye fatigue can be determined.

  Note that the content stored in the correction value calculation schedule storage unit 220 according to the first modification of the first embodiment is not the elapsed time from the start of viewing but the elapsed time from the start of viewing or the change in the contact state of the electrodes. By combining with the first modification 2, the correct correction value is always calculated for the time variation of the impedance until the impedance becomes stable after the contact state largely fluctuates due to reapplying of the three-dimensional glasses 20, etc. The potential can be corrected.

(Embodiment 2)
In the second embodiment, an example will be described in which it is determined that some error has occurred and the correction value is recalculated when the eyeball moves unexpectedly. In the following, the fatigue level is used to determine whether or not the eyeball has moved unexpectedly. However, an index other than the fatigue level may be used as long as it is an index that can determine an unexpected movement of the eyeball. For example, the correction value may be recalculated when an unexpected movement of the eyeball is detected in the eyesight examination apparatus.

  FIG. 20 is a configuration diagram of the three-dimensional display system 2 using the electrooculogram correction system 400 according to the second embodiment. The electrooculogram correction system 400 shows an example of an electrooculogram generation device.

  FIG. 20 includes a correction value calculation unit 170C instead of the correction value calculation unit 170, the fatigue determination unit 14 and the measurement value storage unit 150 are connected, and the fatigue determination unit 14 and the correction value calculation unit 170C are connected. Except for this, it is the same as FIG. The same parts as those in FIG. 6 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  The three-dimensional display system 2 includes a three-dimensional display device 10 </ b> A and three-dimensional glasses 20. The electrooculogram correction system 400 includes a plurality of electrodes 110, an electrooculogram measurement unit 120, a threshold storage unit 130, a blink detection unit 140, a measurement value storage unit 150, a blink potential extraction unit 160, and a correction value. A calculation unit 170C and a correction processing unit 180 are provided. The electrooculogram measurement unit 120 and the blink detection unit 140 of the electrooculogram correction system 400 communicate via the transmission / reception unit 21 and the transmission / reception unit 16.

<Three-dimensional glasses>
20 includes a control signal receiving unit 22, a shutter control unit 23, an electrode 110, an electrooculogram measuring unit 120, and a transmitting / receiving unit 21.

<3D display device>
A three-dimensional display device 10A illustrated in FIG. 20 includes a content information storage unit 11, a screen control unit 12, a display screen 13, a fatigue determination unit 14, a control signal transmission unit 15, a transmission / reception unit 16, and a threshold storage unit. 130, a blink detection unit 140, a measurement value storage unit 150, a blink potential extraction unit 160, a correction value calculation unit 170C, and a correction processing unit 180.

  The fatigue determination unit 14 operates as an abnormality determination unit that determines whether or not the user's eyeball moves unexpectedly. That is, the fatigue determination unit 14 determines that the user's eyeball is moving unexpectedly when it is detected that the degree of fatigue has changed suddenly.

  The correction value calculation unit 170C recalculates the correction value when the fatigue determination unit 14 detects that the degree of fatigue has suddenly changed, that is, when an unexpected movement of the eyeball is detected.

<Operation of the electrooculogram correction system 400>
FIG. 21 is a flowchart showing the operation of the electrooculogram correction system 400 of the present embodiment.

  Steps S1000 and S3000 to S6000 in FIG. 21 are the same as those in FIGS. 11 and 19. Portions similar to those in FIGS. 11 and 19 are denoted by the same reference numerals, and description thereof will be omitted as appropriate. The operation of the electrooculogram correction system 400 according to the second embodiment will be described with reference to FIGS.

  First, a start signal is input to the electrooculogram correction system 400 from an input unit or an input unit (not shown), and the electrooculogram correction system 400 starts operation (step S1000). The fatigue determination unit 14 of the 3D display device 10A determines whether or not the degree of fatigue has changed abruptly (step S2800). That is, the fatigue determination unit 14 determines whether or not the eyeball is moving unexpectedly.

  The case where the degree of fatigue has changed abruptly means that the degree of fatigue has changed by 50% or more with respect to the average of the degree of fatigue over a certain period of time (for example, 10 minutes). If it is determined in step S2800 that there is a sudden change in the degree of fatigue, that is, if yes in step S2800, the process proceeds to step S2400. If it is determined in step S2800 that there is no sudden change in the degree of fatigue, that is, if no in step S2800, the process proceeds to step S2500. In step S <b> 2400, the fatigue determination unit 14 erases the electrooculogram measurement value in the blink interval before a sudden change in the fatigue level is detected from the electrooculogram measurement values stored in the measurement value storage unit 150. (Step S2400). After step S2400, the process proceeds to step S2700. The correction value calculation unit 170C determines whether or not there is a correction value calculated after the most recent sudden change in the fatigue level, that is, after the sudden change in the fatigue level detected at a time earlier than the present and closest to the current time (step) S2700). If it is determined in step S2700 that there is no correction value calculated after the most recent sudden change in fatigue level, that is, if no in step S2700, the process proceeds to step S2600. If it is determined in step S2700 that there is a correction value calculated after the most recent sudden change in the degree of fatigue, that is, if yes in step S2700, the process proceeds to step S4000. In step S2600, correction value calculation section 170C determines whether or not the measurement value storage section 150 stores measurement values for the number of blink intervals necessary for correction value calculation (step S2600). If it is determined in step S2600 that the measurement values for the number of blink intervals necessary for correction value calculation are stored, that is, if yes in step S2600, the process proceeds to step S3000. If it is determined in step S2600 that the measurement values for the number of blink intervals necessary for correction value calculation are not stored, that is, if no in step S2600, the process returns to step S2300.

  In step S3000, the correction value calculation unit 170C calculates the correction value and outputs it to the correction processing unit 180 (step S3000). In step S4000, the correction processing unit 180 corrects and outputs the electrooculogram data measured by the electrooculogram measurement unit 120 according to the correction value (step S4000). The correction processing unit 180 determines whether or not an end signal is input from an input unit or an input unit (not shown) (step S5000). If an end signal is input in step S5000, that is, if yes in step S5000, the electrooculogram correction system 400 ends the operation (step S6000). If no end signal is input in step S5000, that is, if no in step S5000, the process returns to step S2100.

  With the above configuration, in the three-dimensional display system 2 in which the fatigue determination unit 14 detects fatigue of the user and changes the display based on the electrooculogram measured by the electrooculogram measurement unit 120 and corrected by the correction processing unit 180. The determination unit 14 detects a sudden change in the degree of fatigue, and calculates a correction value using the electrooculogram at the blinking time measured after the sudden change in the degree of fatigue. The display value is not changed in the case of a sudden change in the apparent degree of fatigue caused by a sudden change in the contact impedance of the electrode, and a correction value is calculated. This makes it possible to always calculate the correct degree of fatigue by calculating and using the correct correction value without changing the display for changes in the apparent degree of fatigue caused by sudden changes in electrode contact impedance. And correct fatigue judgment can be made.

  A sudden change in the degree of fatigue means an unexpected movement of the eyeball. For this reason, according to the electrooculogram correction system 400, when an unexpected movement of the eyeball occurs, the correction value can be recalculated. Thereby, even if an unexpected movement of the eyeball occurs, a correct electrooculogram can be calculated.

(Embodiment 3)
The electrooculogram at the time of blinking shows a large value in the vertical direction. As shown in the example of FIG. 4, when an electrode is mounted obliquely with respect to the median axis of the user's head and a horizontal axis orthogonal thereto as shown in FIG. Thus, the potential mixed with the horizontal eye movement is not suitable for calculating the correction value. When the eyeball is extremely shifted to the right or left, the eyeball may return to the front position at the same time as blinking, that is, move from the position shifted to the right or left in the horizontal direction. The eyeball is relatively frontal when viewing from a distance, but is congested, that is, in a state of crossing when viewing a close position, and each eyeball is at a position offset toward the head side in the horizontal direction. Therefore, in the third embodiment, by using the depth information of the 3D video, the eye at the time of blinking recorded at the time of presenting an image in which the user's eyeball may be in a congested state The potential is not used for calculation of the correction value, thereby improving the accuracy of the correction value. In other words, by calculating the correction value by using the electrooculogram at the time of blinking recorded other than the time when the image of the user's eyeball may be in a congested state is presented, Increase the accuracy of the correction value.

  FIG. 22 is a configuration diagram of the three-dimensional display system 3 using the electrooculogram correction system 500 according to the third embodiment. The electrooculogram correction system 500 shows an example of an electrooculogram generation device.

  FIG. 22 is the same as FIG. 6 except that a blink detection unit 140A and a content information storage unit 19 are provided instead of the blink detection unit 140 and the content information storage unit 11, and a blink estimation unit 510 is added. . The same parts as those in FIG. 6 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  The three-dimensional display system 3 includes a three-dimensional display device 10 </ b> B and three-dimensional glasses 20. The electrooculogram correction system 500 includes a plurality of electrodes 110, an electrooculogram measurement unit 120, a threshold storage unit 130, a blink detection unit 140A, a measurement value storage unit 150, a blink potential extraction unit 160, and a correction value. A calculation unit 170, a correction processing unit 180, and a blink estimation unit 510 are provided. The electrooculogram measurement unit 120 and the blink detection unit 140 </ b> A of the electrooculogram correction system 500 communicate via the transmission / reception unit 21 and the transmission / reception unit 16.

  The blink estimation unit 510 may cause blinks suitable for calculating correction values without bias in the movements of the left and right eyeballs from the time information and the depth information among the content information stored in the content information storage unit 19. Identify the estimated time interval. That is, the time interval specified by the blink estimation unit 510 is a time interval in which the horizontal movement of the eyeball is less than a predetermined amount.

  The blink detection unit 140A cuts out the blink segment in the same manner as the blink detection unit 140. However, the blink interval is cut out from the time interval specified by the blink estimation unit 510.

<Three-dimensional glasses>
The three-dimensional glasses 20 shown in FIG. 22 includes a control signal receiving unit 22, a shutter control unit 23, an electrode 110, an electrooculogram measurement unit 120, and a transmission / reception unit 21.

<3D display device>
A three-dimensional display device 10B shown in FIG. 22 includes a content information storage unit 19, a screen control unit 12, a display screen 13, a fatigue determination unit 14, a control signal transmission unit 15, a transmission / reception unit 16, and a threshold storage unit. 130, a blink detection unit 140A, a measured value storage unit 150, a blink potential extraction unit 160, a correction value calculation unit 170, a correction processing unit 180, and a blink estimation unit 510.

  FIG. 23 shows an example of content information stored in the content information storage unit 19. The content information storage unit 19 stores at least content time information, left and right image information, and depth information. In the example of FIG. 23, the content information storage unit 19 further stores the average luminance of each image and the scene ID. Further, the depth information includes an object ID and depth information for each object in order to accumulate the depth of the main object in the image. In addition to the example of FIG. 23, the depth information accumulation method includes methods such as an average depth, a maximum value or a minimum value of the entire screen, and an average depth value, a maximum value, or a minimum value for each area into which the screen is divided. is there.

<Operation of the electrooculogram correction system 500>
The outline of the operation of the electrooculogram correction system 500 is the same as that in FIG. 11 of the first embodiment.

  FIG. 24 is a flowchart illustrating a part of the operation of the electro-oculography correction system 500 according to the third embodiment. FIG. 24 is the same as FIG. 12 except that step S3110 is added. The same operations as those in FIG. 12 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  Details of the operation in step S3000 in FIG. 11 will be described with reference to FIGS.

  The electrooculogram measurement unit 120 acquires the user's electrooculogram from the electrode 110 (step S3010). The electrooculogram measurement unit 120 amplifies the potential acquired in step S3010 and digitizes it by analog-digital conversion (step S3020). The blink detection unit 140A determines whether or not the time when the electrooculogram is acquired in step S3010 is within the time interval in which it is estimated that the blink suitable for the correction value calculation output by the blink estimation unit 510 occurs. Determination is made (step S3110). The blink estimation unit 510 determines a time interval in which it is estimated that a blink suitable for correction value calculation will occur based on the depth information accumulated in the content information accumulation unit 19. The blink estimation unit 510 determines a time interval in which it is estimated that a blink suitable for correction value calculation will occur before the image of the time interval is displayed. Any time before display is acceptable. A method in which the blink estimation unit 510 determines a time interval in which it is estimated that a blink suitable for correction value calculation will occur will be described later. If it is determined in step S3110 that the time when the electrooculogram is acquired is within the time interval in which it is estimated that a blink suitable for correction value calculation occurs, that is, if yes in step S3110, the process proceeds to step S3030. . If it is determined in step S3110 that the time when the electrooculogram is acquired is outside the time interval in which it is estimated that a blink suitable for correction value calculation occurs, that is, if no in step S3110, the process returns to step S3010. . In step S3030, the blink detection unit 140A detects time-series data of the blink interval and the potential of the blink interval according to the threshold stored in the threshold storage unit 130 from the potential of each electrode digitized in step S3020. (Step S3030). If a blink interval is detected in step S3030, that is, if yes in step S3030, the process proceeds to step S3040. If no blink segment is detected in step S3030, that is, if no in step S3030, the process returns to step S3010. In step S3040, the blink detection unit 140A cuts out the time series data of the potential of the blink section detected in step S3030 from the time series data of the potential measured by the electrooculogram measurement unit 120 (step S3040). The blink detection unit 140A stores the electrooculogram data of the blink interval cut out in step S3040 in the measurement value storage unit 150 together with the time information (step S3050). The blink potential extraction unit 160 determines whether or not the number of blink segments to which the potential data stored in the measurement value storage unit 150 belongs is greater than the number necessary for correction value calculation (step S3060). The required number of blink intervals is, for example, ten. If it is determined in step S3060 that the measurement value storage unit 150 stores more potential data than the necessary number of blinks, that is, if the answer is yes in step S3060, the process proceeds to step S3070. If it is determined in step S3060 that the measurement value storage unit 150 does not store more than the necessary number of potential data in the blink interval, that is, if no in step S3060, the process returns to step S3010. The blink potential extraction unit 160 extracts potential data of a blink interval that does not include an abnormal value among the latest 10 blink intervals for correction value calculation (step S3070). The correction value calculation unit 170 calculates a correction value for correcting the left and right ocular potentials from the potential data of the plurality of blink intervals extracted in step S3070 (step S3080).

<Configuration and operation of blink estimation section>
FIG. 25 is a block diagram illustrating a detailed configuration of the blink estimation unit 510 of the electrooculogram correction system 500 according to the third embodiment. The blink estimation unit 510 includes a depth information extraction unit 511, a minimum depth extraction unit 512, a matching depth storage unit 513, a matching determination unit 514, and a matching section storage unit 515.

  The depth information extraction unit 511 extracts time information from the information stored in the content information storage unit 19 and depth information of the object corresponding to the time.

  The minimum depth extraction unit 512 obtains a minimum value of depth from the depth information of one or more objects corresponding to a certain time.

  The suitable depth storage unit 513 stores the minimum value of the depth, that is, the threshold value, at which it is estimated that a blink suitable for calculation of the correction value will occur.

  The conformity determination unit 514 compares the minimum depth value extracted by the minimum depth extraction unit 512 with the threshold value stored in the conformance depth storage unit 513 for each time, and the image at the time calculates a correction value. It is determined whether the image is suitable for acquiring potential data at the time of blinking to be used.

  The matching section storage unit 515 stores the determination result of the matching determination unit 514 in association with the content time information. FIG. 26 shows an example of information stored in the compatible section storage unit 515. Information indicating whether the correction value is calculated or not is stored in correspondence with the time information in the content.

  FIG. 27 is a flowchart showing a part of the operation of electrooculogram correction system 500 according to the third embodiment.

  An example of the operation of the blink estimation unit 510 will be described with reference to FIGS.

  The depth information extraction unit 511 extracts and acquires content time information and depth information stored for each object corresponding to the time from the content information storage unit 19 (step S7010). This time information corresponds to one frame of a moving image, for example. Here, description will be made assuming that processing is performed for each frame. Note that the processing may be performed in other units such as a plurality of frames and a fixed time width. The minimum depth extraction unit 512 extracts the minimum value from the depths of the objects included in the frame extracted in step S7010 (step S7020). In the example of FIG. 23, objects with object IDs 19, 20, and 21 correspond to the time 00: 28: 31.032. The depth of the object with object ID 19 and the object with object ID 20 exceeds 700 cm. In contrast, the object ID 21 has a depth of 540 cm. For this reason, the minimum value of the depth is 540 cm. The suitability determination unit 514 compares the minimum value extracted in step S7020 with the threshold value stored in the fit depth storage unit 513 (step S7030). The threshold stored in the suitable depth storage unit is, for example, 300 cm. In step S7030, if the minimum value extracted in step S7020 is less than the threshold, that is, less than 300 cm, that is, if yes in step S7030, the process proceeds to step S7050. In step S7030, if the minimum value extracted in step S7020 is greater than or equal to the threshold value (here, 300 cm), that is, if no in step S7030, the process proceeds to step S7040. In step S7040, the suitability determination unit 514 sets the frame as a fit section (step S7040). In step S7050, the conformity determination unit 514 sets the frame as a non-conforming section (step S7050). The conformity determination unit 514 writes the result determined in steps S7040 and S7050 together with the time information of the frame in the conformity section storage unit 515 (step S7060). After step S7060 ends, the process returns to step S7010. The blink estimation unit 510 executes Steps S7010 to S7060 for all frames of at least one content, and generates information on a suitable section as shown in FIG.

  As described above, the depth information of the content information is used to determine the time interval in which the blink suitable for obtaining the correction value is estimated to occur, and the blink time recorded in the adapted time interval is determined. By determining the correction value using only the electrooculogram, a more accurate correction value can be set.

  In the third embodiment, the blink estimation unit 510 acquires information from the content information storage unit 19, but information is received from the screen control unit 12 like the blink estimation unit 610 of the electrooculogram correction system 600 in FIG. 28. It may be obtained. The electrooculogram correction system 600 shows an example of an electrooculogram generation device.

  In the example of FIG. 28, by acquiring the depth information of the image from the screen control unit 12, it is possible to determine the suitable section for each display time of the content, not for the time attached to the content. As a result, even if the elapsed viewing time does not match the content time due to processing such as fast-forwarding or rewinding, the correction value using the eye potential at the time of blinking obtained in the time interval suitable for correction value calculation Can be calculated.

  In the third embodiment, the blink estimation unit 510 uses only the time information and the depth information in the content information, and divides the time of the content into a time interval suitable for correction value calculation and other time intervals. Furthermore, the content information storage unit 19 may use other information to divide into a time interval suitable for correction value calculation, a time interval not suitable for correction value calculation, and a time interval suitable for recorrection. . For example, when using luminance information, the blink estimation unit 510 performs re-correction using the fact that blinks are likely to occur immediately after a luminance change when the amount of change in luminance is equal to or greater than a predetermined threshold. Determine a suitable time interval. When the scene ID is used, the blink estimation unit 510 uses the fact that blinks are likely to occur due to scene switching, and determines immediately after a scene change as a suitable time interval by re-correction. In addition, when using the depth information, the blink estimation unit 510 recorrects immediately after the sudden change in depth by using the fact that blinks are likely to occur immediately after the sudden change in the depth of the object. It is determined that the time interval is more suitable.

  In FIG. 23, the content information storage unit 19 stores time information, left and right images, average luminance, scene ID, object ID, and depth, but other information may be stored. For example, when the three-dimensional position of the object is accumulated, the blink estimation unit 510 performs not only the depth of the object but also the time interval suitable for the correction value calculation based on the distribution of the three-dimensional position, and the correction value calculation. It is also possible to determine a time interval that does not conform to. For example, even if the depth value of the object is sufficiently large, if the object is on the right or left edge of the screen, the user may be gazing on the right or left side, Sometimes horizontal movement of the eyeball can occur. In order to avoid such a case, the conformity depth storage unit 513 may have a determination threshold value for conformity and nonconformity with respect to the distribution of the three-dimensional position of the object.

  In the third embodiment, the blink estimation unit 510 (610) determines a frame that matches the correction value calculation and a non-conforming frame. However, the content information storage unit 19 preliminarily applies the matching frame and the non-conforming frame for the correction value calculation. May be stored. In this case, the blink detection unit 140A obtains information from the content information storage unit 19 or the screen control unit 12 as to whether each display frame conforms to the correction value calculation or is unsuitable, and the frame conforms to the correction value calculation. The blink is detected from the potential acquired at the time corresponding to.

(Modification of Embodiment 3)
In Embodiment 3, using the depth information of the 3D video, the potential of the blink interval recorded at the time of presenting an image in which the user's eyeball may be in a congested state is By not using the correction value for calculation, the accuracy of the correction value is improved. On the other hand, in this modified example, the moving speed of the object in the 3D video is used to present an image in which the user's eyeball moves greatly and the potential due to eye movement may overlap the potential at the time of blinking. The potential of the blink interval recorded at a certain time is not used for calculation of the correction value, thereby improving the accuracy of the correction value.

  The configuration of the electrooculogram correction system in the present modification is different only in the detailed configuration of the electrooculogram correction system 500 and the blink estimation unit 510 of the third embodiment. The description of the same parts as those in Embodiment 3 is omitted.

  FIG. 29A shows an example of content information stored in the content information storage unit 19. The content information storage unit 19 stores at least content time information, left and right image information, an object ID, and a three-dimensional coordinate position for each object. The three-dimensional coordinate position is stored as, for example, three-axis coordinates of x, y, and z as spatial position coordinates that should be perceived by the viewer.

  FIG. 29B shows an example of the coordinate axis of the three-dimensional coordinate position. In the example of FIG. 29B, the horizontal direction of the display is the x axis, the vertical direction is the y axis, and the depth direction perpendicular to the plane of the display is the z axis. The x axis is positive on the right and negative on the left, and the y axis is positive on the top and negative on the bottom. In the z-axis, the direction approaching the viewer is negative, and the direction away from the viewer is positive. The origin is located at the center of the screen plane.

<Operation of the electrooculogram correction system 500>
The outline of the operation of the electrooculogram correction system 500 is the same as that of the third embodiment shown in FIG.

  The electrooculogram measurement unit 120 acquires the user's electrooculogram from the electrode 110 (step S3010). The electrooculogram measurement unit 120 amplifies the potential acquired in step S3010 and digitizes it by analog-digital conversion (step S3020). The blink detection unit 140A determines whether or not the time when the electrooculogram is acquired in step S3010 is within the time interval in which it is estimated that the blink suitable for the correction value calculation output by the blink estimation unit 510 occurs. Determination is made (step S3110). When the blink suitable for correction value calculation occurs based on the moving speed of the object calculated from the time information accumulated in the content information accumulation unit 19 and the three-dimensional coordinate position information for each object. Determine the estimated time interval. The blink estimation unit 510 determines a time interval in which it is estimated that a blink suitable for correction value calculation will occur before the image of the time interval is displayed. Any time before display is acceptable. A method in which the blink estimation unit 510 determines a time interval in which it is estimated that a blink suitable for correction value calculation will occur will be described later. If it is determined in step S3110 that the time when the electrooculogram is acquired is within the time interval in which it is estimated that a blink suitable for correction value calculation occurs, that is, if yes in step S3110, the process proceeds to step S3030. . If it is determined in step S3110 that the time when the electrooculogram is acquired is outside the time interval in which it is estimated that a blink suitable for correction value calculation occurs, that is, if no in step S3110, the process returns to step S3010. . In step S3030, the blink detection unit 140A detects time-series data of the blink interval and the potential of the blink interval according to the threshold stored in the threshold storage unit 130 from the potential of each electrode digitized in step S3020. (Step S3030). If a blink interval is detected in step S3030, that is, if yes in step S3030, the process proceeds to step S3040. If no blink segment is detected in step S3030, that is, if no in step S3030, the process returns to step S3010. In step S3040, the blink detection unit 140A cuts out the time series data of the potential of the blink section detected in step S3030 from the time series data of the potential measured by the electrooculogram measurement unit 120 (step S3040). The blink detection unit 140A stores the electrooculogram data of the blink interval cut out in step S3040 in the measurement value storage unit 150 together with the time information (step S3050). The blink potential extraction unit 160 determines whether or not the number of blink segments to which the potential data stored in the measurement value storage unit 150 belongs is greater than the number necessary for correction value calculation (step S3060). The required number of blink intervals is, for example, ten. If it is determined in step S3060 that the measurement value storage unit 150 stores more potential data than the necessary number of blinks, that is, if the answer is yes in step S3060, the process proceeds to step S3070. If it is determined in step S3060 that the measurement value storage unit 150 does not store more than the necessary number of potential data in the blink interval, that is, if no in step S3060, the process returns to step S3010. The blink potential extraction unit 160 extracts potential data of a blink interval that does not include an abnormal value among the latest 10 blink intervals for correction value calculation (step S3070). The correction value calculation unit 170 calculates a correction value for correcting the left and right ocular potentials from the potential data of the plurality of blink intervals extracted in step S3070 (step S3080).

<Configuration and operation of blink estimation section>
FIG. 30 is a block diagram illustrating a detailed configuration of the blink estimation unit 510 of the electrooculogram correction system 500 according to a modification of the third embodiment. The blink estimation unit 510 includes an object movement extraction unit 811, a movement speed calculation unit 812, an adapted movement speed storage unit 813, an adaptation determination unit 514, and an adaptation section storage unit 515.

  The object movement extraction unit 811 extracts time information and coordinate position information of each object corresponding to the time from the information stored in the content information storage unit 19.

  The moving speed calculation unit 812 calculates the average moving speed of the time section by obtaining the moving distance of each object in the coordinate space in a predetermined time section and dividing by the time length of the time section.

  The suitable moving speed storage unit 813 stores a range in which the moving speed of the object in the time interval is estimated to cause a blink that matches the correction value calculation.

  The conformity determination unit 514 compares the movement speed of each object calculated by the movement speed calculation unit 812 with the range stored in the adaptation movement speed storage unit 813 for each time, and the image at that time calculates a correction value. It is determined whether or not the image is suitable for acquiring potential data at the time of blinking used in the above.

  The matching section storage unit 515 stores the determination result of the matching determination unit 514 in association with the content time information.

  FIG. 31 is a flowchart showing a part of the operation of the electro-oculography correction system 500 according to the modification of the third embodiment.

  An example of the operation of the blink estimation unit 510 will be described with reference to FIGS. 30 and 31.

  The object movement extraction unit 811 extracts and acquires the time information of the content, the three-dimensional coordinate position information stored for each object corresponding to the time, and the scene ID from the content information storage unit 19 (step S8010). ). Furthermore, the object movement extraction unit 811 cuts out a predetermined time length or the number of frames, for example, a time interval such as 1 second or 60 frames, and coordinate position information of each object corresponding to the time interval (step S8020). The moving speed calculation unit 812 searches for a scene ID change, that is, a scene change in the time section cut out in step S8020 (step S8030). If there is a scene ID change in step S8030 (yes in step S8030), the process returns to step S8020. If the scene ID is not switched in step S8030 (no in step S8030), the moving speed calculation unit 812 calculates the moving speed within the section of each object (step S8040). For example, for each object, the distance between the coordinate positions of adjacent frames is calculated for each object, and this distance is divided by the time length of the cutout section to obtain the average moving speed in the cutout section. calculate. That is, the average moving speed of the object can be obtained by the following formula 1.

However, (x _i , y _i , z _i ) is the coordinate position of the object in frame i in the cutout section, and i represents a time series of n frames, for example, up to 60 frames determined in advance with the start of the cutout section being 0. Shall. T represents a time length of a cut-out section composed of n frames determined in advance, for example, 1 second.

  The suitability determining unit 514 compares the average moving speed of each object calculated in step S8040 with the range of the adapted moving speed stored in the adapted moving speed storage unit 813 (step S8050). The range of the adaptive movement speed stored in the adaptive movement speed storage unit 813 is, for example, less than 60 cm per second. If the average moving speed of each object calculated in step S8040 is within the range of the matching moving speed for all objects, here, if the moving speed of all objects is less than 60 cm per second, that is, yes in step S8050. If YES, go to step S7040. In step S8050, if at least one of the average moving speeds of the objects calculated in step S8040 is outside the range of the matching moving speed, the average moving speed of one of the objects is 60 cm / second or more here. If, in other words, if no in step S8050, the process proceeds to step S7050. In step S7040, the suitability determination unit 514 sets the cut-out section as the fit section (step S7040). In step S7050, the conformity determination unit 514 sets the segmented section as a non-conforming section (step S7050). The conformity determination unit 514 writes the result determined in step S7040 and step S7050 together with the time information of the frame included in the cutout section in the conformity section storage unit 515 (step S7060). After step S7060 ends, the process returns to step S8010. The blink estimation unit 510 executes Steps S8010 to S7060 for all frames of at least one content, and generates information on a suitable section as shown in FIG.

  In this modification, the adaptive movement speed storage unit 813 stores one type of compatible range, that is, less than 60 cm per second. On the other hand, a plurality of types of matching ranges may be stored depending on the depth of the object. For example, for an object whose average depth of the cutout section is less than −100 cm, the average depth of the cutout section is less than 30 cm per second, and for an object whose average depth of the cutout section is −100 cm or more and less than 100 cm, the average depth of the cutout section is less than 60 cm. For objects over 100 cm, it should be less than 90 cm per second. Thus, the range of the adaptive movement speed of the object having a small depth is set to a smaller movement speed, and the range of the adaptive movement speed of the object having a large depth is set to a larger movement speed. The suitability determination unit 514 may determine the suitability section for each object by comparing with the range of the suitability moving speed according to the depth of the object.

  In the present modification, the content information storage unit 19 stores the three-dimensional coordinate position information, and the movement speed calculation unit 812 calculates the movement speed of the object in the three-dimensional coordinate space. On the other hand, the moving speed calculation unit 812 may obtain the angular speed of the eye movement of the standard viewer viewing at the standard position as the moving speed. For example, the standard position is a state in which the distance of the eyeball from the screen plane is three times the vertical length of the screen, and the center between the left and right eyeballs is on the z-axis in FIG. 29B. For example, the standard viewer has a distance between the left and right pupils of 6.5 cm, the eyeball is a sphere having a diameter of 2.3 cm, and the center of rotation of the eyeball is the center of the sphere. In such a case, the adaptive movement speed storage unit 813 stores the compatible range as the angular velocity range of the rotational movement of the eyeball. Based on the angular velocity of the rotational movement of the eyeball, the suitability determination unit 514 determines whether the segmented section of the processing unit is a section where a blink that is suitable for the correction value calculation is expected to occur or is incompatible with the correction value calculation. It is determined whether it is a section in which a short blink is predicted to occur.

  Also, in this modification, the blink estimation unit 510 uses the time information of the content information and the three-dimensional coordinate position information of the object to adapt the time of the content to the correction value calculation based on the moving speed of each object. It was divided into time intervals and other time intervals. On the other hand, the blink estimation unit 510 may generate information on a suitable section suitable for correction value calculation using only the moving speed in the horizontal direction among the three-dimensional coordinate positions. In this case, in step S8040, the moving speed calculation unit 812 calculates the moving distance on the x-axis in FIG. 29B, not the moving distance in the three-dimensional space. The suitable moving speed storage unit 813 stores a suitable range for horizontal movement. In many contents, the object moves greatly in the horizontal direction, and the viewer follows the object by moving the eyeball in the horizontal direction. In addition, the amount of left and right eye movement is equivalent to the movement in the vertical direction, and the potential due to the eye movement is also equivalent to the left and right, but the amount of eye movement in the horizontal direction varies depending on the three-dimensional coordinate position of the object. For this reason, the potential due to the eye movement is also different on the left and right. Therefore, even if the potential due to the eye movement in the vertical direction is superimposed on the potential at the time of blinking, the correction value is not affected, but the eye movement in the horizontal direction is likely to affect the correction value. The accuracy of the correction value can be increased by determining a time interval suitable for the correction value calculation based on the moving speed of the object in the horizontal direction.

  In addition, although this modification demonstrated 3D content, it can apply similarly to the content of 2D display.

  In each of the above embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, the software that realizes each device of each of the above embodiments is the following program.

  That is, this program causes a computer to connect, for each of the left and right eyes, a straight line connecting the electrode arranged on the nose side of the eye and an electrode arranged on the ear side of the eye and a straight line connecting both eyes. To obtain the potential of each electrode arranged in a viewer used by wearing on the user's head, to detect a blink interval that is a time interval in which the user has blinked, the blink In the section, using the ratio of the left eye potential that is the potential difference between the electrodes arranged on the left eye side of the user and the right eye potential that is the potential difference between the electrodes arranged on the right eye side of the user, A potential correction value is calculated, and the left eye potential or right eye potential is corrected using the correction value.

  The blink detection unit described above detects the blink interval from the electrooculogram, but the method for detecting the blink interval is not limited to this. For example, the blinking section may be detected by photographing the user's face with a camera and detecting the timing when the user blinks by image processing.

  FIG. 32 shows a configuration of an electrooculogram correction system that is an example of an electrooculogram generation device including components essential to the present invention. The electrooculogram correction system includes a potential acquisition unit 125, a blink detection unit 140, a correction value calculation unit 170, and a correction processing unit 180. The blink detection unit 140, the correction value calculation unit 170, and the correction processing unit 180 are as described above. The potential acquisition unit 125 acquires the potentials of the plurality of electrodes 110. That is, for each of the left and right eyes, the potential acquisition unit 125 intersects the straight line connecting the electrodes arranged on the nose side of the eye and the electrodes arranged on the ear side of the eye and the straight line connecting both eyes. As described above, the potential of each electrode arranged in the viewer used by being worn on the user's head is acquired. According to the above-described embodiment, the potential acquisition unit 125 may be configured by the electrooculogram measurement unit 120, the transmission / reception unit 21, and the transmission / reception unit 16, for example.

  The electrooculogram correction system according to one or more aspects has been described based on the embodiment, but the present invention is not limited to this embodiment. Unless it deviates from the gist of the present invention, various modifications conceived by those skilled in the art have been made in this embodiment, and forms constructed by combining components in different embodiments are also within the scope of one aspect of the present invention. May be included.

  The electrooculogram correction system according to the present invention can be widely used in a video display system for viewing stereoscopic video using 3D display glasses and a head-mounted display, and is stereoscopically displayed on a cinema screen, a television, a computer display screen, or the like. This is useful when displaying visual images. Further, it can be applied not only to content viewing but also to applications such as image display devices for medical devices such as diagnostic imaging devices and endoscopes, surgery and vehicle simulation games, educational training systems, and the like.

1, 2, 3 3D display system 10, 10A, 10B 3D display device 11, 19 Content information storage unit 12 Screen control unit 13 Display screen 14 Fatigue judgment unit 15 Control signal transmission unit 16, 21 Transmission / reception unit 20 3D use Glasses 22 Control signal receiving unit 23 Shutter control unit 100, 200, 300, 400, 500, 600 Ocular potential correction system 110, 110a, 110b, 110c, 110d, 110E, 110R Electrode 120 Ocular potential measurement unit 125 Ocular potential acquisition unit 130 Threshold storage unit 140, 140A Blink detection unit 141 Negative potential detection unit 142 Negative potential change rate determination unit 143 Positive potential detection unit 144 Positive potential change rate determination unit 145 Cutout section storage unit 146 Cutout unit 150 Measurement value storage unit 160 Blink Potential extraction unit 170, 170A, 170B, 17 C Correction Value Calculation Unit 180 Correction Processing Unit 210 Timekeeping Unit 220 Correction Value Calculation Schedule Storage Unit 310 Electrode Movement Detection Unit 510, 610 Blink Estimation Unit 511 Depth Information Extraction Unit 512 Minimum Depth Extraction Unit 513 Suitable Depth Storage Unit 514 Suitability Determination Unit 515 Suitable section storage unit 811 Object movement extraction unit 812 Movement speed calculation unit 813 Suitable movement speed storage unit

Claims (17)

For each of the left and right eyes, on the user's head so that the straight line connecting the electrode arranged on the nose side of the eye and the electrode arranged on the ear side of the eye intersects the straight line connecting both eyes. A potential acquisition unit that acquires the potential of each electrode arranged in the viewer used by being mounted;
A blink detection unit that detects a blink interval that is a time interval in which the user has blinked;
A ratio of a left eye potential that is a potential difference between electrodes arranged on the left eye side of the user and a right eye potential that is a potential difference between electrodes arranged on the right eye side of the user in the blink interval. And a correction value calculation unit for calculating a correction value of the potential,
An electrooculogram generation device comprising: a correction processing unit that corrects a left eye potential or a right eye potential calculated from the potential acquired by the potential acquisition unit using the correction value calculated by the correction value calculation unit. The electro-oculogram generation device according to claim 1, wherein the blink detection unit detects the blink interval by performing threshold processing on a potential of at least one electrode arranged in the viewer and a change speed of the potential. The correction value calculation unit calculates a ratio of peak values of the left eye potential and the right eye potential for each of a plurality of blink intervals, and calculates an average value of the ratios of the peak values of the plurality of blink intervals. The electrooculogram generation device according to claim 1, wherein the electrooculogram generation device is calculated as the correction value. further,
An electrode movement detection unit that detects movement of the electrode from a potential change for each electrode,
When the electrode movement detection unit detects the movement of the electrode, the correction value calculation unit uses the ratio of the left eye potential and the right eye potential in the blink interval after the movement of the electrode, Recalculate the correction value,
The electrooculogram generation device according to claim 1, wherein the correction processing unit corrects the left eye potential or the right eye potential acquired by the potential acquisition unit using a recalculated correction value. further,
An abnormality determination unit that determines whether or not the user's eyeball is moving unexpectedly from a potential change for each electrode,
The correction value calculation unit, when the abnormality determination unit determines that the user's eyeball is moving unexpectedly, the left eye potential and the right eye in a blink interval after the movement of the electrode Recalculate the correction value using the potential ratio,
The electrooculogram generation device according to claim 1, wherein the correction processing unit corrects the left eye potential or the right eye potential acquired by the potential acquisition unit using a recalculated correction value. further,
Based on display position information of an object included in the video content viewed by the user, a blink estimation unit that estimates a time interval in which the horizontal movement of the eyeball at the blink is less than a predetermined amount,
The correction value calculation unit is a ratio of the left eye potential and the right eye potential in a blink interval within a time interval in which the eye movement is estimated to be less than a predetermined amount during blinking by the blink estimation unit. The electrooculogram generation device according to claim 1, wherein the correction value is calculated using a. further,
A content information storage unit that stores display position information of the object included in the video content;
The eyeblink estimation unit estimates a time interval in which the horizontal movement of the eyeball at the time of blinking is less than a predetermined amount based on the display position information stored in the content information storage unit. Electrooculogram generator. further,
Based on depth information of an object included in the video content viewed by the user, a blink estimation unit that estimates a time interval in which the horizontal movement of the eyeball at the blink is less than a predetermined amount,
The correction value calculation unit is a ratio of the left eye potential and the right eye potential in a blink interval within a time interval in which the eye movement is estimated to be less than a predetermined amount during blinking by the blink estimation unit. The electrooculogram generation device according to claim 1, wherein the correction value is calculated using a. Based on the depth information of the object included in the video content, the blink estimation unit performs a horizontal movement of the eyeball during blinking in a time interval in which the depth of the object is greater than a predetermined threshold. The electrooculogram generation device according to claim 8, wherein the electrooculogram generation device is estimated as a time interval smaller than the predetermined amount. further,
A blink estimation unit that detects a time interval in which the amount of change in luminance of video content viewed by the user is equal to or greater than a predetermined threshold;
The correction value calculation unit calculates the correction value using a ratio of the left eye potential and the right eye potential in a blink interval within a time interval detected by the blink estimation unit. Electrooculogram generation device. The electrooculogram generation device according to claim 1, wherein the left eye electrode of the user and the right eye electrode of the user are arranged so as to be axisymmetric with respect to a median line of the user's head. . For each of the left and right eyes, the angle formed by the straight line connecting the electrode arranged on the nose side of the eye and the electrode arranged on the ear side of the eye and the straight line connecting both eyes is the same for both electrodes. The electrooculogram generation device according to claim 1, wherein the electrooculogram generation apparatus is 10 degrees or more when disposed at a sandwiched position, and is 40 degrees or more when both electrodes are disposed at a position not sandwiching the eye. A viewer that is used together with an electrooculogram generation device that corrects the potential of an electrode arranged around a user's eye using the potential of the electrode at the time of blinking of the user,
For each of the left and right eyes of the user, the viewer is arranged such that a straight line connecting the electrode arranged on the nose side of the eye and the electrode arranged on the ear side of the eye intersects with a straight line connecting both eyes. Viewer equipped with electrodes installed in the. The viewer according to claim 13, wherein the left eye electrode of the user and the right eye electrode of the user are arranged so as to be axisymmetric with respect to a median line of the user's head. For each of the left and right eyes, the angle formed by the straight line connecting the electrode arranged on the nose side of the eye and the electrode arranged on the ear side of the eye and the straight line connecting both eyes is the same for both electrodes. The viewer according to claim 13 or 14, wherein the angle is 10 degrees or more when arranged at a sandwiched position, and 40 degrees or more when both electrodes are arranged at a position not sandwiching the eye. For each of the left and right eyes, on the user's head so that the straight line connecting the electrode arranged on the nose side of the eye and the electrode arranged on the ear side of the eye intersects the straight line connecting both eyes. Obtain the potential of each electrode placed in the viewer used by wearing,
Detecting a blink interval that is a time interval in which the user has blinked;
A ratio of a left eye potential that is a potential difference between electrodes arranged on the left eye side of the user and a right eye potential that is a potential difference between electrodes arranged on the right eye side of the user in the blink interval. To calculate the correction value of the potential,
An electrooculogram generation method that corrects a left ocular potential or a right ocular potential using the correction value.   A program for causing a computer to execute the electrooculogram generation method according to claim 16.

Images