KR19990046531A - Remote controlling apparatus of electronic device using eye motion - Google Patents

Abstract

1. TECHNICAL FIELD OF THE INVENTION

The present invention relates to a remote control device for an electronic device using eye movement.

2. The problem to be solved by the invention

According to the present invention, a minimum number of electrodes is attached to a pair of glasses worn by a person, and a radio frequency control signal is generated by detecting a biopotential generated by eye movement through the electrodes, so that the physically handicapped or the like is convenient for electronic devices such as computers. The purpose of the present invention is to provide a remote control device for an electronic device that can minimize the noise while removing the inconvenience caused by the wiring when wearing the glasses by inserting the wiring of the electrode into the metal frame of the glasses.

3. Summary of Solution to Invention

The present invention, the first to fourth biopotential detection means attached to the glasses to be in contact with the body of the person in order to detect the biopotential generated by the movement of the eyes; The reference signal indicating the size of the vertical motion of the eye is calculated by using the biopotentials detected by the first to fourth biopotential sensing means and a predetermined weighting coefficient to detect the vertical motion of the eye directly above the eye. Reference signal generating means for outputting a reference signal; Eye movement discrimination means for distinguishing the eye movements using the detected biopotentials and the transmitted reference signal, and outputting a signal indicative of the distinguished eye movements, movement directions, and movement magnitudes; Eye motion signal processing means for amplifying and filtering the output signals of the eye motion discrimination means, and then amplifying the separated door motion signal and eye blindness signal and converting the amplified signals into digital signals and outputting the digital signals; And control means for receiving signals from the eye movement signal processing means for the eye movement, the movement direction, and the magnitude of the movement, and outputting a control signal for controlling the operation of the electronic device.

4. Important uses of the invention

The present invention is used to control electronic devices such as computers.

Description

REMOTE CONTROLLING APPARATUS OF ELECTRONIC DEVICE USING EYE MOTION}

The present invention relates to a remote control device of an electronic device using the movement of the eye, and in particular, to the electronic device such as a computer by using the eye movement and eye blink detected through an electrode embedded in the skin contact portion of the spectacle frame. It relates to a remote control device of the adjustable electronic device.

In general, the eye has a '+' potential toward the pupil, and thus the bioelectric potential is induced by the movement of the eye. The measurement of the biopotential is called EOG (electrooculograph).

As described above, there are many prior arts using EOG. Among them, representative technologies include US Patent No. 5726916 and US Patent No. 06360971.

First, US Pat. No. 5726916 is for attaching an electrode to glasses having a structure such as a water goggles, and for measuring the movement of the eye only through these electrodes.

Next, US Patent No. 06360971 is intended to enable a person to communicate with external devices using EOG signals generated by eye movement or to communicate with a counterpart.

However, in the prior art using the EOG signal as described above, it is not only difficult to attach the electrodes to glasses or the like, because it takes too many electrodes, and even if the electrodes are attached, there is a considerable amount of human movement due to the wires connected to the electrodes. Constraints follow, resulting in difficulties in practical use.

Accordingly, the present invention has been made to solve the above problems, and attaches a minimum number of electrodes to a pair of glasses worn by a person, and detects a biopotential generated by eye movement through the electrodes to generate a radio frequency control signal. This allows the physically handicapped to easily control electronic devices such as a computer, and also embeds the wiring of electrodes into the metal frame of the glasses to minimize noise while eliminating the inconvenience of wiring when wearing glasses. Its purpose is to provide a remote control device for an electronic device.

1 is a configuration diagram of an embodiment of a remote control device for an electronic device using the movement of the eye according to the present invention.

FIG. 2 is a diagram illustrating an embodiment of the reference signal generator of FIG. 1. FIG.

3 is a diagram illustrating an embodiment of an eye movement discriminating unit of FIG. 1;

FIG. 4 is a characteristic diagram of a subtracted value indicating a direction of an eye among subtracted values of the subtractors of FIG. 3. FIG.

FIG. 5 is a diagram illustrating an embodiment of the first amplifier and the bandpass filtering unit of FIG. 1. FIG.

Explanation of symbols on the main parts of the drawings

111 to 114: potential sensing electrode 115: reference electrode

120: reference signal generator 130: eye movement discriminating unit

140 and 162: first and second amplifiers 150: bandpass filtering unit

161: filtering unit 163: A / D conversion unit

164: control unit 165: radio frequency transmission unit

166: reverse current insulation

The present invention for achieving the above object, in the remote control device of the electronic device using the movement of the eye, in order to detect the biopotential generated by the movement of the eye, attached to the glasses so as to contact the body of the person First to fourth biopotential sensing means; By using the biopotentials sensed by the first to fourth biopotential sensing means and a predetermined weighting coefficient, a reference signal indicating the size of the vertical motion of the eye is calculated to detect the vertical motion of the eye directly above the eye. Reference signal generating means for outputting the reference signal; Distinguish eye movement using the biopotentials detected by the first to fourth biopotential sensing means and the reference signal transmitted, and output a signal indicating the eye movement, the moving direction, and the magnitude of the eye. Eye movement discrimination means; Eye motion signal processing means for amplifying and filtering output signals of the eye motion discrimination means, and then amplifying the separated eye motion signal and eye blink signal and converting the amplified signals into digital signals and outputting the digital signals; And control means for receiving signals from the eye movement signal processing means for eye movements, movement directions, and movement magnitudes, and outputting control signals for controlling the operation of the electronic device.

Hereinafter, with reference to the accompanying drawings will be described in detail a preferred embodiment of the present invention.

1 is a configuration of an embodiment of a remote control device for an electronic device using the movement of the eye according to the present invention.

As shown in Figure 1, the remote control device of the electronic device of the present invention, in order to detect the biopotential generated by the movement of the eye, a biopotential sensing electrode attached to the glasses so as to contact the body of the person ( 111 to 114, the reference potential 115 attached to the glasses to match the ground voltage of the human body and the ground voltage of the remote control device, and the biopotential detected by the biopotential sensing electrodes 111 to 114. A reference waveform that outputs a reference waveform Vref that detects the vertical movement of the eye directly above the eye by calculating the reference waveform Vref indicating the magnitude of the vertical movement of the eye using (v1, v2, v3, v4) and a preset weighting coefficient. Eye using the signal generator 120, the biopotential (v1, v2, v3, v4) detected by the biopotential sensing electrodes (111 to 114) and the reference waveform transmitted from the reference signal generator 120 To distinguish the movement of An eye motion discrimination unit 130 for outputting a signal indicating a distinct eye movement, a moving direction, and a motion magnitude, a first amplifier 140 for amplifying the output signals of the eye motion discrimination unit 130, and 1 band pass filtering unit 150 to remove noise by separating the high-frequency eye blink signal and the low-frequency eye movement signal (EOG) of the signals for the eye movement amplified by the amplification unit 140 and remove noise A second amplifier comprising a filtering unit 161 for re-filtering the signals transmitted from the filtering unit 150 to remove noise, and a plurality of amplifiers for amplifying the signals output from the filtering unit 161, respectively. 162, an A / D converter 163 composed of a plurality of A / D converters for converting analog signals transmitted from the second amplifier 162 into digital signals, and an A / D converter 163, respectively. Eye movements transmitted from The controller 164 outputs a control signal for controlling the operation of an electronic device such as a computer by determining the direction and the movement of the eye by applying signals about the direction and the motion to a predetermined algorithm, and from the controller 164. It is provided with a radio frequency transmitter 165 for converting the transmitted serial control signal into a radio frequency control signal for transmission.

In addition, the present invention further includes a reverse current insulation unit 166 connected between the first amplifier 162 and the A / D converter 163 to prevent an overcurrent from flowing into the human body.

Here, since the biopotential sensing electrodes 111 to 114 can detect the biopotential as the contact with the face of a person becomes better, it is preferable to arbitrarily move the position attached to the glasses. That is, since the size and shape of the face are different from each other, it is possible to move the biopotential sensing electrodes 111 to 114 fluidly according to the face of the person so as to make good contact with the face.

The configuration and operation of the remote control device for an electronic device of the present invention having the structure as described above will be described in more detail with reference to FIGS. 2 to 5 as follows.

The biopotential detecting electrodes 111 and 112 are attached to the right eyeglass leg portion so as to contact the right side of the human face and the left eyeglass leg so as to contact the left side of the human face, respectively. The potential is sensed and transmitted to the reference signal generator 120 and the eye movement discriminator 130.

The biopotential sensing electrodes 113 and 114 are attached to the left nose support of the glasses so that the right nose support of the glasses and the left nose of the person are well contacted, respectively, so that the contact with the right nose of the person is good. It detects the biopotential generated during the transmission and transmits it to the reference signal generator 120 and the eye movement discriminator 130.

On the other hand, the reference electrode 115 is attached to the portion of the eyeglasses to the human ear, it is used to match the ground voltage of the human body and the ground voltage of the remote control device of the present invention.

Here, the electrodes 111 to 115 are made of a silver Ag plate, which is a metal, and then plated with silver chloride AgCl.

In addition, the electrical wiring connected to the electrodes 111 to 115 embeds a shield line along the frame of the glasses, and the frame of the glasses is made of metal so that the remote control device of the present invention is grounded. The noise can be reduced as much as possible.

FIG. 2 is a diagram illustrating an exemplary configuration of the reference signal generator of FIG. 1, a multiplier 121 for multiplying a preset weighting factor w1 and a biopotential v1 detected by the biopotential sensing electrode 111 and the weighting factor. A multiplier 122 for multiplying the coefficient w2 and the biopotential v2 detected by the biopotential sensing electrode 112, and for multiplying the weighting factor w3 and the biopotential v3 sensed by the biopotential sensing electrode 113 A reference added by adding a multiplier 123, a multiplier 124 for multiplying the weighting factor w 4 and the biopotential v4 sensed by the biopotential sensing electrode 114, and a multiplier of the multipliers 121 to 124; The adder 125 outputs the waveform Vref.

Here, the weighting coefficients w1 to w4 are weights calculated experimentally according to the patient's situation (ie, electrode position and eye movement) by performing a simulation using data obtained from the patient as information for determining the reference waveform Vref. to be.

Thus, the reason for calculating the weighting factors (w1 to w4) through computer simulation is to obtain the same effect as measuring the vertical movement of the eye directly above and below the eye without attaching electrodes to the upper and lower parts of the eye. It is for.

On the other hand, since the weighting coefficient may vary slightly from person to person, an algorithm that optimizes using data in calculations calculates such weights that the eye's up and down motion signals can be seen best in the patient's situation.

As described above, the reference waveform Vref calculated by using the calculated weighting coefficient in the calculation process as shown in FIG. 2 is a waveform as measured by the vertical movement of the eye immediately above or below the eye.

FIG. 3 is a diagram illustrating an embodiment of the eye movement discriminating unit of FIG. 1, including a buffer 131 for buffering the biopotential v1 detected by the biopotential sensing electrode 111, and a biopotential sensing electrode 112. A buffer 132 for buffering the biopotential v2 detected by the buffer, a buffer 133 for buffering the biopotential v3 detected by the biopotential sensing electrode 113, and a biopotential sensing electrode 114. A buffer 134 for buffering the biopotential v4, a buffer 135 for buffering the reference waveform Vref transferred from the reference signal generator 120, and a buffer in the biopotential v1 transferred through the buffer 131 A subtractor 136 for subtracting the biopotential v4 delivered through 134 and a subtractor 137 for subtracting the biopotential v3 delivered through the buffer 133 from the biopotential v2 delivered through the buffer 132. ) And delivered through the buffer 132 at the biopotential v1 delivered through the buffer 131. It comprises a subtractor 138, a buffer 133, a subtracter 139 for subtracting the reference wave Vref is passed through the transfer buffer 135 in the living body through the electric potential v3 for subtracting the above Sports Festival v2.

The subtractors 136 and 137 are for easily identifying the eye movement direction, and the subtraction values are signals indicating the eye movement direction.

That is, FIG. 4 is a characteristic diagram of subtraction values of the subtractors 136 and 137.

The subtractor 138 is for easily identifying the size of the left and right movements of the eye, and the subtraction value is a signal indicating the size of the left and right movements of the eye.

The subtractor 139 is for easily confirming the size of the vertical movement of the eye, and the subtraction value is a signal indicating the size of the vertical movement of the eye.

On the other hand, when the reference signal generator 120 estimates the reference waveform at a position where the eye's up-and-down motion signal can be easily seen, the subtractor 139 detects the biopotential detected through the reference waveform and the electrode properly arranged. Calculate the size of the eye's up and down movement using v3.

FIG. 5 is a diagram illustrating an embodiment of the first amplifier and the bandpass filtering unit of FIG. 1.

As illustrated in FIG. 5, the first amplifier of FIG. 1 may amplify the subtracted value of the subtractor 136 to adjust the subtracted value of the amplifier 141 and the subtractor 137 so that the eye movement can be clearly identified. An amplifier 142 which amplifies the subtraction value of the subtractor 138 by amplifying the subtractor 138 so as to clearly check the movement of the eye, and the subtractor 143 which subtracts the subtractor 139 from the subtractor 139. It is composed of an amplifier 144 to amplify the value so that the size of the eye movement up and down clearly.

The bandpass filtering unit of FIG. 1 filters the output signals of the amplifiers 141 to 144 to remove noise and outputs a low frequency band filter that outputs EOG (that is, an eye motion signal) that is a low frequency band of about 0 to 15 Hz. 151 and a high frequency band filter 152 that filters the output signal of the amplifier 144 to remove noise and outputs a blink signal that is a high frequency band of about 20 to 50 Hz.

That is, since the eye movement is very slow compared to the eye blinking operation, the eye motion detection signal has a low frequency band of about 0 to 15 Hz, and on the contrary, the eye blinking is very fast, and thus has a high frequency band of about 20 to 50 Hz.

The filtering unit 161 cancels the power supply noise of 60 Hz, which is the main cause of the noise, by using a phase locked loop and a switched capacitor filter.

The controller 164 determines the movement direction and movement of the eye by using the eye motion signal transmitted through the above-described process, and controls the electronic device such as a computer according to the determination result.

In detail, the controller 164 may digitally filter the transmitted eye movement signal and calculate an eye movement direction and a magnitude of displacement of the eye using an algorithm using fuzzy logic, wherein each eye The size of the moving direction is divided into about three stages according to the displacement of the eye to determine in which direction on the monitor how much, and according to the determination outputs a control signal for controlling an electronic device such as a computer.

The eye blink is distinguished from the waveform of the upper and lower electrodes and used as a signal to control the mouse click of the computer. The waveform of the eye blink and the vertical motion signal are similar. I use it.

As such, the mouse of the computer is emulated using the distinct eye movement signal.

It also sends the protocol of the signal sent by the mouse to the personal computer in order to use the mouse driver that is common to Widow. The beauty of this method is that a spectacle wireless mouse can be used in a typical operating system without any adaptation program.

On the other hand, the control unit 164 outputs all control information to the RF transmitter 165 in the RS232 serial transmission protocol.

Accordingly, the radio frequency transmitter 165 wirelessly transmits the serial signal transmitted from the controller 164 at a bit rate of 38400 bps and 9600 bps at a radio frequency. Here, when emulating a mouse, a wireless transmission is performed at a bit rate of 9600bps, and when a EOG waveform is transmitted, a 38400 bps or 115200 bps wireless transmission is performed.

Although the technical spirit of the present invention has been described in detail according to the above preferred embodiment, it should be noted that the above embodiment is for the purpose of description and not of limitation. In addition, those skilled in the art will understand that various embodiments are possible within the scope of the technical idea of the present invention.

As described above, the present invention detects the vertical movement of the eyes and the left and right movements of the eyes and blinks of the eyes through the minimum electrodes attached to the glasses, and controls the electronic devices such as computers according to the detected biopotential. You can get it.

First, the handicapped person with a limb or upper limb can use the present invention as an adjusting device such as a mouse of a computer, so that the computer can be conveniently operated.

Second, people with disabilities may use the present invention as a device for conveniently controlling electronic devices such as televisions and lights other than computers.

Third, the device may be used as a device for manipulating a character or changing a screen background using a direction of the eye as control information in a virtual reality or a computer game, such as a joy stick.

Fourth, since the movement of the eye can be known during the speed method or the movement of the eye, the position of the eye can be fed back when implementing the speed reading or eye training program.

Fifth, it is possible to prevent traffic accidents by using the driver to detect drowsy driving when driving a car.

Sixth, if the blinking and eye movements detected through the present invention is encoded and executed in a speech synthesis program, communication can be performed even in the case of a disabled person who is uncomfortable or dumb.

Seventh, it is possible to obtain the waveform of the EOG used in the eye movement test in the hospital.

Eighth, it is possible to feed back the direction of the eye in the plane simulation and the like to enable more realistic simulation.

Ninth, if you connect the camera to something like filming, you can shoot realistically as your eyes move.

Tenth, since a change in the movement of the eye occurs when the driver sleeps while driving, it may be used to prevent traffic accidents by detecting the movement of the eye that changes during the drowsy driving.

Claims (12)

In the remote control device of the electronic device using the movement of the eyes, First to fourth biopotential sensing means attached to the glasses to be in contact with a human body so as to detect a biopotential generated by the movement of the eye; By using the biopotentials sensed by the first to fourth biopotential sensing means and a predetermined weighting coefficient, a reference signal indicating the size of the vertical motion of the eye is calculated to detect the vertical motion of the eye directly above the eye. Reference signal generating means for outputting the reference signal; Distinguish eye movement using the biopotentials detected by the first to fourth biopotential sensing means and the reference signal transmitted, and output a signal indicating the eye movement, the moving direction, and the magnitude of the eye. Eye movement discrimination means; After amplifying and filtering the output signals of the eye motion discrimination means, the eye motion signal and the eye blink signal are separated, and then the eye motion signal is converted into a digital signal by amplifying the separated eye motion signal and the eye blink signal and outputting the digital signal. Signal processing means; And Control means for receiving signals from the eye movement signal processing means for the movement of the eye, the direction of movement and the magnitude of the movement to output a control signal for controlling the operation of the electronic device Remote control device of the electronic device using the movement of the eye, including. The method of claim 1, Reference electrode attached to the glasses to match the ground voltage of the human body and the ground voltage of the remote control device Remote control device of the electronic device using the movement of the eye made further comprising. The method of claim 1, The first biopotential detecting means, In order to detect the biopotential generated when the left and right movement of the eye, the remote control device of the electronic device using the movement of the eye, characterized in that the electrode attached to the right leg portion so as to make good contact with the right side of the human face. The method of claim 1, The second biopotential detecting means, Remote control device of the electronic device using the movement of the eye, characterized in that the electrode attached to the left eyeglasses so as to contact the left side of the human face in order to detect the biopotential generated when the left and right movement of the eye. The method of claim 1, The third biopotential detecting means, In order to detect the biopotential generated when the pupil moves up and down, the remote control device of the electronic device using the movement of the eye, characterized in that the electrode attached to the right nose support of the glasses so that the contact with the right nose of the person well. The method of claim 1, The fourth biopotential detecting means, In order to detect the biopotential generated during the vertical movement of the eye, the remote control device of the electronic device using the movement of the eye, characterized in that the electrode attached to the left nose support of the glasses so that the contact with the left nose of the person well. The method of claim 1, The reference signal generating means, First multiplication means for multiplying the predetermined first weighting factor and the biopotential detected by the first biopotential sensing means; Second multiplication means for multiplying the predetermined second weighting factor with the biopotential detected by the second biopotential sensing means; Third multiplication means for multiplying the predetermined third weighting factor by the biopotential sensed by the third biopotential sensing means; Fourth multiplication means for multiplying the predetermined fourth weighting factor with the biopotential detected by the fourth biopotential sensing means; And Addition means for outputting the reference signal added by adding multiplication values of the first to fourth multiplication means to the eye movement discriminating means; Remote control device of the electronic device using the movement of the eye, including. The method of claim 1, The eye movement discrimination means, First buffering means for buffering the first biopotential sensed by the first biopotential sensing means; Second buffering means for buffering a second biopotential detected by the second biopotential sensing means; Third buffering means for buffering the third biopotential sensed by the third biopotential sensing means; Fourth buffering means for buffering a fourth biopotential sensed by the fourth biopotential sensing means; Fifth buffering means for buffering the reference signal; First subtracting means for subtracting a fourth biopotential delivered through the fourth buffering means from a first biopotential delivered through the first buffering means; Second subtracting means for subtracting a third biopotential transferred through the third buffering means from a second biopotential delivered through the second buffering means; Third subtracting means for subtracting a second biopotential delivered through said second buffering means from a first biopotential delivered through said first buffering means; And Fourth subtracting means for subtracting the reference signal transmitted through the fifth buffering means from the third biopotential transmitted through the third buffering means Remote control device of the electronic device using the movement of the eye, including. The method of claim 8, The subtraction values of the first and second subtraction means are signals representing the eye movement direction, The subtracted value of the third subtracting means is a signal indicating the size of the left and right movements of the eye, The subtraction value of the fourth subtraction means is a remote control device for electronic devices using eye movements, characterized in that the signal indicating the size of the eye movement. The method of claim 1, The eye movement signal processing means, First amplifying means for amplifying output signals of the eye movement discriminating means; First filtering means for removing noise by separating and filtering an eye blink signal, which is a high frequency band, and an eye motion signal, which is a low frequency band, from among the signals for the eye movement amplified by the first amplifying means; Second filtering means for filtering out the signals transmitted from the first filtering means to remove noise; Second amplifying means for amplifying the signals output from the second filtering means; A / D conversion means for converting the analog signals transmitted from the second amplification means into a digital signal; And Reverse current insulating means connected between the second amplifying means and the A / D conversion means to insulate that overcurrent flows to the human body. Remote control device of the electronic device using the movement of the eye, including. The method according to any one of claims 1 to 10, The control means, A control unit which receives signals about eye movement, movement direction, and movement magnitude transmitted to the eye movement signal processing unit and outputs a control signal for controlling the operation of the electronic device; And A radio frequency transmitter converting the serial control signal transmitted from the controller into a radio frequency control signal and transmitting the same Remote control device of the electronic device using the movement of the eye, including. The method of claim 11, The first to fourth biopotential sensing means, respectively, is not fixed to the glasses attached, the remote control device of the electronic device using the movement of the eye, characterized in that it can move the attached position arbitrarily.