KR101580397B1 - Apparatus and method for controlling device using eye movement - Google Patents

Abstract

The present invention relates to a control apparatus for a device using eye movement, comprising: a sensing device for acquiring eye movement information including information on blinking of a user's eye and information on movement of the eyeball; A storage unit in which a control command database is stored in advance; And information on the flicker of the eye and information on the motion of the eyeball from the eyeball operation information obtained through the sensing device, Information on the blinking of the filtered eye and control command information corresponding to the blinking information of the filtered eye and information on the movement of the eye is extracted from the control command database And transmitting the extracted control command information to the device, and controlling the driving unit of the device to operate according to the control command information.

Description

[0001] APPARATUS AND METHOD FOR CONTROLLING DEVICE USING EYE MOVEMENT [0002]

Embodiments of the present invention relate to a control device of a device using eye movement, for example, a technique for controlling a device based on eye movement of a user.

A mouse is a type of computer input device. It is considered to be the most popular input device to date with the keyboard because of the relatively simple usage in which the cursor moves in the display screen when the mouse is moved and the command is executed when the button is clicked.

The mouse is divided into a ball mouse, an optical mouse, and a gyroscope mouse (air mouse) depending on the driving method.

However, in the case of a conventional mouse, it is difficult to use the input device or the two hands of the disabled person whose behavior is invariable as a user's input device used for controlling other input devices. Therefore, an eye mouse is being studied. Conventional eye mice have a problem in that they are expensive to use an eye mouse because of the use of a three-dimensional control method using four cameras or a three-dimensional control method using GPS, and they are difficult to popularize and popularize because they are expensive.

KR 1020070043469 A

Embodiments of the present invention provide an apparatus and method for controlling a device using eye movement capable of recognizing a user's eye movement and providing a function of controlling movement of an actual device in accordance with recognized eye movement.

Embodiments of the present invention also provide an apparatus and method for controlling a device using an eye movement capable of discriminating a user's eye movement for controlling an actual device from a natural eye movement of a user.

A method of controlling a device using an eye movement according to an embodiment of the present invention is a method of controlling a device using an eye movement, the method comprising the steps of: acquiring information about a flicker of a user's eye, Acquiring information; Extracting information on blinking of the eye and information on motion of the eyeball from the acquired eyeball operation information; Calculating filtered flicker information for controlling the device by filtering the flicker information of the extracted eye based on preset natural flicker information of the eye; Extracting control command information corresponding to information on the motion of the extracted eyeball and blinking information of the filtered eye from the control command database stored in advance in the storage unit; And transmitting the extracted control command information to the device, and controlling the driving unit to operate in accordance with the control command information.

The method of controlling a device using the eye movement may further include: obtaining the blinking information of the natural eye for a set time previously stored in the storage unit, before acquiring the eye action information; Storing the acquired natural eye flicker information in the storage unit; It is determined whether information on eye movement including information on flickering of the eye including information on an instruction to start acquiring the eye movement information or information on an instruction to start acquiring the eye movement information is obtained step; And information on eye movement including information about an eye blinking including information about an instruction to start acquiring eye movement information or information about an instruction to start acquiring eye movement information as a result of the determination And if so, moving to acquiring the eye movement information.

Wherein the step of calculating the blinking information of the filtered eye comprises the step of obtaining information on blinking of the eye including information on an instruction to start acquiring the ocular action information or information on an instruction to start acquiring the ocular action information It is possible to calculate the flicker information of the filtered eye on the blink information of the eye extracted from the obtained eye action information from the time when information on the motion of the eye is obtained.

A method of controlling a device using the eye movement includes generating a screen including a mouse pointer for displaying the eye movement information and displaying the generated screen through a display unit; And controlling an operation of the mouse pointer in accordance with information on motion of the eyeball extracted from the eyeball action information and blinking information of the filtered eyeball. As shown in FIG.

The information on the movement of the eyeball may include at least one of information on the position of the user's pupil, information on the moving direction of the pupil, and information on the moving speed of the pupil.

The blinking information of the filtered eye may include at least one of information on the number of blinks of the user's eye and information on the blinking speed of the eye within the set time set in the storage unit.

The control command database includes information on a position of the user's pupil, information on a moving direction of the pupil, information on a moving speed of the pupil, information on the number of blinks of the user's eyes, And a plurality of control command information generated based on at least two pieces of information on the control command information.

A control device of a device using an eye movement according to an embodiment of the present invention is a control device of a device using an eye movement. The control device includes an eye movement A sensing device for acquiring information; A storage unit in which a control command database is stored in advance; A transfer unit for transferring data to the device; And information on the flicker of the eye and information on the motion of the eyeball from the eyeball operation information obtained through the sensing device, Information on the blinking of the filtered eye and control command information corresponding to the blinking information of the filtered eye and information on the movement of the eye is extracted from the control command database And a control unit transmitting the extracted control command information to the device through the transmission unit and controlling the driving unit included in the device to operate according to the control command information.

Wherein the control unit acquires the flicker information of the natural eye for a set time preset in the storage unit before acquiring the eye action information through the sensing unit, Storing information in the storage unit and information about an eye flicker including information about an instruction to start acquiring the eye movement information or information about an instruction to start acquiring the eye movement information Information on an eye flicker including information on an instruction to start acquiring eye movement information, or information on an instruction to start acquiring the eye movement information, as a result of the determination When information on the motion of the eyeball is acquired, More can be done.

Wherein the blinking information of the filtered eye includes information about flickering of the eye including information about an instruction to start acquiring the eye movement information or information about an instruction to start acquiring the eye movement information Eye blinking information extracted from the eyeball action information obtained from the time when the information about the motion is obtained can be calculated from the object.

Wherein the control unit generates a screen including a mouse pointer for displaying the eye movement information, displays the generated screen through the display unit, and displays information about motion of the eyeball extracted from the eye movement information, And controlling the operation of the mouse pointer in response to the blinking information of the blinking eye.

The information on the movement of the eyeball may include at least one of information on the position of the user's pupil, information on the moving direction of the pupil, and information on the moving speed of the pupil.

The blinking information of the filtered eye may include at least one of information on the number of blinks of the user's eye and information on the blinking speed of the eye within the set time set in the storage unit.

The control command database includes information on a position of the user's pupil, information on a moving direction of the pupil, information on a moving speed of the pupil, information on the number of blinks of the user's eyes, And a plurality of control command information generated based on at least two pieces of information on the control command information.

According to the embodiment of the present invention, it is possible to recognize a user's eye movement through a single camera without using a plurality of cameras or GPS signals, and to control the movement of the actual device according to the recognized eye movement It is possible to provide an apparatus and method for controlling a device using eye movement.

In addition, according to the embodiment of the present invention, it is possible to distinguish the eye movement of the user for controlling the actual device from the natural eye movement of the user, so that the eye movement capable of performing a function capable of preventing malfunction in device control It is possible to provide a control apparatus and method for a device to be used.

1 is a block diagram showing the operation of a control apparatus of a device using eye movement according to an embodiment of the present invention;
2 is a flow chart illustrating a method of controlling a device using eye movement according to an embodiment of the present invention.

Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the contents described in the accompanying drawings. However, the present invention is not limited to or limited by the exemplary embodiments. Like reference numerals in the drawings denote members performing substantially the same function.

Terms including ordinals, such as first, second, etc., may be used to describe various elements, but the elements are not limited to these terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. Also, in certain cases, there may be a term selected arbitrarily by the applicant, in which case the meaning thereof will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term, not on the name of a simple term, but on the entire contents of the present invention.

When an element is referred to as "including" an element throughout the specification, it is to be understood that the element may include other elements as well, without departing from the spirit or scope of the present invention. Also, the terms "part," " module, "and the like described in the specification mean units for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software .

The storage unit according to an embodiment of the present invention may include all kinds of recording media in which programs and data are stored so that the storage medium can be read by a computer system. For example, a punch card, a paper tape, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a flash-EPROM, Read only memory (ROM), a random access memory (RAM), a compact disk (CD), a compact disk read-only memory (CD-ROM), a digital video disk (DVD) , A flexible disk, an optical data storage device, a hard disk drive (HDD), a solid state drive (SSD), a solid state hybrid drive (SSHD), and an embedded multimedia card (eMMC) . And may also be implemented in the form of a carrier wave (e.g., transmission over the Internet).

First, the following terms will be described in order to facilitate understanding of an apparatus and method for controlling a device using eye movement according to an embodiment of the present invention.

The sensing device according to an embodiment of the present invention can acquire eye movement information of a user. The eye movement information may include information on the flicker of the user's eye and information on the movement of the eyeball.

In addition, the sensing device according to the embodiment of the present invention may include an infrared LED and a camera.

The control device of the device using the eye movement according to the embodiment of the present invention is a device for controlling the device by filtering based on the blinking information of the natural eye pre- It is possible to calculate the flicker information.

The blinking information of the filtered eye according to the embodiment of the present invention may include at least one of information on the number of blinks of the user's eye and information on the blinking speed of the eye within the preset time stored in the storage unit.

Also, the blinking information of the filtered eye according to the embodiment of the present invention may include information on flickering of the eye including information on the start of the eye movement information, or movement of the eyeball including information on the start of the eye movement information Eye blinking information extracted from the received eye movement information from the time when the information is received.

The information on the motion of the eyeball according to the embodiment of the present invention may include at least one of information on the position of the user's pupil, information on the moving direction of the pupil, and information on the moving speed of the pupil.

The blinking information of the natural eye according to the embodiment of the present invention can acquire the blinking information of the natural eye of the user through the sensing device for a set time preset in the storage unit.

The storage unit according to an exemplary embodiment of the present invention may store preset time, natural blinking information, and a control command database in advance.

The control command information according to an embodiment of the present invention corresponds to information on eye movement and blinking information of the filtered eye, and indicates information that can control the operation of the driving unit of the device.

The control command database according to an exemplary embodiment of the present invention includes information on the position of the user's pupil, information on the direction of the pupil's moving direction, information on the moving speed of the pupil, information on the number of blinks of the user's eyes, And a plurality of pieces of control command information generated based on at least two pieces of information related to the control command information.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an apparatus for controlling safe operation of a vehicle using an EEG signal according to an embodiment of the present invention will be described with reference to the accompanying drawings.

FIG. 1 is a block diagram illustrating the operation of a control apparatus for a device using eye movement according to an embodiment of the present invention, in accordance with an embodiment of the present invention. Referring to FIG. 1, a control device 100 of a device using eye movement according to an embodiment of the present invention includes a sensing device 110, a storage unit 120, a transmission unit 130, and a control unit 140 can do.

The sensing device 110 may acquire eye movement information of the user under the control of the controller 140. The sensing device 110 may transmit the acquired ocular action information to the control unit 140. The eye movement information may include information on the flicker of the user's eye and information on the movement of the eyeball.

The storage unit 120 may store the set time, the natural blinking information, and the command database in advance according to the control of the controller 140. The control command database includes at least two of information on the position of the user's pupil, information on the moving direction of the pupil, information on the moving speed of the pupil, information on the number of blinks of the user's eyes, And may include a plurality of control command information generated based on the information. The control command information corresponds to the information on the movement of the eyeball and the blinking information of the filtered eye, and represents information that can control the operation of a driving unit (not shown) of a device (not shown).

The transmission unit 130 may transmit data to the device under the control of the control unit 140. [ For example, the transmission unit 130 may transmit control command information to the device under the control of the control unit 140. [

The control unit 140 may control the operation of the sensing device 110, the storage unit 120, and the transmission unit 130.

In addition, the control unit 140 may acquire natural eye blinking information through the sensing device 110 for a set time preset in the storage unit 120. [ The natural eye flicker information is information obtained from the flicker of the eye except the eye flicker for the user to control the device.

Thereafter, the control unit 140 may store the acquired natural eye flicker information in the storage unit 120. FIG.

Thereafter, the control unit 140 transmits information about the flicker of the eye including the information about the instruction to start acquisition of the eye movement information through the sensing apparatus 110, or information about an instruction to start acquiring the eye movement information It can be determined whether or not the information on the movement of the eyeball is acquired.

As a result of the determination, information on eye movement including information on an eye blinking including information on an instruction to start acquiring eye movement information or information on an instruction to start acquiring eye movement information is obtained The controller 140 may acquire the eye movement information of the user through the sensing device 100. FIG.

Thereafter, the control unit 140 can extract information on blinking of the eye and information on the motion of the eyeball from the acquired eye movement information, respectively.

Thereafter, the control unit 140 filters the extracted eye flicker information based on the natural eye flicker information stored in advance in the storage unit 120, and controls the blinking of the filtered eye for controlling the device (not shown) Information can be calculated.

In addition, the control unit 140 may include information on eye flickering including information on an instruction to start acquiring eye movement information, or information on eye movement including information on an instruction to start acquiring eye movement information Eye blinking information extracted from the eye movement information obtained through the sensing device 110 can be calculated based on the blinking information of the eyes.

Thereafter, the control unit 140 may extract control command information corresponding to the blinking information of the filtered eye and the eye movement information from the control command database stored in the storage unit 120 in advance.

Thereafter, the control unit 140 transmits the extracted control command information to the device (not shown) through the transfer unit 130 so that a driving unit (not shown) included in the device (not shown) .

In addition, the control unit 140 may generate a screen including a mouse pointer for displaying eye movement information, and display the generated screen through a display unit (not shown). Thereafter, the controller 140 may control the operation of the mouse pointer according to the information on the movement of the eyeball extracted from the eyeball operation information and the blinking information of the filtered eye.

On the other hand, the respective components of the control device 100 of the device using the eye movement are separately shown in the drawings to show that they can be functionally and logically separated, and are physically separate components, And the like.

In this specification, each function means a functional and structural combination of hardware for carrying out the technical idea of the present invention and software for driving the hardware. For example, each functional unit may refer to a logical unit of a predetermined code and a hardware resource for executing the predetermined code, and may be a code physically connected to the functional unit, But can be easily deduced to the average expert in the field of the invention.

Hereinafter, a method of controlling a device using eye movement according to an embodiment of the present invention will be described with reference to the accompanying drawings.

2 is a flowchart showing a control method of a device using eye movement according to an embodiment of the present invention. Referring to FIG. 2, the control unit 140 of the control system of the device using the eye movement according to the exemplary embodiment of the present invention controls the user's natural Eye flicker information (201).

Thereafter, the control unit 140 may store the acquired natural eye flicker information in the storage unit 120 (202).

Thereafter, the control unit 140 transmits information about the flicker of the eye including the information about the instruction to start acquisition of the eye movement information through the sensing apparatus 110, or information about an instruction to start acquiring the eye movement information (203) whether information about the motion of the involved eye is obtained.

As a result of the determination, information on eye movement including information on an eye blinking including information on an instruction to start acquiring eye movement information or information on an instruction to start acquiring eye movement information is obtained The controller 140 may acquire eye movement information of the user through the sensing device 100 (204). On the other hand, the control unit 140 may include information on eye flickering, including information on an instruction to start acquiring eye movement information, or information about movement of the eyeball, including information about an instruction to start acquiring eye movement information It is possible to move to step 201 and perform the corresponding step.

Thereafter, the controller 140 may extract 205 information about eye flicker and eyeball motion from the acquired eye motion information, respectively.

Thereafter, the control unit 140 filters the extracted eye flicker information based on the natural eye flicker information stored in advance in the storage unit 120, and controls the blinking of the filtered eye for controlling the device (not shown) Information (206).

In addition, the control unit 140 may include information on eye flickering including information on an instruction to start acquiring eye movement information, or information on eye movement including information on an instruction to start acquiring eye movement information Eye blinking information extracted from the eye movement information obtained through the sensing device 110 can be calculated based on the blinking information of the eyes.

Thereafter, the control unit 140 extracts control command information corresponding to the blinking information of the filtered eye and information about the motion of the eyes from the control command database stored in the storage unit 120 (207).

Thereafter, the control unit 140 transmits the extracted control command information to the device (not shown) through the transfer unit 130 so that a driving unit (not shown) included in the device (not shown) (208).

In addition, the control unit 140 may generate a screen including a mouse pointer for displaying eye movement information, and display the generated screen through a display unit (not shown).

Thereafter, the controller 140 may control the operation of the mouse pointer according to the information on the movement of the eyeball extracted from the eyeball operation information and the blinking information of the filtered eye.

The methods according to exemplary embodiments of the present invention may be implemented in the form of program instructions that may be executed through various computer means and recorded on a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be known and available to those skilled in the computer software field.

Although the present invention has been described with reference to the accompanying drawings and the foregoing embodiments, the present invention is not limited thereto but is limited by the following claims. Accordingly, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope of the present invention as defined by the appended claims.

100: Control device of a device using eye movement
110:
120:
130:
140:

Claims (14)

A method of controlling a device using eye movement,
Acquiring flicker information of a natural eye during preset time stored in a storage unit;
Storing the acquired natural eye flicker information in the storage unit;
Determining whether information on eye movement including information on eye blinking including information about an instruction to start acquiring eye movement information or information about an instruction to start acquiring eye movement information is obtained ;
As a result of the determination, information on eye movement including information on flickering of eyes including information on an instruction to start acquiring eye movement information or information on an instruction to start acquisition of eye movement information is acquired Obtaining eye movement information including information on blinking of a user's eye and information on movement of the eyeball;
Extracting information on blinking of the eye and information on motion of the eyeball from the acquired eyeball operation information;
Calculating filtered flicker information for controlling the device by filtering the flicker information of the extracted eye based on preset natural flicker information of the eye;
Extracting control command information corresponding to information on the motion of the extracted eyeball and blinking information of the filtered eye from the control command database stored in advance in the storage unit; And
Transmitting the extracted control command information to the device, and controlling the driving unit to operate in accordance with the control command information; And controlling the movement of the eyeball. delete 2. The method of claim 1, wherein calculating the blinking information of the filtered eye comprises:
From the time point when information on the movement of the eye including the information on the flicker of the eye including the information on the instruction to start acquisition of the eye movement information or the information on the instruction to start the acquisition of the eye movement information is obtained Wherein the blinking information of the filtered eye is calculated on the blinking information of the eye extracted from the eye movement information.
4. The method of controlling a device using eye movement according to claim 3,
Generating a screen including a mouse pointer for displaying the eye movement information, and displaying the generated screen through a display unit; And
Controlling an operation of the mouse pointer in accordance with information on motion of the eyeball extracted from the eyeball action information and flicker information of the filtered eyeball; Further comprising the steps of: determining whether the at least one eye movement is in the eye movement state;
5. The method according to claim 4,
Wherein the information includes at least one of information on a position of the user's pupil, information on a moving direction of the pupil, and information on a moving speed of the pupil.
6. The method of claim 5, wherein the blinking information of the filtered eye comprises:
Wherein the information includes at least one of information on the number of blinks of the user's eyes and information on blink speed of the eyes within a set time set in the storage unit. 7. The apparatus of claim 6, wherein the control command database comprises:
At least two of information on the position of the pupil of the user, information on the moving direction of the pupil, information on the moving speed of the pupil, information on the number of blinks of the user's eyes, and information on the blink rate of the eyes Wherein the control command information includes a plurality of control command information generated based on the plurality of pieces of control command information.
A control device of a device using eye movement,
A sensing device for acquiring eye movement information including information on blinking of a user's eye and information on movement of the eyeball;
A storage unit in which a control command database is stored in advance;
A transfer unit for transferring data to the device; And
Extracting information on the flicker of the eye and information on the motion of the eyeball from the eyeball operation information obtained through the sensing device, Extracting from the control command database control command information corresponding to the blinking information of the filtered eye and information about the movement of the eyeball, And a controller for transmitting the extracted control command information to the device through the transmission unit and controlling the driving unit included in the device to operate according to the control command information,
Wherein the control unit acquires the flicker information of the natural eye for a set time preset in the storage unit before acquiring the eye action information through the sensing unit, Storing information in the storage unit and information about an eye flicker including information about an instruction to start acquiring the eye movement information or information about an instruction to start acquiring the eye movement information Information on an eye flicker including information on an instruction to start acquiring eye movement information, or information on an instruction to start acquiring the eye movement information, as a result of the determination When the information on the motion of the eyeball is acquired, the eyeball action information Controller of the device using the eye movement further carried out to control to obtain.
delete 9. The method of claim 8, wherein the blinking information of the filtered eye comprises:
From the time when information on eye movement including information on flickering of eyes including information on an instruction to start acquisition of eye movement information or information on an instruction to start acquiring eye movement information is obtained And eye blinking information extracted from the obtained eye movement information is calculated from the object.
11. The apparatus according to claim 10,
A screen including a mouse pointer for displaying the eye movement information, displaying the generated screen through a display unit, information on movement of the eyeball extracted from the eye movement information, Further comprising controlling, in response to the information, the operation of the mouse pointer.
12. The method of claim 11,
And information on at least one of information on a position of the user's pupil, information on a moving direction of the pupil, and information on a moving speed of the pupil.
13. The method of claim 12, wherein the blinking information of the filtered eye comprises:
Wherein the information includes at least one of information on the number of blinks of the user's eyes and information on blink rate of the eye within the set time set in the storage unit.
14. The system of claim 13,
At least two of information on the position of the pupil of the user, information on the moving direction of the pupil, information on the moving speed of the pupil, information on the number of blinks of the user's eyes, Wherein the control instruction information includes a plurality of control instruction information items generated based on the information items.

Images