KR100650062B1 - Eyeball focus cognition device controlling portable device by using eyeball focus information and operating method of the device - Google Patents

Abstract

An eyeball focus recognition device for controlling a portable terminal by using eyeball focus information and an operating method thereof are provided to perform a certain input operation in the portable terminal without carrying the portable terminal. A local area communication module(210) performs local area communication with a portable terminal. A display device(220) outputs image data received from the portable terminal through the local area communication module(210). An eyeball focus recognizing module(230) recognizes an eyeball focus direction of a user who steadily gazes at the display device(220). An input information generating unit(240) converts the eyeball focus direction into information on the input of the portable terminal. The input information is transmitted to the portable terminal through the local area communication module(210).

Description

EYEBALL FOCUS COGNITION DEVICE CONTROLLING PORTABLE DEVICE BY USING EYEBALL FOCUS INFORMATION AND OPERATING METHOD OF THE DEVICE}

1 is a view showing an example of the operation of the portable terminal and the eye focus recognition device according to the present invention.

2 is a block diagram showing an internal configuration of an apparatus for recognizing eye focus according to the present invention.

3 is a diagram illustrating an example of checking a text message received by a communication terminal in an eye focus recognition apparatus according to the present invention.

4 is a flowchart illustrating a method of operating an eye focus recognition apparatus according to the present invention.

5 is an internal block diagram of a general-purpose computer device that may be employed to perform a method of operating the eye focus recognition device according to the present invention.

<Explanation of symbols for main parts of the drawings>

200: eye focus recognition device 210: short-range communication module

220: display means 230: eye focus recognition module

240: input information generation unit 250: determination unit

The present invention relates to an eye focus recognizing apparatus for performing near field communication with a portable terminal, and more particularly, to output a predetermined image information received from the portable terminal to a display means, and the user's eye focus direction corresponding to the image information. The present invention relates to an eye focus recognizing apparatus and a method of operating the apparatus, for converting the information into input information and transmitting the same to the portable terminal, wherein the portable terminal receives the input information and performs a predetermined operation.

Today, with the rapid development of information and communication, the types of portable terminals carried by individuals are diversified. Such portable terminals include mobile communication terminals, PSTN terminals, VoIP, SIP, Megaco, personal digital assistant (PDA), cellular phones, personal communication service (PCS) phones, hand-held PCs, and CDMA-2000 (1X). , 3X) phones, WCDMA (Wideband CDMA) phones, Dual Band / Dual Mode phones, Global Standard for Mobile (GSM) phones, Mobile Broadband System (MBS) phones, satellite / terrestrial DMB (Digital Multimedia) Broadcasting) This concept includes a phone, a PMP or an MP3 player, and provides various information to the user.

Such a portable terminal can be portable when the user moves due to miniaturization and light weight, and provides multimedia data such as movie, music, or image data to the user when necessary.

However, when a user carrying a portable terminal is placed at a short distance such as a bag or a desk, the user often cannot immediately read the information provided from the portable terminal. For example, when a communication terminal carried in a bag receives a text message during a specific task, the user should stop the specific task and take the communication terminal out of the bag to check the text message. In another example, the user acknowledges the text message after completing the particular task. However, such a method of checking a text message may suspend the specific task, thereby interrupting the specific task, and in case of later checking the text message, the user may be disadvantaged by checking late for important information. In addition, the user may not be aware of the fact that the text message has been received by not recognizing a vibration or a ring tone generated from the communication terminal.

As another example, when a user watches a movie through a portable multimedia player (PMP) on a bus, the user must carry the PMP in one hand and hold the handle of the bus with the other hand. In this case, the user must take the risk of an accident due to sudden stop of the bus by concentrating on the movie.

Therefore, there is an urgent need for a more advanced technical idea of reading image data generated from a portable terminal carried in an adjacent location and transmitting predetermined input information to the portable terminal.

The present invention has been made to improve the prior art as described above, and an object of the present invention is to provide convenience for reading predetermined image information generated from the portable terminal without carrying the portable terminal.

An object of the present invention is to provide a convenience of performing a predetermined input operation on the portable terminal without carrying the portable terminal.

Another object of the present invention is to provide a convenience of recognizing a user's eye focus direction and controlling the operation of a portable terminal.

In order to achieve the above object and to solve the above-described problems of the prior art, the apparatus for recognizing the eye focus according to the present invention is a short range communication module for performing short range communication with a portable terminal, and received from the portable terminal through the short range communication module. A display means for outputting image data, an eye focus recognition module for recognizing an eye focus direction of a user looking at the display means, and an input information generator for converting the eye focus direction into input information of the portable terminal; The input information is transmitted to the portable terminal through the short range communication module.

In addition, the operation method of the eye focus recognition apparatus according to the present invention comprises the steps of receiving a predetermined output information from the portable terminal through the short-range communication module, outputting the output information to the display means, recognize the eye focus of the user, Generating input information corresponding to a gaze direction of the eye focus, and transmitting the generated input information to the portable terminal through the short range communication module, wherein the portable terminal is connected to the short range communication module. And a predetermined processing operation according to the received input information.

Hereinafter, an eye focus recognition device and an operating method of the eye focus recognition device according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a view showing an example of the operation of the portable terminal and the eye focus recognition device according to the present invention.

As shown in Figure 1, the portable terminal 110 is a mobile communication terminal, PSTN terminal, VoIP, SIP, Megaco, PDA (Personal Digital Assistant), cellular phone, PCS (Personal Communication Service) phone, hand-held PC (Hand-) Held PC), CDMA-2000 (1X, 3X) phones, Wideband CDMA (WCDMA) phones, Dual Band / Dual Mode phones, Global Standard for Mobile (GSM) phones, Mobile Broadband System (MBS) All personal handheld devices such as phones, satellite / terrestrial digital multimedia broadcasting (DMB) phones, or MP3 players.

The portable terminal 110 may provide predetermined image data to a user through a display means and receive input information from the user to perform a control operation. 1 illustrates an eye focusing apparatus 100 according to the present invention through a mobile communication terminal as an example of a portable terminal.

The mobile terminal receives a text message from a messaging server. The mobile communication terminal and the eye focus recognition apparatus 100 according to the present invention maintain a short-range communication module such as a Bluetooth module, an infrared communication module, or a wired communication module, and the mobile communication terminal notifies the fact that the text message is received. It transmits to the eye focus recognition apparatus 100 through the short range communication module. When the short range communication module is the Bluetooth module, the mobile communication terminal may process the reception of the text message into a Bluetooth communication packet and transmit the received message to the eye focus recognition apparatus 100.

Accordingly, the eye focus recognition apparatus 100 receives the received facts through the short range communication module and outputs the received facts to the display means 101. The display means 101 according to the present invention may be implemented as output means capable of miniaturization and weight reduction, such as a liquid crystal display (LCD) or organic light emitting diodes (OLED).

When the user wants to transmit predetermined input information to the portable terminal 110 according to the reception fact output to the display means 101, the user moves the eyeball to generate the predetermined movement information. Accordingly, the eye focus recognition module of the eye focus recognizing apparatus 100 reads the eye movement information, switches to an operation mode for collecting the input information, and records the eye focus direction of the user. The recorded eye focus direction is converted into input information of the mobile communication terminal, and the converted input information is transmitted as input information of the mobile communication terminal through the short range communication module. The mobile communication terminal receiving the input information can process a predetermined operation. Recognizing eye movement is a well-known conventional technique, and thus the detailed description thereof will be omitted.

For example, after the user views the text message through the display means 101, moves the eye up and down twice to delete it, and then watches the 'delete' icon displayed on the user's right display means 101. . Accordingly, the eye focus recognition apparatus 100 detects the movement of the user's eye focus up and down twice, switches to a reception mode of input information to receive input information from the user, and the eye focus of the user is right. Recognize that you are watching. Accordingly, the eye focus recognition apparatus 100 generates input information for requesting deletion of a text message and transmits it to the mobile communication terminal.

As a result, the user may view predetermined image information generated from the portable terminal 110 without carrying the portable terminal 110. In addition, the user may perform a predetermined input operation on the portable terminal 110 without carrying the portable terminal 110.

2 is a block diagram showing an internal configuration of an apparatus for recognizing eye focus according to the present invention.

As shown in FIG. 2, the eye focus recognition apparatus 200 according to the present invention includes a short range communication module 210, a display unit 220, an eye focus recognition module 230, and an input information generator 240. .

The short range communication module 210 provides a short range communication capability with a portable terminal. To this end, the portable terminal maintains a short range communication module corresponding to the short range communication module 210. The short range communication module 210 according to the present invention may include a wired communication module such as a USB cable, as well as a wireless communication module such as a Bluetooth module and an infrared communication module.

The display unit 220 outputs image data received from the portable terminal through the short range communication module 210. To this end, the display means 220 includes an output means such as a liquid crystal display (LCD) or organic light emitting diodes (OLED). The image data may include multimedia data such as moving image data, image data, and text data.

The eye focus recognition module 230 recognizes the eye focus direction of the user who looks at the display means 220 as a means for recognizing the movement of the eye of the user. To this end, the eye focus recognition module 230 includes an iris recognition sensor that can recognize the iris.

The input information generator 240 converts the eye focus direction of the user recognized by the eye focus recognition module 230 into input information of the portable terminal. That is, when the user views the image data output to the display means 220 and requests a predetermined control from the portable terminal, the eye focus is moved, and the eye focus recognition module 230 adjusts the moving direction of the eye focus. Recognizes and generates input information for transmission to the portable terminal.

The eye focus recognition apparatus 200 according to an embodiment of the present invention further includes a determination unit 250. The determination unit 250 determines whether the movement of the eye focus of the user is for generating input information or for viewing the image data output to the display unit 220. That is, the user moves the eye focal point according to predetermined motion information set in advance, and when the movement of the eye focal point corresponding to the motion information is detected, the determination unit 250 activates the input information generator 240. As a result, from the time when the user moves the eye focus to correspond to the predetermined movement information, the eye focus recognition apparatus 200 generates input information corresponding to the eye focus direction of the user and transmits the generated input information to the portable terminal. .

The interworking of the components of FIG. 2 will be described in more detail with reference to the example of FIG. 3.

3 is a diagram illustrating an example of checking a text message received by a mobile communication terminal in an eye focus recognition apparatus according to the present invention.

Referring to FIG. 3, the mobile communication terminal receives a text message from a predetermined messaging server and transmits the received fact of the text message to the eye focus recognition apparatus. Accordingly, the apparatus for recognizing the eyeball outputs the fact that the text message is received on the display means, the "Yes 301" icon for confirming the text message on the display means on the left side of the user, and the text message on the right display means of the user. Further outputs a "no 302" icon to disable. Thus, the user watches "Yes 301" to browse the text message.

The eye focus recognition module 301 recognizes the eye focus of the user, and recognizes that the eye focus of the user watches the left direction. Accordingly, the input information generator generates input information for reading the text message and transmits the input information to the mobile communication terminal through the short range communication module.

The mobile communication terminal receiving the input information transmits the text message received from the messaging server to the eye focus recognition apparatus through a short range communication module, and the eye focus recognition apparatus receiving the text message transmits the text message. Is output as the image data to the display means.

3 illustrates an eye focus recognition apparatus for reading a text message received from a mobile communication terminal and transmitting predetermined input information to the mobile communication terminal, the eye focus recognition apparatus of the present invention can be variously modified.

4 is a flowchart illustrating a method of operating an eye focus recognition apparatus according to the present invention.

Referring to FIG. 4, in step 401, the portable terminal transmits output information including predetermined image data to the eye focus recognition apparatus through a short range communication module. The eye focal recognizing apparatus receives the image data from the portable terminal and outputs it via display means (step 402). When the user wants to perform a predetermined control on the image data, the user moves the focus of the eyeball according to the preset motion information. In response, the apparatus for recognizing eye focus detects the motion information according to the movement (step 403), determines that the user inputs the input information, and switches to the input information reception mode by activating the input information generator (step 404). )). Accordingly, the apparatus for recognizing the eye focus recognizes the moving direction of the eye focus of the user in step 405, and generates input information corresponding to the moving direction of the eye focus (step 406). For example, the user may move the cursor CURSOR of the portable terminal to the left by moving the eye focus from left to right. By applying this, the eye focus recognition apparatus according to the present invention can recognize the eye focus of the user to generate the input information of the portable terminal, thereby controlling the portable terminal.

5 is an internal block diagram of a general-purpose computer device that may be employed to perform a method of operating the eye focus recognition device according to the present invention.

Embodiments of the invention include a computer readable medium containing program instructions for performing various computer-implemented operations. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical recording media such as CD-ROMs, DVDs, magnetic-optical media such as floppy disks, and ROM, RAM, flash memory, and the like. Hardware devices specifically configured to store and execute the same program instructions are included. The medium may be a transmission medium such as an optical or metal wire, a waveguide, or the like including a carrier wave for transmitting a signal specifying a program command, a data structure, or the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

Computer device 500 includes one or more processors 510 connected to a main memory device including random access memory (RAM) 520 and read only memory (ROM) 530. The processor 510 is also called a central processing unit (CPU). As is well known in the art, the ROM 530 serves to transfer data and instructions to the CPU unidirectionally, and the RAM 520 typically transfers data and instructions bidirectionally. Used to. RAM 520 and ROM 530 may include any suitable form of computer readable media. Mass storage 540 is bidirectionally coupled to processor 510 to provide additional data storage capability and may be any of the computer readable recording media described above. The mass storage device 540 is used to store programs, data, and the like, and is a secondary memory device such as a hard disk which is generally slower than the main memory device. Certain mass storage devices such as CD ROM 560 may be used. The processor 510 may include one or more input / output interfaces, such as video monitors, trackballs, mice, keyboards, microphones, touchscreen displays, card readers, magnetic or paper tape readers, voice or handwriting readers, joysticks, or other known computer input / output devices. 550 is connected. Finally, the processor 510 may be connected to a wired or wireless communication network through the network interface 570. Through this network connection, the procedure of the method described above can be performed. The apparatus and tools described above are well known to those skilled in the computer hardware and software arts.

While specific embodiments of the present invention have been described so far, various modifications are possible without departing from the scope of the present invention.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the scope of the following claims, but also by those equivalent to the scope of the claims.

According to the present invention, a user can view predetermined image information generated from the portable terminal without carrying the portable terminal.

According to the present invention, a user can perform a predetermined input operation on the portable terminal without carrying the portable terminal.

According to the present invention, by recognizing the eye focal direction of the user, it is possible to control the operation of the portable terminal.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above-described embodiments, which can be variously modified and modified by those skilled in the art to which the present invention pertains. Modifications are possible. Accordingly, the spirit of the present invention should be understood only by the claims set forth below, and all equivalent or equivalent modifications thereof will belong to the scope of the present invention.

Claims (10)

In the eye focus recognition device for controlling the input and output of the portable terminal, A near field communication module configured to perform near field communication with the portable terminal; Display means for outputting image data received from the portable terminal through the near field communication module; An eye focus recognition module for recognizing an eye focus direction of a user looking at the display means; An input information generator for converting the eye focal direction into input information of the portable terminal Including, And the input information is transmitted to the portable terminal through the short range communication module. The method of claim 1, Determination unit for controlling the activation or deactivation of the input information generation unit when the eye focal direction corresponds to predetermined movement information Eye focus recognition device further comprises. The method of claim 1, The short range communication module includes a communication module of any one of a Bluetooth communication module, an infrared communication module, and a wired communication module. The method of claim 1, The portable terminal includes a mobile communication terminal, a PSTN terminal, VoIP, SIP, Megaco, PDA (Personal Digital Assistant), cellular phone, PCS (Personal Communication Service) phone, hand-held PC, CDMA-2000 (1X). , 3X) phones, WCDMA (Wideband CDMA) phones, Dual Band / Dual Mode phones, Global Standard for Mobile (GSM) phones, Mobile Broadband System (MBS) phones, satellite / terrestrial DMB (Digital Multimedia) Broadcasting) Eyephone focus recognition device comprising any one of a phone, or an MP3 player, wherein the eye focus recognition device is included in the glasses. The method of claim 1, The display means is any one of a liquid crystal display (LCD) or an organic light emitting diodes (OLED). In the operation method of the eye focus recognition device for controlling the input and output of the portable terminal, Receiving predetermined output information from a portable terminal through a short range communication module; Outputting the output information to a display means; Recognizing a user's eye focus and generating input information corresponding to a gaze direction of the eye focus; And Transmitting the generated input information to the portable terminal through the short range communication module; Including, And the portable terminal performs a predetermined processing operation according to the input information received from the short range communication module. The method of claim 6, Controlling activation or deactivation of the input information generator when the eye focal direction corresponds to predetermined movement information; Method of operation of the eye focus recognition device further comprises. The method of claim 6, The short range communication module includes a communication module of any one of a Bluetooth communication module, an infrared communication module, and a wired communication module. The method of claim 6, And the display means is one of a liquid crystal display (LCD) or an organic light emitting diode (OLED). A computer-readable recording medium having recorded thereon a program for executing the method of any one of claims 6 to 9.

Images