KR20030022538A - Input Method of Data Using Position Data Input System by Eye Tracking Camera - Google Patents

Abstract

PURPOSE: A method for inputting data using a viewpoint chasing coordinate input system is provided to display a keyboard layout having a plurality of keys corresponded to input data and to input desired data in a computer. CONSTITUTION: A cursor is displayed on a computer display and a position variation of a pupil of an eye is photographed using a CCD(Charge Coupled Device) element while chasing the pupil of a user(S10). A cursor coordinate is substituted for a position of the pupil from an image photographed using the CCD element, and new coordinates are created according to positions of the pupil(S11). A sound being generated in a nasal cavity is set as click data using a microphone(S12). The user requests to display a keyboard layout for inputting data(S13). A control unit of the computer displays a keyboard layout(S14). If the user watches the displayed layout and fixes a viewpoint on a key to be inputted using a viewpoint chasing unit(S15), a viewpoint-fixed position is converted into the coordinate(S16). A cursor is displayed on the corresponding coordinate(S17). The cursor is moved on the keyboard layout, the user fixes a viewpoint on a wanted key, and the nasal cavity sound is generated through the sound click unit. Thus, a corresponding key is selected(S18). The control unit extracts data corresponded to a selected key, and displays data adapted to a text display unit, or processes a corresponding function(S19). If a key is inputted, the corresponding key is displayed on the text display unit as an echo(S20).

Description

{Input Method of Data Using Position Data Input System by Eye Tracking Camera}

The present invention relates to a data input method using a viewpoint tracking coordinate input system, and more particularly, to a viewpoint tracking coordinate input system having a keyboard function for inputting data into a computer using coordinates generated by tracking a pupil location of a user. It relates to a data input method using.

In general, a conventional mouse for inputting coordinate data on a display when using a computer is to select and click the position of the input coordinate using one hand.

Therefore, in case of a handicapped person who cannot use his hand or when his hand cannot be used for other reasons, the coordinate data must be input through another method.

On the other hand, if you stare at any point of the human body through both eyes, both eyes move at the same angle and speed.

Therefore, by staring at a point through one of the two eyes and tracking the movement of the pupil of the other eye, it is possible to track the point at which one eye is staring.

On the other hand, the conventional keyboard (key board) used for inputting data to the computer is a number of character keys that can enter ASCII characters, such as alphanumeric characters, auxiliary functions for the character keys and to control the operation of the computer A plurality of keys comprising a plurality of function keys and the like are arranged, and data is input by inputting a signal generated when the key is turned on to a computer.

Therefore, in order to input data using the keyboard, the keys must be pressed to generate a signal of the key.

However, since the keyboard is composed of a very large number of keys, such as 101 keys and 103/106 keys, depending on the type of keyboard, it is advantageous for normal people who can use both hands smoothly.

Therefore, those who are in a condition where both hands cannot be freely used are often not used because the keyboard is not smooth.

Also, in some cases, data is input by converting words into characters using speech-to-text, but this function does not yet provide full functionality. Therefore, the data input method is not a proper means.

Accordingly, the present invention has been made in view of the problems of the prior art, and an object thereof is to display a keyboard layout composed of a plurality of keys corresponding to data to be input to a display of a computer, and to display desired data using a viewpoint tracking device. The present invention provides a data input method using a viewpoint tracking coordinate input system capable of inputting into a computer.

1 is a block diagram illustrating a configuration of a viewpoint tracking coordinate input system according to the present invention.

Figure 2 is a plan view for explaining the structure of the wear stand in the viewpoint tracking coordinate input system according to the present invention.

Figure 3 is a photograph for explaining the structure of the wearing table in the viewpoint tracking coordinate input system according to the present invention.

Figure 4 is a photograph for explaining another structure of the wear table in the viewpoint tracking coordinate input system according to the present invention.

5 is a flowchart illustrating a data input method using a viewpoint tracking coordinate input system according to the present invention.

6 is an exemplary view for explaining an embodiment of a keyboard layout in a data input method using a viewpoint tracking coordinate input system according to the present invention.

FIG. 7 is an explanatory diagram for explaining the types of keys of the keyboard layout shown in FIG. 6 for each type; FIG.

Explanation of symbols on the main parts of the drawings

10 control unit 15 display

20: sound click unit 21: microphone

30: viewpoint tracking unit 31: CCD element

35 cable 40 wear band

42: mount 43: lens

44: bridge 45: nose ring

46: fixed band

In order to achieve the above object, the present invention captures a pupil of the user looking at the display to track the viewpoint to generate coordinates on the display and the click unit for recognizing the sound generated by the user to generate the click data A method of inputting data by using a computer, comprising: (a) calibrating for setting a viewpoint and clicking data for each user, while (a1) moving and displaying a cursor on a computer display and tracking the pupil of the user. Photographing the change in the position of the pupil; (a2) generating coordinates for the position of the pupil by matching the position coordinates of the pupil with the coordinates of the cursor displayed by movement; (a3) setting a sound generated by a user as click data for data selection; (b) a user requesting a keyboard layout from a computer; (c) displaying the keyboard layout on the display; (d) tracking the user's viewpoint using the viewpoint tracking device, converting the tracked viewpoint into coordinates, and displaying the coordinates on a display by a cursor; (e) providing a data input method using a viewpoint tracking coordinate input system, comprising converting the data into data corresponding to a key selected by a user on a keyboard layout.

The present invention further comprises the step of confirming the key selected by the user by echo-displaying the key selected in the step (e) on the display.

The keyboard layout displayed in step (c) includes a plurality of character keys for inputting phonology, a plurality of numeric keys for inputting numbers, a plurality of auxiliary function keys for inputting the character keys or numeric keys, It includes a plurality of function keys for controlling the operation of the computer.

The keyboard layout is a 'b', 'b', 'c', 'ㄹ', 'ㅁ', 'ㅂ', 'ㅅ', 'ㅇ', 'ㅈ' and '' The sound of ',' ㄷ ',' ㅂ ',' ㅇ 'and' ㅈ 'can be sounded to input the sound of' ㅋ ',' ㅌ ',' ㄷ ',' ㅎ 'and' ?? 'respectively. A vowel key; '· (천)', 'ㅡ (지)', 'ㅣ (인)' keys for forming and inputting Korean vowels; Space key to input blank space, Enter key with return function and input completion function, Backspace key to move cursor back, Up, down, left and right keys to move cursor up, down, left and right, Save key with save function, Input Delete key to delete data, 'SMS' key to send short message, 'TTS' key to convert input text contents into words, and to call Internet domain 'WWW' key with function, 'No.' to enable number input Key; An input message is echoed, and a text display unit for confirming the user input contents and a receiver display unit for displaying the address of the other party receiving the message are formed.

As described above, the present invention facilitates data input to a user who is in a difficult situation using a keyboard.

(Example)

Hereinafter, with reference to the accompanying drawings showing a preferred embodiment of the present invention described above in more detail.

1 is a block diagram for explaining the configuration of the viewpoint tracking coordinate input system according to the present invention, Figure 2 is a plan view for explaining the structure of the wear stand in the viewpoint tracking coordinate input system according to the present invention, Figure 3 Photo for explaining the structure of the wear table in the viewpoint tracking coordinate input system according to the present invention, Figure 4 is a photograph for explaining another structure of the wear table in the viewpoint tracking coordinate input system according to the present invention, Figure 5 is a view in accordance with the present invention 6 is a flowchart illustrating a data input method using a tracking coordinate input system, FIG. 6 is an exemplary diagram for explaining an embodiment of a keyboard layout in a data input method using a viewpoint tracking coordinate input system according to the present invention, and FIG. It is explanatory drawing for demonstrating the keys of the keyboard layout shown by type for each kind.

According to the present invention, the front side photographing is performed by placing the CCD element 31 in front of the user's left eye 1 to 2 cm (see Fig. 2). This uses the physiological feature that the user's left and right eyes move at the same rate at the same time as they stare at any point.

And, as in the present invention, the technique of capturing the pupil at the position of 1-2 cm of the pupil of the CCD element 31 closes the error by observing the pupil movement extremely precisely compared to the conventional viewpoint tracking apparatus which photographs the pupil movement at a long distance. It is a fact that the level is greatly lowered. In other words, when the pupil is photographed at a distant distance from the side angle rather than the front as in the prior art, the accuracy of the analogy is considerably reduced.

According to the present invention, the position of the pupil may be determined to display a virtual keyboard capable of inputting data on the display 15, and input data using the same principle.

To this end, as shown in FIG. 1, the present invention selects data from a viewpoint tracker 30 that tracks a viewpoint by capturing a pupil whose position changes according to a viewpoint, and a sound generated from a user's nasal cavity. Sound control unit 20 for converting the click data for the control unit 10 and the display unit 15 using the viewpoint tracking unit 30 and the sound click unit 20 as a coordinate data input device Use a computer system.

The viewpoint tracking unit 30 is a CCD device 31 for capturing a pupil of the user's left eye, and a frame grabber for converting an image captured by the CCD device 31 into a frame signal and outputting image data. It consists of a hardware component such as a frame grabber (not shown in the drawing) and a program driven by the controller 10, that is, a computer CPU, and analyzes the image data output from the frame grabber to analyze the pupil of each pupil. Know your location.

In addition, the sound clicker 20 is in close contact with the outside of the nasal cavity of the user to detect the sound generated in the nasal cavity, and the sound to convert the analog signal input through the microphone 21 into a digital signal It is composed of a processing unit (not shown in the figure) and a program driven by the control unit 10 to generate click data using a sound input through the sound card.

In particular, in order to use the click data, the sound input through the microphone 21 should be a special sound that is distinguished from the ambient noise.

In other words, most of the sounds that can occur in the human body are generated through the mouth, and in order to use the sound as click data, sounds of a specific frequency band having a narrow frequency band must be used. In addition, the error in the click operation can be reduced only by setting and using the sound of this specific frequency band as the click data.

Therefore, using a sound in a narrow frequency band generated in the nasal cavity may reduce the error in the click operation, rather than using a sound generated in the mouth having a frequency distribution between 400 and 1600 Hz.

To this end, in the present invention, the microphone 21 is in close contact with the user's nose (outside the portion where the nasal cavity is located), and because it is in close contact with the nose, the sound generated from the nasal cavity is input without inserting other sounds (noise). There is an advantage to receive.

In the present invention, a wear table 40 having a structure as shown in FIGS. 2 and 3 is used to allow the user to wear the CCD element 31 and the microphone 21 as described above.

The wear table 40 is in close contact with one eye of the user, and the mount 42 on which the CCD element 31 is mounted, and the lens 34, etc., are formed at the other eye position of the user. A receiving portion 41, a bridge 44 connecting the mount 33 and the receiving portion 41, and the microphone 21 are provided on either side thereof, and one end thereof is connected to the bridge 44. And a pair of nose rings 45 covering the user's nose, a fixing band 46 for fixing the mount 33 and the receiving portion 41 to the user, and an image output from the CCD element 31. It is composed of a cable 35 for transmitting data and the audio signal output from the microphone (20).

Here, the lens 43 accommodated in the accommodation portion 41 should be made of a lens made of the frequency according to the eyesight of the user using the wearing table 40.

In the embodiment of the present invention, the fixing band 37 has been described using a band type as an example. However, as shown in FIG.

In addition, the CCD element 31 mounted on the mount 33 is installed at a closest distance from the pupil of the user. In order to minimize the relative positional change between the pupil and the CCD element 31, the CCD element 31 is minimized. Use to maintain the nearest distance to shoot the user's pupils.

Meanwhile, in the present invention, in addition to the wear table as shown in FIGS. 2 and 3, the CCD 15 and the small display 54 are mounted together on the wear table 50 as shown in FIG. The same contents as displayed in are displayed.

In this case, since the relative positional change between the user's gaze and the small display 54 does not occur, the coordinate data according to the viewpoint tracking is more accurate.

A virtual keyboard for inputting data into a computer using the viewpoint tracking coordinate input system configured as described above is performed as follows.

Since the present invention is for inputting data using a virtual keyboard without using an actual keyboard, a keyboard layout 50 as shown in FIG. 6 should be designed.

In the embodiment of the present invention, in the case of Hangul consonants' ㄱ (62) ',' ㄴ (69) ',' c (61) ',' ㄹ (68) ',' ㅁ (67) ',' ㅂ (63) ) ',' ㅅ (66) ',' ㅇ (64) ',' ㅈ (60) ', and' ㄱ (62) ',' c (61) ',' ㅂ (63) ',' ㅇ ( 64) ',' ㅈ (60) 'to sound the sound of' ㅋ ',' ㅌ ',' ㄷ ',' ㅎ 'and' ?? ', respectively. 65) '.

Also, in the case of Hangul vowels, 'ㅏ', 'ㅑ', '' using three keys consisting of '· (cheon, 70)', 'ㅡ (지, 71)', 'ㅣ (인, 72)' Vowel input such as 'ㅓ', 'ㅕ', 'ㅗ', 'ㅛ', 'TT', 'ㅠ', 'ㅡ', 'ㅣ', etc. were made possible.

And the space (SPACE, 73) to input the blank, 'ENTER (75)' having a return function and the input completion function, '◀ (backspace, 74)' to reverse the cursor, the cursor up, down, left, right △ (76) ',' ▽ (77) ',' ◁ (78) ',' ▷ (79) ', etc. are placed to move to' Save (51) ', which has a storage function. 'Delete (52)' to delete the old data, 'SMS (54)' having the function to send a short message, 'TTS (Text-To-Speech) having the function to convert the input text content into words 55) ',' WWW 56 'having a function of calling the Internet domain, and' No. 76 'which enables the input of a number are additionally arranged.

In addition, a text display unit 58 in which the input message is echoed to confirm the contents input by the user, and 'To: 57', in which the address of the receiving party is displayed, are displayed as data display areas. Is formed.

Since the keyboard layout 50 according to the present invention made as described above allows all data to be input using the minimum keys, the display area of the keys displayed on the display 15 can be displayed as large as possible. By using the coordinate data through the viewpoint tracking, an error of inputting a desired key can be reduced.

In addition, in order to reduce an error in data input, when the user fixes the viewpoint to any key, the color of the selected key is changed to a different color or shape with respect to other keys nearby. Before you click on a key, you are assured of which key will be entered.

In addition, in order to perform the corresponding functions of the respective keys, the keyboard layout 50 formed as described above has to be connected and executed with a corresponding program corresponding to the function of each key.

In the exemplary embodiment of the present invention, although Hangul is displayed as an example, language systems of English or other countries may be arranged and used similarly to Hangul.

As shown in FIG. 5, the process of inputting data into the computer using the keyboard layout having the above arrangement and the viewpoint tracking coordinate input system is performed by the viewpoint tracker 30 and the sound clicker 20 for each user. In order to calibrate for the viewpoint setting and the click data setting of the viewpoint tracking coordinate input system, the CCD display 31 moves and displays the cursor on the computer display 15 and changes the position of the pupil while the pupil of the user is tracked. Shooting using (S 10).

Coordinates are generated for each pupil position by substituting the cursor coordinates at the time from the image photographed using the CCD element 31 at the pupil position (S 11).

Then, the sound generated in the nasal cavity is set as click data using the microphone 21 (S 12).

Through the steps S 10 to S 12, a calibration process is performed by registering coordinates according to the pupil position and registering a sound for clicking.

When the calibration process is completed as described above, the user requests the computer to display the keyboard layout shown in FIGS. 6 and 7 for data input (S 13).

Accordingly, the controller 10 of the computer displays the keyboard layout on the display 15 (S 14), which is similar to the user inputting data using the actual keyboard.

When the user fixes the viewpoint using the viewpoint tracker 30 to a key to be input while looking at the keyboard layout displayed on the display 15 (S 15), the position at which the viewpoint is fixed is converted into coordinates (S 16).

If the viewpoint is converted to coordinates, a cursor is displayed on the coordinates (S 17).

While moving the cursor on the keyboard layout 50 through the steps S15 to S17, the user fixes a viewpoint to a desired key and generates a nasal sound through the sound clicker 20 to select the corresponding key (S). 18).

The controller fetches data corresponding to the selected key and displays the corresponding data on the text display unit 58 or processes the corresponding function (S19).

At this time, if a key is selected to inform the user whether the key selected by the user is correctly input, the corresponding key is echoed on the text display unit 58 (S20).

The echo display simply confirms correct input before inputting data or performing a function corresponding to the corresponding key.

According to the present invention made as described above, a user who is in a situation where it is difficult to use a real keyboard because data can be input similarly to a real keyboard using a viewpoint tracking coordinate input system that can input coordinate data by tracking a pupil position. Provides the effect of smooth data entry.

In the above, the present invention has been illustrated and described with reference to specific preferred embodiments, but the present invention is not limited to the above-described embodiments and the general knowledge in the technical field to which the present invention pertains without departing from the spirit of the present invention. Various changes and modifications will be made by those who possess.

Claims (5)

A method of inputting data using a computer having a viewpoint tracking device that generates coordinates on a display by tracking a viewpoint by photographing a pupil of a user looking at a display and a click unit that generates click data by recognizing a sound generated by the user. To (a) In order to calibrate for user setting of viewpoint and click data, (a1) moving and displaying the cursor on a computer display and capturing a change in the position of the pupil while the pupil of the user is being tracked; (a2) generating coordinates for the position of the pupil by matching the position coordinates of the pupil with the coordinates of the cursor displayed by movement; (a3) setting a sound generated by a user as click data for data selection; (b) a user requesting a keyboard layout from a computer; (c) displaying the keyboard layout on the display; (d) tracking the user's viewpoint using the viewpoint tracking device, converting the tracked viewpoint into coordinates, and displaying the coordinates on a display by a cursor; and (e) converting the data into data corresponding to a key selected by the user on the keyboard layout. The keyboard layout of claim 1, wherein the keyboard layout displayed in step (c) includes a plurality of character keys for inputting phonology, a plurality of number keys for inputting numbers, and a number of inputs for the character keys or number keys. And a plurality of function keys for controlling the operation of the computer. The data input method according to claim 1, further comprising the step of echo-displaying the key selected in the step (e) on the display to confirm the key selected by the user. The data input using the viewpoint tracking coordinate input system according to claim 1, wherein if the viewpoint is fixed to a key to be input by the user in step (e), the color or shape of the corresponding key is displayed differently from other surrounding keys. Way. The keyboard layout of claim 1, wherein the keyboard layout includes 'a', 'b', 'c', 'ㄹ', 'ㅁ', 'ㅂ', 'ㅅ', 'ㅇ', 'ㅈ' 'And, the sound of' ㄱ ',' ㄷ ',' ㅂ ',' ㅇ ',' ㅈ 'is pronounced so that' ㅋ ',' ㅌ ',' ㄷ ',' ㅎ ',' ?? ' A vowel key for inputting a sound; '· (천)', 'ㅡ (지)', 'ㅣ (인)' keys for forming and inputting Korean vowels; Space key to input blank space, Enter key with return function and input completion function, Backspace key to move cursor back, Up, down, left and right keys to move cursor up, down, left and right, Save key with save function, Input WWW with delete key to delete data, SMS key with ability to send short messages, TTS key with text-to-speech, and Internet domain call A No key for enabling key and number input; Data using the viewpoint tracking coordinate input system, characterized in that the input message is echoed to form a text display unit for confirming the user input, and a receiver display unit for displaying the address of the other party receiving the message. Input method.