KR20030022537A - Position Data Input System Using Eye Tracking Camera And Method Thereof - Google Patents

Abstract

PURPOSE: A system and a method for inputting coordinates using a pupil tracking camera are provided to input the correct coordinates data fast by improving a structure and performance of a viewpoint tracking device. CONSTITUTION: A CCD(Charge-Coupled Device)(31) pictures one pupil of a user. A frame graver outputs the image data by converting the image pictured by the CCD(31) into a frame signal. A pupil position detector detects the position variation of the pupil by analyzing the image data inputted from the frame graver and converts it into the coordinates. A microphone detects a sound generated by the user. A sound processor output the audio data by converting the sound generated by the user into a digital signal. A peak detector detects a preset peak signal by receiving the audio data inputted through the sound processor. A controller displays a cursor on a display by picturing the pupil and generating the coordinates, and executes a program by generating a click data as detecting a nasal cavity sound.

Description

Coordinate input system using pupil tracking camera and method thereof {Position Data Input System Using Eye Tracking Camera And Method Thereof}

The present invention relates to a coordinate input system and method using a pupil tracking camera, and more particularly, to a coordinate input system and a method using a pupil tracking camera that can input coordinates to a computer using a change in the pupil position of the user. It is about.

In general, a conventional mouse for inputting coordinate data on a display when using a computer is to select and click the position of the input coordinate using one hand.

Therefore, in case of a handicapped person who cannot use his hand or when his hand cannot be used for other reasons, the coordinate data must be input through another method.

On the other hand, staring at any point in the human body through both eyes, both eyes move at the same angle and speed.

Therefore, by staring at a point through one of the two eyes and tracking the movement of the pupil of the other eye, it is possible to track the point at which one eye is staring.

One of the problems of the conventional viewpoint tracking equipment using this principle is that the accuracy is poor because the pupils are photographed at a considerable distance from the pupils and the pupils are photographed at a blind spot instead of the front face.

Accordingly, the present invention has been made in view of the problems of the prior art, and an object thereof is to improve the structure and performance of a viewpoint tracking device capable of inputting coordinate data, and to accurately input coordinate data more quickly. The present invention provides a coordinate input system using a camera and a method thereof.

1 is a block diagram illustrating a configuration of a coordinate input system using a pupil tracking camera according to the present invention.

Figure 2 is a plan view showing one embodiment of a pupil tracking camera in the present invention.

Figure 3 is a photograph showing an embodiment of the pupil tracking camera in the present invention.

4 is a flowchart illustrating a coordinate input method using a pupil tracking camera according to the present invention.

5 is a view illustrating a viewpoint tracking principle in the present invention.

6 is a cell diagram of a display for explaining the viewpoint tracking principle in the present invention.

Figure 7 is a photograph showing another embodiment of the pupil tracking camera in the present invention.

Explanation of symbols on the main parts of the drawings

10 control unit 12 peak detection unit

14: pupil position detection unit 20: microphone

22: sound processor 30: wearing table

31 CCD device 32 Frame grabber

40: display 42: video card

In order to achieve the above object, the present invention includes a photographing element for photographing the pupil of either eye of the user; A frame grabber for converting an image photographed by the photographing element into a frame signal and outputting image data; A pupil position detector for analyzing image data input through the frame grabber to detect a change in the position of the pupil and converting the pupil position into coordinates; A microphone for detecting a sound generated by the user; A sound processor for converting a sound generated by the user through the microphone into the nasal cavity and converting the sound into a digital signal to output audio data; A peak detector configured to receive audio data input through the sound processor and detect a preset peak signal; When the user selects the calibration mode, the cursor is scanned in the computer display to allow the user's pupil to track it, and then the pupil's position is photographed by the photographing device to convert the pupil's position according to the position of the scanned cursor into coordinates. Set the coordinates of the cursor for each position of the user, input the sound generated by the user through the microphone and the sound processor for the click operation for data selection, and set the click data set through the peak detector, and the user selects the tracking mode. When selected, the user's pupil is photographed using the photographing element to track the change in the position of the pupil, and the cursor coordinates set for each pupil position are retrieved to display the cursor at the corresponding position on the computer display, and the user generates a sound to display the computer. One Day on Pinterest Selecting provides a coordinate input system using a pupil tracking camera comprises a control unit that by executing the program associated with the data.

The present invention further includes a wearing table which wears the photographing device and the microphone to the user, and the photographing device includes a CCD device.

The wear table is in close contact with either eye of the user, the mount to which the photographing element is mounted; A receiving portion installed in front of the other eye of the user in a position symmetrical with the mount; A bridge connecting the mount and the receiving portion; A pair of nose rings installed at either side of the microphone, one end of which is connected to the bridge and spans the user's nose; It includes a fixing for fixing the mount and the receiving portion to the user.

The accommodating part may be equipped with a lens made of a frequency suitable for a user's vision, or a small display for displaying display contents. Then, the microphone is in close contact with the user's nose receives the sound generated in the nasal cavity.

In addition, the present invention provides a method of moving a cursor on a computer display using a change in the position of the pupil, the method comprising: (a) photographing a pupil of one eye of a computer user using both eyes; (b) selecting one of a calibration mode for calibrating the moving distance of the pupil on a computer display and a tracking mode for inputting coordinate data using the movement of the pupil; (c 1) selecting a calibration mode, scanning the cursor in the computer display to allow the user's pupil to track it and photographing the change in the pupil's position; (c 2) setting the coordinates of the cursor for each position of the pupil by converting the position of the pupil according to the position of the scanned cursor into coordinates; (c 3) a user generating a sound and setting it as click data to set a click operation for selecting data displayed at coordinates of a specific position on a computer display; (d 1) selecting a trekking mode, photographing the pupil to track a change in the position of the pupil; (d 2) extracting cursor coordinates for each position of the pupil set in the calibration step for each position of the photographed pupil; (d 3) positioning the cursor at a corresponding position on the computer display by using the cursor coordinates extracted for each pupil position; (d 4) It provides a coordinate input method using a pupil tracking camera, comprising the step of executing a corresponding event when the cursor is located on the computer display using the sound set in the calibration step.

The resolution of the cursor displayed on the computer display is proportional to the frame rate of the imaging device, and the click data utilizes sound generated in the nasal cavity of the user.

As described above, in the present invention, the user's left eye is completely blocked by a small camera, and the front side is photographed at 1 to 2 cm to significantly reduce the level of error through observation of the pupil movement extremely precisely compared to the conventional viewpoint tracking device. You can easily select any data on the display using the sound generated by.

(Example)

Hereinafter, with reference to the accompanying drawings showing a preferred embodiment of the present invention described above in more detail.

1 is a block diagram illustrating a configuration of a coordinate input system using a pupil tracking camera according to the present invention, FIG. 2 is a plan view showing a pupil tracking camera in the present invention, and FIG. 3 is a pupil tracking camera according to the present invention. 4 is a flowchart illustrating a coordinate input method using a pupil tracking camera according to the present invention, and FIG. 5 is a view illustrating a viewpoint tracking principle in the present invention.

The present invention relates to extracting coordinate data by inferring in real time where a user is staring at a display by photographing a pupil of a computer user.

For example, as shown in FIG. 5, when staring at point a (FIG. 5B) of the display 40, the pupil center of the user's eye 5 is located at 1 (FIG. 5A), When staring at b, d, and c, the fact that the position of the user's pupil changes to the positions 2, 4, and 3, respectively, is set during the calibration process at the beginning of the course of use, and when the user's pupil is at 5, he displays We can infer that we are staring at point e of 40 through the ratio calculation.

In the present invention, the viewpoint tracking algorithm is a modification of using a ratio calculation equation, and as shown in FIG. 6, the first calibration is performed by using 16 intersection points generated by firstly dividing the width and length of the display 40 into three equal parts. do. That is, the position of the pupil center corresponding to each intersection point is set.

In addition, the first divided nine cells are further divided into 16 × 16 segments, and the entire display 40 is divided into 48 × 48 segments, and the pupil coordinates measured during the calibration process are determined in which cell among the 2304 cells. Inferred based on the position of.

That is, the pupil coordinates are also divided into 48 × 48 and the display 40 is also divided into 48 × 48 so that each cell matches 1: 1.

According to the present invention, the front side photographing is performed by placing the CCD element 31 in front of the user's left eye 1 to 2 cm (see Fig. 2). This uses the physiological feature that the user's left and right eyes move at the same rate at the same time as they stare at any point.

In addition, the present invention uses a technique of positioning the CCD element 30 at a distance of 1 to 2 cm of the pupil and using the technique of photographing the pupil in close proximity, thereby providing a conventional viewpoint tracking apparatus for capturing the movement of the pupil at a distance rather than at the front. This allows for extremely precise tracking.

The present invention has the following configuration to move the cursor on the display 40 by grasping the position of the pupil using the principle described above.

As shown in FIG. 1, the present invention converts an image photographed by the CCD element 31 for capturing the pupil of one eye of the user, and an image photographed by the CCD element 31 into a frame signal to convert the image data. An output frame grabber 32, a pupil position detector 14 for detecting a change in the position of the pupil and converting it into coordinates by analyzing image data input through the frame grabber 32, and a sound generated by a user The microphone 20 for detecting the sound, the sound processor 22 for outputting audio data by converting the sound generated by the user input through the microphone 20 to a digital signal, the audio input through the sound processor 22 Peak detection unit 12 that receives data and detects a preset peak signal. When the user selects a calibration mode, the cursor is scanned in the computer display 40. After allowing the pupil of the user to track this, the position of the pupil is photographed by the CCD device 31 to convert the position of the pupil according to the position of the cursor to be scanned into coordinates, and the coordinates of the cursor are set for each pupil position. For the click operation for selection, the sound generated by the user is input through the microphone 20 and the sound processor 22 to set the click data set through the peak detector 12, and the user selects a trekking mode. Then, by capturing the pupil of the user using the CCD device 31 to track the change of the position of the pupil, the cursor coordinates set for each pupil position are extracted and the cursor is positioned at the corresponding position on the computer display 40 through the video card 42. When a user selects any data on the computer display 40 by generating a sound, a program connected to the data is displayed. It consists of a control unit 10 for executing.

The present invention further comprises a wear table 30 having a structure such as glasses (or goggles) to wear the CCD element 31 and the microphone 20 on the face of the user, the wear table is shown in FIG. 3, the mount 33 in close contact with one eye of the user, on which the CCD element 31 is mounted, the lens 34 mounted on the other eye of the user, and the mount. A bridge 35 connecting the lens 34 and the lens 34, and the microphone 20 is installed on either side, one end of which is connected to the bridge 35, and a pair of the user's nose Nose ring 36, fixed band 37 for fixing the mount 33 and the lens 34 to the user, the image data output from the CCD element 31 and the audio output from the microphone 20 It is composed of a cable 38 for transmitting a signal.

Here, the lens 34 should be made of a lens made of the frequency according to the eyesight of the user using the wearing table 30.

In addition, since the microphone 20 is mounted on the nose ring 36 as described above, the microphone 20 is in close contact with the user's nose, so that the sound generated in the nasal cavity can be input without inserting another sound (noise). .

In the embodiment of the present invention, the fixing band 37 has been described using a band type as an example. However, the fixing band 37 may have a structure similar to that of a general eyeglass leg.

The CCD element 31 mounted on the mount 33 is installed at the nearest distance from the pupil of the user. The CCD element 31 is used to minimize the relative positional change between the pupil and the CCD element 31. To maintain the closest distance to photograph the pupil of the user.

In the above description, the pupil position detecting unit 14 and the peak detecting unit 12 are described as separate components with respect to the control unit 10. However, the control unit 10, which is a CPU of a computer like a conventional mouse driver, may be used. It may be implemented as a separate program to be executed.

In order to use the coordinate input system using the pupil tracking camera according to the present invention configured as described above as a coordinate input device of a computer, as shown in FIG. 4, a calibration process may be performed first and then used in a tracking mode.

In order to perform the calibration mode, the user wears the wear table 30, and in the process of photographing the pupil of the user using the CCD device 31 (S 1), the user selects the calibration mode (S 2).

The selection of the calibration mode (S 2) displays a selection display on the display 40 through a program operated by the controller 10 to select a calibration mode and a tracking mode.

When the user selects the calibration mode, the controller 10 displays a cursor on the display 40 and moves the cursor up, down, left, and right at a constant speed (S 10).

The user changes the viewpoint to track the cursor displayed on the display 40, and at the same time, the pupil whose position is changed through the CCD element 31 is captured (S 11).

Therefore, the coordinate data of the pupil is generated with respect to the position of the cursor displayed on the display 40 through the pupil position detector 14 (S12).

The coordinate data of the cursor according to the position of the pupil thus generated is used as the coordinate data in the tracking mode described below.

In order to set the click data for the click operation, the user generates a sound using the nasal cavity and inputs the nasal sound into the microphone 20 (S 13).

Here, since the microphone 20 is installed on the nose ring 36 of the wear table 30 and is closely attached to the user's nose, the microphone 20 may clearly receive a nasal sound generated as a small sound.

The user can select and use the most commonly generated sound among the sounds that can be generated through his nasal passages as click data (S14).

Through the above process, the calibration process for setting the coordinates and setting the click data is completed (S15).

The calibration process must be newly executed every time the user uses the computer. This is because a slight change in the position of the CCD element 31 occurs with respect to the pupil when the wearing table 30 is worn, and the sound generated in the nasal cavity may also change depending on the physical condition.

When the calibration process is completed as described above, when the user looks through the lens 34 with the right eye, the display 40 displaying the screen according to the program executed by the controller 10 to use the computer, the left eye Because the pupil moves at the same speed and angle in compliance with the right eye, the cursor is placed at the point on the display 40 currently viewed.

In other words, while photographing the pupil with the CCD element 31 (20), the pupil position is tracked through the pupil position detector 14 (S21).

When the pupil position is determined by the pupil position detector 14, the controller 10 converts coordinates corresponding to the position of the current pupil into display coordinates using the coordinates set in the calibration process (S22). Then, the cursor is positioned at the corresponding position converted to the display coordinates (S 23).

The display period of the cursor displayed on the display 40 in the above process depends on the frame rate of the frame generated by the frame grabber 32.

In other words, since the change in the position of the pupil is detected by the pupil position detector 14 which receives the image generated by the CCD element 31 and the frame grabber 32, the display period of the cursor is the frame of the frame grabber 32. If the frame rate of the frame grabber 32 is lower than 30 fps (frame per second), the maximum display period of the cursor is displayed at 30 times or less per second.

As described above, when the cursor is displayed on the display 40 according to a change in the pupil position of the user, the user generates a nasal sound while viewing an icon (link) or data located at a specific position of the display 40 (S 24). Click data is input through 20 to execute another program connected to the data (S 25).

The present invention made as described above can be used as an input means that can replace the mouse, a typical representative coordinate data input device, which results in the use of a computer for the handicapped people.

In addition, the general public can increase efficiency by using the present invention as a separate input means in the situation of computer games, work, and the like.

Meanwhile, in the above embodiment, the desk type display 40 is viewed through the lens 34 portion of the wear table 30, and the coordinate movement is generated by tracking the pupil movement at this time with the CCD element 31. In another embodiment of the present invention, as shown in FIG. 7, the CCD element 52 is mounted on one side of the wearing table 50, and the small display 54 is provided on the other side.

When the CCD device 52 and the small display 54 are integrally formed, the relative position between the user's eyes and the small display is always kept constant even when the user's head moves, so that the user moves the head slightly after calibration. The advantage is that the accuracy of the trace is not compromised.

The small display 54 may use a small CRT display or an LCD panel, but it is advantageous to use a relatively light LCD panel.

According to the present invention made as described above, by taking a front close-up photograph using a CCD element 1 to 2 cm in front of one eye of the user, the error level of the coordinate data can be significantly lowered, and thus it can be smoothly used as a coordinate input means.

The sound generated in the nasal cavity of the user may be used as click data for selecting any data on the display.

In the above, the present invention has been illustrated and described with reference to specific preferred embodiments, but the present invention is not limited to the above-described embodiments and the general knowledge in the technical field to which the present invention pertains without departing from the spirit of the present invention. Various changes and modifications will be made by those who possess.

Claims (11)

A photographing element for photographing the pupil of either eye of the user; A frame grabber for converting an image photographed by the photographing element into a frame signal and outputting image data; A pupil position detector for analyzing image data input through the frame grabber to detect a change in the position of the pupil and converting the pupil position into coordinates; An audio sensor detecting a sound generated by a user; A sound processor for converting a sound generated by a user input through the audio sensor into a digital signal and outputting audio data; A peak detector configured to receive audio data input through the sound processor and detect a preset peak signal; When the user selects the calibration mode, the cursor is scanned in the computer display to allow the user's pupil to track it, and then the pupil's position is photographed by the photographing device to convert the pupil's position according to the position of the scanned cursor into coordinates. Set the coordinates of the cursor for each position of the user, receive the sound generated by the user through the audio sensor and the sound processor for the click operation for data selection, and set the click data set through the peak detector, and the user If you select and capture the user's pupil using the photographing element to track the change in the position of the pupil, retrieve the cursor coordinates set for each pupil position to display the cursor at the corresponding position on the computer display, the user generates a sound Any on display Selecting data coordinate input system using a pupil tracking camera comprises a control unit that by executing the program associated with the data. The coordinate input system using a pupil tracking camera according to claim 1, wherein the photographing element comprises a CCD element. The coordinate input system using a pupil tracking camera according to claim 1, further comprising a wearing table which wears the photographing element and the audio sensor to a user. The mount of claim 3, wherein the wear stand is mounted to the eye of either user to mount the photographing device thereon; A receiving portion installed in front of the other eye of the user in a position symmetrical with the mount; A bridge connecting the mount and the receiving portion; A pair of nose rings installed at either side of the audio sensor, one end of which is connected to the bridge and spans the user's nose; Coordinate input system using a pupil tracking camera characterized in that it comprises a fixing for fixing the mount and the receiving portion to the user. The coordinate input system using the pupil tracking camera of claim 4, wherein the accommodation unit further comprises a lens having a frequency corresponding to a user's vision. The coordinate input system using the pupil tracking camera of claim 4, wherein the accommodation unit further includes a small display displaying the same contents displayed on the display. The coordinate input system using the pupil tracking camera of claim 1, wherein the audio sensor is a sensor that is in close contact with a user's nose and receives a sound generated from the nasal cavity. The coordinate input system using a pupil tracking camera according to claim 1, wherein the pupil position detector and the peak detector are configured by a program driven by the controller. In a method of moving a cursor on a computer display using a change in the position of a pupil, (a) photographing the pupils of either eye of a computer user using both eyes; (b) selecting one of a calibration mode for calibrating a moving distance of the pupil on a computer display and a tracking mode for inputting coordinate data using the movement of the pupil; (c 1) selecting a calibration mode, scanning the cursor in the computer display to allow the user's pupil to track it and photographing the change in the pupil's position; (c 2) setting the coordinates of the cursor for each position of the pupil by converting the position of the pupil according to the position of the scanned cursor into coordinates; (c 3) a user generating a sound and setting it as click data to set a click operation for selecting data displayed at coordinates of a specific position on a computer display; (d 1) selecting a trekking mode, photographing the pupil to track a change in the position of the pupil; (d 2) extracting cursor coordinates for each position of the pupil set in the calibration step for each position of the photographed pupil; (d 3) positioning the cursor at a corresponding position on the computer display by using the cursor coordinates extracted for each pupil position; and (d 4) moving the cursor position on the computer display to the step of executing a corresponding event when the cursor is clicked using the sound set in the calibration step. 10. The method of claim 9, wherein the resolution of the cursor displayed on the computer display is proportional to the frame rate of the camera. 10. The method of claim 9, wherein the click data uses sound generated in the nasal cavity of the user.