KR101673694B1 - Gaze tracking apparatus and method for detecting glint thereof - Google Patents

Abstract

The present invention relates to a line-of-sight tracking apparatus and a method of detecting a reflection point of the line-of-sight tracing apparatus. The line-of-sight tracing apparatus and the reflection point detecting method of the present invention are characterized by obtaining eye images through a camera, extracting candidate reflection points reflected by two or more lights in the eye image, Selects one of the pair of reflection points of the at least one reflection point pair based on the arrangement information of the two or more light sources, determines a representative point of each reflection point constituting the selected reflection point pair , And the gaze is traced using the representative point of the reflection point and the pupil center point.

Description

TECHNICAL FIELD [0001] The present invention relates to a gaze tracking apparatus and a method for detecting a reflection point of the gaze tracking apparatus,

The present invention relates to a line-of-sight tracking apparatus for detecting a reflection point reflected by a pupil in consideration of an illumination position in line-of-sight tracking, and a reflection point detecting method therefor.

Eye tracking technology is attracting attention as a new user input technology in HCI (Human Computer Interaction) field. Such eye tracking technology has been developed and commercialized so that the disabled can use the computer. A person's gaze reflects special interest or biological change. Therefore, much research has been conducted to apply the eye tracking technology to the fields of Attendive User Interface (AUI) and Safety System for preventing drowsiness.

Conventionally, a technique of tracking a user's gaze using a glint reflected from a pupil has been introduced. This conventional line-of-sight tracking technology detects the brightest reflection point among the reflection points existing in the eye image photographed through the camera. However, when a reflection point is detected using only the brightness of the reflection point as in the conventional art, a lot of noise is generated. For example, when a user wears glasses, there is a high possibility that a reflection point is mistaken due to a reflection point (noise) formed in the glasses.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a line-of-sight tracking apparatus for accurately detecting a reflection point reflected by a pupil in consideration of an illumination position in line-of-sight tracking, and a reflection point detecting method thereof.

According to another aspect of the present invention, there is provided a method of detecting a reflection point of a line-of-sight tracking apparatus, comprising: acquiring an eye image through a camera; Selecting one or more pairs of reflection points of the at least one pair of reflection points on the basis of the arrangement information of the two or more light sources; And determining a representative point of each reflection point constituting the selected reflection point pair.

In addition, the candidate refracting point extracting step may extract an outline of pixels having a reference brightness or more from the eye image.

Further, the reflec- tion point pair generation step is characterized by determining the number of candidate reflections constituting the reflec- tion point pair according to the number of the illuminations.

The reflec- tion point pair selecting step selects a reflec- tion point pair composed of candidate reflections that are arranged to coincide with the arrangement information of the two or more light sources at a predetermined ratio or more.

The arrangement information is a reference axis distance between the two or more light sources.

The arrangement information may be a ratio of an X-axis distance and a Y-axis distance between the two or more light sources.

In addition, the refracting point selecting step determines a pixel having the brightest value among the candidate refracting points constituting the refracting point pair as a representative point.

Meanwhile, the eye tracking apparatus according to an embodiment of the present invention includes a light source configured to emit infrared light, a camera configured to acquire an eye image, a candidate reflex point generated by the illumination in the eye image, Selecting a pair of reflection points when the arrangement of the candidate reflection points of the pair of reflection points is equal to or more than a certain ratio with the arrangement information of the two or more light sources, And a processing unit for determining representative points of the respective reflection points.

In addition, the light source is realized as a light source emitting infrared rays such as an infrared ray emitting diode and an infrared ray lamp.

Further, the camera is characterized by including an infrared filter.

The arrangement information is a reference axis distance between the two or more light sources.

The arrangement information may be a ratio of an X-axis distance and a Y-axis distance between the two or more light sources.

Further, the processing unit tracks the line of sight using the representative point of each reflection point and the pupil center point.

The present invention can accurately detect a reflection point reflected by a pupil in consideration of an illumination position in line-of-sight tracking, thereby being robust against noise and improving accuracy.

1 is a block diagram showing a gaze tracking apparatus according to an embodiment of the present invention;
2 is an exemplary view showing the arrangement of a light and a camera associated with the present invention;
Figs. 3 and 4 are diagrams showing the relationship between the illumination position and the reflection point position in relation to the present invention; Fig.
5 is a flowchart illustrating a method of detecting a reflection point of a gaze tracking apparatus according to an embodiment of the present invention.
6 is a diagram illustrating an example of a reflection point detection process according to an embodiment of the present invention.

The terms "comprises", "comprising", "having", and the like are used herein to mean that a component can be implanted unless otherwise specifically stated, Quot; element ".

Also, the terms " part, "" module, " and" module ", as used herein, refer to a unit that processes at least one function or operation and may be implemented as hardware or software or a combination of hardware and software . It is also to be understood that the articles "a", "an", "an" and "the" Can be used.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a block diagram illustrating a gaze tracking apparatus according to an exemplary embodiment of the present invention, and FIG. 2 is an exemplary view illustrating an arrangement of a lighting and a camera associated with the present invention.

As shown in Fig. 1, the gaze tracking device includes a light source 10, a camera 20, and a processing unit 30.

The illumination 10 is composed of two or more light sources 10a and 10b. Here, the light sources 10a and 10b may be implemented by an IR LED (Light Emitting Diode) or an IR lamp that emits infrared rays (IR). In the present embodiment, as shown in FIGS. 1 and 2, the illumination 10 including two light sources 10a and 10b is described as an example, but the present invention is not limited thereto.

The camera 20 is installed so that the lens faces the user (e.g. driver) and acquires image information for the user. The camera 20 may be implemented by an infrared camera composed of an image sensor and an IR filter.

The illumination 10 and the camera 20 are arranged to face the user's eyes as shown in Fig. 2 and the camera 20 photographs a specific area (e.g., an eye area) of the user's face when the illumination 10 is lit .

The processing unit 30 acquires the eye image through the camera 20, and detects the pupil from the eye image. Then, the processing unit 30 extracts a candidate reflex point (glint) in the eye image. The reflection point is a set of pixels with a certain brightness or more. The processing unit 30 extracts a contour of pixels having a reference intensity or more within the eye image to detect a candidate reflection point. Alternatively, the processing unit 30 may detect a candidate reflex point by applying a high pass filter.

The processing unit 30 constructs a pair of predetermined reflections with respect to the extracted candidate reflections. Here, the number of candidate reflection points constituting a pair is equal to the number of light sources. For example, if the number of light sources is 2, and the reflection points are 4, A, B, C, and D, B, C), (B, D), and (C, D).

The processing unit 30 calculates the distance between the candidate reflections constituting the pair, the distance between each pair of candidate reflections and the center of the pupil, and the size (length) of each of the candidate reflections constituting the pair. That is, the processing unit 30 calculates information about the candidate reflections constituting each pair.

For example, when the reflection points A and B constitute a pair, the distance between the reflection point A and the reflection point B, the distance between the center point of the pupil and the reflection point A, the distance between the center point of the pupil and the reflection point B, do.

When the first light source 10a and the second light source 10b are installed on the same axis (Y axis), the reflection points of the light sources 10a and 10b are likely to be formed on the same axis as the light sources 10a and 10b Is high. Therefore, the distance (Y-axis distance) on the same axis (Y-axis) between the candidate reflections constituting the pair is calculated, and a weight is assigned to the pair whose calculated distance is close to zero.

In addition, the processing unit 30 calculates the distance between the pupil center point and the candidate reflection point, and applies a weight according to the distance. At this time, the processing unit 30 may apply a weight value using a distance function or a Gaussian function.

The processing unit 30 calculates the size (size) of each candidate reflection point because the actual reflection point is small. The processing unit 30 assigns weights according to the sizes of the calculated candidate reflection points.

In this way, the processing unit 30 calculates information about each pair of candidate reflections, and assigns a weight to the information according to the calculated information.

The processing unit 30 adds the weighted candidate reflection point information and selects a pair of candidate reflection points having the minimum value as a pair of reflection points.

Then, the processing unit 30 determines a pixel having the brightest value in the selected reflection point pair as a representative value of each reflection value. The processing unit (30) tracks the line of sight using the detected reflection point and pupil center point.

Figs. 3 and 4 are diagrams showing the relationship between the illumination position and the reflection point position according to the present invention. Fig.

As shown in the figure, the light sources 10a or 10b may be installed at the same height or at different heights.

3, when the first light source 10a and the second light source 10b are disposed at the same height, the positions of the reflection points g1 and g2 reflected in the image also coincide with each other. In other words, for example, when the light sources 10a and 10b are arranged on the same axis as shown in FIG. 3, the reflection points g1 and g2 by the light sources 10a and 10b are also reflected by the light sources 10a and 10b And 10b, respectively. Thus, the processing unit 30 selects a pair of reflection points whose arrangement of candidate reflection points constituting a pair is similar to or more than a certain ratio (for example, 90%) to the actual arrangement of the light sources 10a and 10b.

4, when the first light source 10a and the second light source 10b are installed at different heights, the height (vertical distance) between the first light source 10a and the second light source 10b) The ratio of the height H1 and the width (horizontal distance) W1 and the height H2 and the width W2 of the reflection points formed on the image are the same.

Therefore, the eye-gaze tracking apparatus according to an embodiment of the present invention finds the relative position ratio of the illumination, and then finds a reflection point having the same ratio in the image.

In addition, the present invention can be applied by using the relative positions of the respective lights even when there are three or more light sources.

FIG. 5 is a flowchart illustrating a method of detecting a reflection point of a line-of-sight tracking apparatus according to an exemplary embodiment of the present invention, and FIG. 6 is an exemplary view illustrating a reflection point detection process according to an exemplary embodiment of the present invention.

First, the processing unit 30 acquires a snow image through the camera 20 (S11). At this time, the processing unit 30 controls the illumination 10 to irradiate the infrared rays with the user's face. For example, the eye image is obtained as shown in Fig. 6 (a).

The processing unit 30 detects the pupil in the eye image (S12). The processing unit 30 extracts the eye image of the center of the pupil as shown in FIG. 6 (b).

The processing unit 30 detects the pupil from the eye image and extracts the candidate reflection point from the eye image (S13). The processing unit 30 extracts outlines of pixels having a reference brightness or more in the eye image. Alternatively, as shown in FIG. 6 (c), a high frequency component is detected from the eye image.

The processing unit 30 generates a pair of reflection points that can be composed of certain candidate reflections among the extracted candidate reflections (S14). For example, when there are two light sources, the processing unit 30 selects two candidate reflection points from the extracted candidate reflection points to generate configurable reflection point pairs.

The processing unit 30 calculates information about candidate reflections constituting each pair of reflection points (S15). Here, the information on the candidate reflections includes the distance between the candidate reflections constituting each reflections pair, the distance from the pupil center point to each reflections, and the size of each candidate reflections. The processing section (30) assigns a weight to the information according to the calculated information.

The processing unit 30 selects one reflection point pair among the generated reflection point pairs (S16). The processing unit 30 assigns weights to information about candidate reflections constituting each pair, and selects a reflec- tion point pair having a minimum sum. At this time, the processing unit 30 weights the pair of reflection points composed of candidate reflection points arranged similarly to the arrangement information of the light sources 10a and 10b constituting the illumination 10. [

The processing unit 30 determines the pixel having the brightest value at the reflection point constituting the selected reflection point pair as the representative point of the reflection point (S17).

Then, the processing unit 30 tracks the line of sight using the pupil and the reflection point.

The embodiments described above are those in which the elements and features of the present invention are combined in a predetermined form. Each component or feature shall be considered optional unless otherwise expressly stated. Each component or feature may be implemented in a form that is not combined with other components or features. It is also possible to construct embodiments of the present invention by combining some of the elements and / or features. The order of the operations described in the embodiments of the present invention may be changed. Some configurations or features of certain embodiments may be included in other embodiments, or may be replaced with corresponding configurations or features of other embodiments. It is clear that the claims that are not expressly cited in the claims may be combined to form an embodiment or be included in a new claim by an amendment after the application.

Embodiments in accordance with the present invention may be implemented by various means, for example, hardware, firmware, software, or a combination thereof. In the case of hardware implementation, an embodiment of the present invention may include one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs) field programmable gate arrays, processors, controllers, microcontrollers, microprocessors, and the like.

In the case of an implementation by firmware or software, an embodiment of the present invention may be implemented in the form of a module, a procedure, a function, or the like which performs the functions or operations described above. The software code can be stored in a memory unit and driven by the processor. The memory unit is located inside or outside the processor, and can exchange data with the processor by various known means.

It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit of the invention. Accordingly, the foregoing detailed description is to be considered in all respects illustrative and not restrictive. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

10: Lighting
10a: a first light source
10b: a second light source
20: Camera
30:

Claims (13)

Acquiring an eye image through a camera;
Extracting a candidate reflex point reflected by at least two light sources in the eye image;
Selecting one or more candidate reflex points among the candidate reflex points to generate one or more pairs of reflex points;
Selecting one of the one or more reflection point pairs based on the arrangement information of the two or more light sources;
Determining a representative point of each reflection point constituting the selected reflection point pair,
Wherein the selecting of the pair of reflection points is performed by calculating information on candidate reflection points constituting each pair of reflection points and using the information on the calculated candidate reflection points to select pairs of reflection points A method for detecting a reflection point. The method according to claim 1,
The candidate refracting point extraction step may include:
And extracting an outline of pixels having a reference brightness or more from the eye image. The method according to claim 1,
The reflection point pair generation step includes:
Wherein the number of candidate reflection points constituting the pair of reflection points is determined according to the number of the light sources. The method according to claim 1,
The reflection point pair selection step includes:
Wherein a pair of reflection points constituted by candidate reflection points arranged so as to coincide with arrangement information of the two or more light sources at a predetermined ratio or more is selected. 5. The method of claim 4,
The arrangement information includes:
Wherein the reference point is a reference axis distance between the two or more light sources. 5. The method of claim 4,
The arrangement information includes:
Axis distance between the at least two light sources and the Y-axis distance between the at least two light sources. The method according to claim 1,
Wherein the step of determining representative points of the reflection points comprises:
Wherein a pixel having the brightest value among the candidate reflex points constituting the pair of reflex points is determined as a representative point. A light consisting of two or more light sources emitting infrared rays,
A camera for acquiring an eye image,
A candidate reflection point generated by the illumination is extracted from the eye image, and pairs of the candidate reflection points are grouped into a plurality of pairs of reflection points to generate a pair of reflection points. The arrangement of the candidate reflection points of the pair of reflection points matches the arrangement information of the two or more light sources And a processing unit for selecting a corresponding reflection point pair and determining a representative point of each reflection point constituting the selected reflection point pair,
Wherein the processing unit calculates information on candidate reflections constituting each pair of reflections and uses the information on the calculated reflections to select the pair of reflections. 9. The method of claim 8,
The light source includes:
An infrared ray emitting diode, and an infrared ray lamp. 9. The method of claim 8,
The camera comprises:
And an infrared ray filter. 9. The method of claim 8,
The arrangement information includes:
Wherein the reference point is a reference axis distance between the two or more light sources. 9. The method of claim 8,
The arrangement information includes:
Axis distance and the Y-axis distance between the two or more light sources. 9. The method of claim 8,
Wherein,
Wherein the line of sight is tracked using a representative point of each reflection point and a pupil center point.

Images