KR101643724B1 - Method and Apparatus for identifying eyeball position, Method and Apparatus for tracking eyeball position - Google Patents

Abstract

An eye identification method is disclosed. An eyeball identification method according to an embodiment of the present invention includes obtaining an initial image that is an image of both eyes of the user using a camera positioned between both eyes of a user; Inverting the initial image to generate an inverted image; Generating a first pre-processing image that is processed to separately display positions of both eyes of the user in the initial image using the initial image and the inverted image; And identifying positions of both eyes of the user based on the first pre-processing image.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for identifying an eyeball position,

An embodiment of the present invention relates to eye position identification and tracking, and more particularly, to an eye position identification method and apparatus based on image processing, an eye position tracking method, and an apparatus therefor.

The eye tracking technique is a technique of tracking the movement of the user's eyeball, grasping the user's area of interest or desired motion, and applying the detected information to various fields.

 Conventionally, eye tracking was performed by acquiring an image using a small image device such as a general web camera and then analyzing the obtained image to track the position of the eyeball.

1 is a view illustrating a conventional eye tracking method.

FIG. 1 (a) is a side view of a scene in which a user's eyeball is photographed using one camera. In such a case, the camera is installed to be deflected toward one eyeball of the user, Since the viewing angle of the camera is narrow, the camera may be positioned obliquely below the user's eye in order to obtain the overall appearance of the eyeball, which may cause fatigue of the user.

FIG. 1 (b) shows an embodiment in which two eyes of a user are photographed using two cameras. The use of two cameras increases the price and weight of the eye tracking device .

In addition, although not shown in FIG. 1, an infrared artificial light source may be additionally irradiated to acquire an image to detect an accurate position of the eyeball. This may cause a user to increase eye fatigue and adversely affect the user's eyes during long- There is a problem.

Therefore, there is a need to develop an eye tracking device which is low in cost, light in weight, and able to reduce fatigue of a user.

An object of an embodiment of the present invention is to provide an eyeball discriminating apparatus and a tracking apparatus which are low in cost and light in weight and can reduce fatigue of a user.

According to another aspect of the present invention, there is provided an eyeball discriminating method comprising: obtaining an initial image, which is an image of both eyes of a user, using a camera positioned between both eyes of a user; Inverting the initial image to generate an inverted image; Generating a first pre-processing image that is processed to separately display positions of both eyes of the user in the initial image using the initial image and the inverted image; And identifying positions of both eyes of the user based on the first pre-processing image, wherein the step of generating the first pre-processing image comprises the steps of: The pixel value can be calculated.

(1)

Here, Upper is the pixel value of the initial image, Lower is the pixel value of the inverted image, and MAX_BIT is the maximum pixel value that can be expressed by the corresponding bits of the initial image.

Advantageously, the step of generating the first pre-processing image comprises converting both the ocular regions of the user in the initial image into relatively bright regions and displaying the remaining regions except for both ocular regions of the user relatively dark Region by performing pre-processing to display the first preprocessed image.

Preferably, the method for identifying an eye according to an embodiment of the present invention includes the steps of dividing and displaying positions of both eyes of the user in the first preprocessed image using the first preprocessed image and the inverted image, 2 < / RTI > pre-processing image.

Preferably, the method for discriminating an eye according to an embodiment of the present invention may further comprise compensating for a distortion caused by a characteristic of a lens mounted on the camera in the obtained initial image.

Preferably, the camera may be located in the middle of both eyes of the user.

Preferably, the lens mounted on the camera may be a fisheye lens having a viewing angle capable of photographing both eyes of the user, or a special purpose lens for photographing both eyes.

According to another aspect of the present invention, there is provided an eyeball discriminating apparatus comprising: a camera configured to be positioned between both eyes of a user when attached to a user; An image obtaining unit that obtains an initial image that is an image of both eyes of the user using the camera; An inverted image generating unit for inverting the initial image to generate an inverted image; An image preprocessing unit for generating a first pre-processing image processed to display the positions of both eyes of the user in the initial image by using the initial image and the inverted image; And an eye position identifying unit for identifying positions of both eyes of the user based on the first preprocessed image, and the image preprocessing unit may calculate a pixel value of the first preprocessed image based on the following Equation (1) Can be calculated.

(1)

Here, Upper is the pixel value of the initial image, Lower is the pixel value of the inverted image, and MAX_BIT is the maximum pixel value that can be expressed by the corresponding bits of the initial image.

Preferably, the image preprocessing unit transforms and displays both eye regions of the user in the initial image into a relatively bright region, and displays a region other than the eyeball regions of the user as a relatively dark region Thereby generating the first preprocessed image.

Advantageously, the image preprocessing unit is operable to generate a second pre-processing image that is processed to distinguish and display the positions of both eyes of the user in the first pre-processing image using the first pre-processing image and the inverted image You can do more.

Preferably, the eyeball discriminating apparatus according to an embodiment of the present invention may further include a distortion compensating unit that compensates distortion caused by the characteristics of the lens mounted on the camera in the obtained initial image.

Preferably, the camera may be located in the middle of both eyes of the user.

Preferably, the lens mounted on the camera may be a fisheye lens having a viewing angle capable of photographing both eyes of the user, or a special purpose lens for photographing both eyes.

According to another aspect of the present invention, there is provided an eye tracking method for tracking an eye of a user using a camera positioned between both eyes of a user, ; Inverting the plurality of initial images to generate a plurality of inverted images; Generating a plurality of first pre-processing images processed to display the positions of both eyes of the user in each of the plurality of initial images using the plurality of initial images and the plurality of inverted images; And tracking the position of both eyes of the user based on the plurality of first pre-processing images, wherein the step of generating the first pre-processing image comprises the steps of: 1 pixel value of the preprocessed image can be calculated.

(1)

Here, Upper is the pixel value of the initial image, Lower is the pixel value of the inverted image, and MAX_BIT is the maximum pixel value that can be expressed by the corresponding bits of the initial image.

According to another aspect of the present invention, there is provided an eye tracking apparatus comprising: a camera configured to be positioned between both eyes of a user when attached to a user; An image obtaining unit that obtains a plurality of initial images for the image; An inverted image generating unit for inverting the plurality of initial images to generate a plurality of inverted images; An image preprocessing unit for generating a plurality of first pre-processing images processed to display the position of both eyes of the user in each of the plurality of initial images using the plurality of initial images and the plurality of inverted images; And an eye position tracking unit for tracking the positions of both eyes of the user based on the plurality of first pre-processing images.

Preferably, the step of generating the first preprocessed image may calculate the pixel value of the first preprocessed image based on the following equation (1).

(1)

Here, Upper is the pixel value of the initial image, Lower is the pixel value of the inverted image, and MAX_BIT is the maximum pixel value that can be expressed by the corresponding bits of the initial image.

According to an embodiment of the present invention, it is possible to reduce the cost of the eyeball discriminating apparatus and the tracking apparatus, while making it possible to construct the weight lightly.

According to another embodiment of the present invention, it is possible to detect an eye position without an additional light source, thereby minimizing fatigue of a user.

1 is a view illustrating a conventional eye tracking method.
2 is a view for explaining an eyeball discriminating apparatus according to an embodiment of the present invention
FIG. 3 is a flowchart illustrating an eyeball discriminating method according to an embodiment of the present invention
FIG. 4 is a flowchart illustrating an eyeball discriminating method according to another embodiment of the present invention
5 is a diagram illustrating a configuration of a camera according to an embodiment of the present invention.
6 is a diagram showing a pseudo code representing an eyeball discriminating method according to an embodiment of the present invention.
7 is a view for explaining a pre-processed image according to an embodiment of the present invention.
8 is a view for explaining an eye tracking apparatus according to an embodiment of the present invention.
9 is a flowchart illustrating an eye tracking method according to an embodiment of the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. Like reference numerals are used for like elements in describing each drawing.

The terms first, second, A, B, etc. may be used to describe various elements, but the elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the related art and are to be interpreted as either ideal or overly formal in the sense of the present application Do not.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

2 is a view for explaining an eyeball discriminating apparatus according to an embodiment of the present invention.

2, an eyeball discriminating apparatus 200 according to an embodiment of the present invention includes a camera 210, an image acquiring unit 220, an inverted image generating unit 230, an image preprocessing unit 240, And an identification unit 250.

The camera 210 is configured to be positioned between both eyes of the user upon attachment to the user.

At this time, the camera 210 is positioned at the center of both eyes of the user, so that the center of gravity can be adjusted.

The lens mounted on the camera 210 may be a fisheye lens having a viewing angle capable of photographing both eyes of the user or a special purpose lens for photographing both eyes.

Preferably, the lens mounted on the camera 210 may be a fisheye lens having a viewing angle capable of photographing both eyes of the user, so that both eyes of the user can be photographed through one camera 210, It is possible to reduce the price and weight of the position identifying device 200. [

The image obtaining unit 220 uses the camera 210 to obtain an initial image that is an image of both eyes of the user.

At this time, the initial image may be a monochrome image or a color image.

The inverse image generating unit 230 inverts the initial image to generate an inverse image.

The image preprocessing unit 240 generates a first preprocessed image processed to display the position of both eyes of the user in the initial image by using the initial image and the inverted image.

The eye position identification unit 250 identifies the positions of both eyes of the user based on the first pre-processing image.

As described above, according to an embodiment of the present invention, since the positions of both eyes of a user can be accurately identified using the first preprocessed image generated through image preprocessing, the eye position can be detected without additional light source, Can be minimized.

FIG. 3 is a flowchart illustrating an eyeball discriminating method according to an embodiment of the present invention

In step 310, the eye identification device 200 acquires an initial image of both eyes of the user using a camera located between the eyes of the user.

At this time, the camera mounted on the ocular discrimination apparatus 200 may be configured to be positioned between both eyes of the user when attached to the user.

 In step 320, the eye distinguishing device 200 inverts the initial image to generate an inverted image.

In step 330, the eye identification device 200 generates a first pre-processing image that is processed to separately display the position of both eyes of the user in the initial image using the initial image and the inverted image.

At this time, the ocular distinguishing device 200 performs a pre-processing for displaying both the ocular regions of the user in the initial image into a relatively bright region and displaying the remaining regions except the both ocular regions of the user as a relatively dark region A first preprocessed image can be generated.

For example, the pixel value of the pixels included in the first preprocessed image can be generated by the following Equation (1).

Here, Upper represents the pixel value of the initial image, Lower represents the pixel value of the inverted image, and MAX_BIT represents the maximum pixel value that can be represented by the corresponding bit of the initial image. If the corresponding bit of the initial image is 8 bits, MAX_BIT will be 255, and if it is 16 bits, MAX_BIT will be 65535.

According to Equation (1), pixels having pixel values close to 0 in the initial image have pixel values close to the maximum pixel value in the first preprocessed image, and pixels having pixel values close to the maximum pixel value in the initial image are The preprocessed image has pixel values close to zero.

More specifically, according to Equation (1), only pixels having a pixel value very close to 0 in the initial image have a high pixel value in the first preprocessed image, and are brightly displayed. That is, since the pupil of the user's eyeball region is displayed in black in the initial image, only the pupil of the eyeball region is brightly displayed in the first preprocessed image, and the remaining region is displayed in darkness.

In step 340, the position of both eyes of the user is identified based on the first pre-processing image.

As described above, according to the embodiment of the present invention, since the user's eye area is identified through the image preprocessing, it is possible to identify the user's eye without an additional light source.

FIG. 4 is a flowchart illustrating an eyeball discriminating method according to another embodiment of the present invention

In step 410, the eye identification device 200 acquires an initial image of both eyes of the user using a camera positioned between the eyes of the user.

In step 420, the eye discrimination apparatus 200 compensates for the distortion caused by the characteristics of the camera lens in the acquired initial image.

That is, according to the characteristics of the camera lens, distortion occurs in the image taken by the camera. In this way, the distortion of the image caused by the characteristics of the camera lens is compensated.

Preferably, the eyeball discriminating apparatus 200 according to an embodiment of the present invention detects a distortion that compensates for distortion caused by the characteristics of the lens mounted on the camera 210 in the initial image obtained through the image acquiring unit 220 And may further include a compensation unit (not shown).

On the other hand, in another embodiment, step 420 may be omitted.

In step 430, the eye discrimination apparatus 200 inverts the distortion-compensated initial image to generate an inverted image.

If step 420 is omitted, the eye distinguishing device 200 can reverse-process the first image for which the distortion is not compensated to generate an inverted image.

In step 440, the eye discrimination apparatus 200 generates a first pre-processing image using the distortion-compensated initial image and the reversed image.

In step 450, the eye recognition device 200 generates a second pre-processing image using the first pre-processing image and the inverted image.

At this time, the second pre-processing image can be generated by Equation 1, where the pixel value of the first pre-processing image is substituted for Upper of Equation 1, the pixel value of the inverted image is substituted for Lower, The second preprocessed image can be generated by substituting the maximum pixel value that can be represented by the corresponding bit of the image.

As described above, the second pre-processed image generated using the first pre-processed image and the inverted image is more clearly distinguished from the first pre-processed image.

In step 470, the eye identification device 200 identifies the position of both eyes of the user based on the second pre-processing image.

5 is a diagram illustrating a configuration of a camera according to an embodiment of the present invention.

5 (a) is a side view of a scene in which a user's eyeball is photographed using a camera according to an embodiment of the present invention, and a lens having a wide viewing angle is mounted on a camera according to an embodiment of the present invention Therefore, the camera is photographed on the same line as the eyeball of the user, and the fatigue of the user is reduced.

5B illustrates that the camera according to the embodiment of the present invention is located in the center of both eyes of the user. In the camera according to the embodiment of the present invention, a lens having a wide viewing angle is mounted, The camera will be able to capture both eyes of the user.

As described above, the camera according to an embodiment of the present invention can photograph both eyes of a user with a single camera, thereby reducing the price and weight of the eye recognition device 200.

6 is a diagram showing a pseudo code representing an eyeball discriminating method according to an embodiment of the present invention.

Referring to FIG. 6, the first line indicates that the original image is an 8-bit image.

In the second line, the pixel values of the initial image for both eyes of the user are input.

In the third row, pixel values of the inverse image are generated by subtracting the pixel value of the initial image from the maximum pixel value of 255. [

In the fourth line, pixel values of the first preprocessed image (Burn) are generated using the initial image and the inverted image.

In the fifth line, if the pixel value of the first preprocessed image exceeds 255, the pixel value of the first preprocessed image is set to 255, otherwise, the pixel value of the first preprocessed image is used as it is.

In the sixth line, pixel values of the second preprocessed image DBurn are generated using the first preprocessed image Burn and the inverted image.

In the seventh line, if the pixel value of the second preprocessed image DBurn exceeds 255, the pixel value of the second preprocessed image is set to 255; otherwise, the pixel value of the second preprocessed image is set to 255 As the final pixel value.

7 is a view for explaining a pre-processed image according to an embodiment of the present invention.

7A shows an initial image of both eyes of a user according to an embodiment of the present invention. It can be seen that both eyes of a user are displayed on a single initial image by photographing with one camera.

7B shows a second pre-processing image for both eyes of a user according to an embodiment of the present invention, wherein both ocular regions of the user are brightly displayed and all of the remaining regions are darkly displayed, It can be confirmed that it is clearly identified.

8 is a view for explaining an eye tracking apparatus according to an embodiment of the present invention. 8, an eye tracking apparatus 202 according to an embodiment of the present invention includes a camera 210, an image acquiring unit 220, an inverted image generating unit 230, an image preprocessing unit 240, And a tracking unit 252.

The camera 210 is configured to be positioned between both eyes of the user upon attachment to the user.

The image acquiring unit 220 acquires a plurality of initial images for both eyes of the user at predetermined time intervals using the camera 210. [

The inverse image generating unit 230 inverts a plurality of initial images to generate a plurality of inverse images.

The image preprocessing unit 240 generates a plurality of first preprocessed images processed to display the positions of both eyes of the user in each of the plurality of initial images using a plurality of initial images and a plurality of inverted images.

The eye position tracking unit 250 tracks the positions of both eyes of the user based on the plurality of first pre-processing images.

That is, the eye position tracking unit 250 tracks the positions of both eyes of the user by analyzing a plurality of first pre-processing images by time and analyzing how the position of the user's eye changes.

9 is a flowchart illustrating an eye tracking method according to an embodiment of the present invention.

In step 910, the eye tracking device 202 acquires a plurality of initial images for both eyes of the user at predetermined time intervals using a camera located between both eyes of the user.

In step 920, the eye tracking device 202 inverts a plurality of initial images to generate a plurality of inverted images.

In step 930, the eye tracking device 202 uses a plurality of initial images and a plurality of reversed images to generate a plurality of first pre-processing images processed to display the position of both eyes of the user in each of the plurality of initial images .

In step 940, the eye tracking device 202 tracks the position of both eyes of the user based on the plurality of first pre-processing images.

The above-described embodiments of the present invention can be embodied in a general-purpose digital computer that can be embodied as a program that can be executed by a computer and operates the program using a computer-readable recording medium.

The computer readable recording medium includes a magnetic storage medium (e.g., ROM, floppy disk, hard disk, etc.), optical reading medium (e.g., CD ROM, DVD, etc.).

The present invention has been described with reference to the preferred embodiments. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

Claims (12)

Obtaining an initial image that is an image of both eyes of the user using a camera positioned between both eyes of the user;
Inverting the initial image to generate an inverted image;
Generating a first pre-processing image that is processed to separately display positions of both eyes of the user in the initial image using the initial image and the inverted image; And
Identifying the position of both eyes of the user based on the first pre-processing image,
The step of generating the first pre-
Wherein the pixel value of the first preprocessed image is calculated based on the following equation (1).
(1)

Here, Upper is the pixel value of the initial image, Lower is the pixel value of the inverted image, and MAX_BIT is the maximum pixel value that can be expressed by the corresponding bits of the initial image. The method according to claim 1,
The step of generating the first pre-
Processing the image of both eyes of the user in the initial image by converting the image into a relatively bright region and displaying the remaining region except for both ocular regions of the user as a relatively dark region, Wherein the at least one eye distinguishes the at least one eye. The method according to claim 1,
Further comprising generating a second pre-processing image processed to display the positions of both eyes of the user in the first pre-processing image using the first pre-processing image and the inverted image, Identification method. The method according to claim 1,
The camera
Wherein the eyeball is positioned at the center of both eyes of the user. The method according to claim 1,
The lens mounted on the camera
A fisheye lens having a viewing angle at which both eyes of the user can be photographed or a special-purpose lens for photographing both eyes. A camera configured to be positioned between both eyes of the user upon attachment to the user;
An image obtaining unit that obtains an initial image that is an image of both eyes of the user using the camera;
An inverted image generating unit for inverting the initial image to generate an inverted image;
An image preprocessing unit for generating a first pre-processing image processed to display the positions of both eyes of the user in the initial image by using the initial image and the inverted image; And
And an eye position identifying unit for identifying positions of both eyes of the user based on the first pre-processing image,
The image pre-
Wherein the pixel value of the first preprocessed image is calculated on the basis of the following equation (1).
(1)

Here, Upper is the pixel value of the initial image, Lower is the pixel value of the inverted image, and MAX_BIT is the maximum pixel value that can be expressed by the corresponding bits of the initial image. The method according to claim 6,
The image pre-
Processing the image of both eyes of the user in the initial image by converting the image into a relatively bright region and displaying the remaining region except for both ocular regions of the user as a relatively dark region, Wherein the eye recognition device generates an eyeball. The method according to claim 6,
The image pre-
Further comprising the step of generating a second pre-processing image processed to display the positions of both eyes of the user in the first pre-processing image using the first pre-processing image and the inverted image, Identification device. The method according to claim 6,
The camera
Wherein the eyeball is positioned at the center of both eyes of the user. The method according to claim 6,
The lens mounted on the camera
A fisheye lens having a viewing angle at which both eyes of the user can be photographed, or a special-purpose lens for photographing both eyes. Acquiring a plurality of initial images that are a plurality of images for both eyes of the user at predetermined time intervals using a camera positioned between both eyes of the user;
Inverting the plurality of initial images to generate a plurality of inverted images;
Generating a plurality of first pre-processing images processed to display the positions of both eyes of the user in each of the plurality of initial images using the plurality of initial images and the plurality of inverted images; And
And tracking the position of both eyes of the user based on the plurality of first pre-processing images,
The step of generating the first pre-
Wherein the pixel value of the first preprocessed image is calculated based on the following equation (1).
(1)

Here, Upper is the pixel value of the initial image, Lower is the pixel value of the inverted image, and MAX_BIT is the maximum pixel value that can be expressed by the corresponding bits of the initial image. A camera configured to be positioned between both eyes of the user upon attachment to the user;
An image obtaining unit that obtains a plurality of initial images, which are a plurality of images for both eyes of the user, at predetermined time intervals using the camera;
An inverted image generating unit for inverting the plurality of initial images to generate a plurality of inverted images;
An image preprocessing unit for generating a plurality of first pre-processing images processed to display the position of both eyes of the user in each of the plurality of initial images using the plurality of initial images and the plurality of inverted images; And
And an eye position tracking unit for tracking the positions of both eyes of the user based on the plurality of first pre-processing images,
The image pre-
And calculates a pixel value of the first preprocessed image based on the following equation (1).
(1)

Here, Upper is the pixel value of the initial image, Lower is the pixel value of the inverted image, and MAX_BIT is the maximum pixel value that can be expressed by the corresponding bits of the initial image.

Images