KR20220036449A - Image-based pupil detection method - Google Patents

Abstract

The present invention relates to an image-based pupil detection method. In more detail, it is possible to improve the reliability of pupil measurement by accurately detecting the pupil even if the pupil is distorted through reflected light or occlusion of the eyelids in the eyeball image input through a camera, in the image-based pupil detection method.

Description

Image-based pupil detection method

The present invention relates to an image-based pupil detection method, and more particularly, to accurately detect the pupil even if the pupil is distorted through reflected light or eyelid occlusion in an eyeball image input through a camera, thereby improving the reliability of pupil measurement. It relates to an image-based pupil detection method.

The pupil is a circular empty space located at the center of the iris, and controls the amount of light reaching the retina in order to maximize visual perception, and reduces chromatic and spherical aberration by increasing the depth of focus.

Recently, the pupil is used for eye tracking, pupil examination, and user authentication for tracking the user's gaze, and in order to be utilized in these fields, a pupil measurement technology capable of automatically detecting and measuring the pupil is required.

Conventionally, in order to detect the pupil, a method of detecting the boundary line between the iris and the pupil and then searching for the pupil has been used. When using images of

In addition, the current pupil detection technology has a problem in that measurement accuracy is lowered due to reflected light reflected in the eye, occlusion of the eyelids, and rapid pupil size change.

KR10-2013275 B1 "Pupillary detection device and pupil detection method" KR10-2015-0081680 A "Pupillary detection method, computer readable storage medium recording the method, and pupil detection apparatus KR10-2014-0106926 A "Pupillary detection device and pupil detection method"

The present invention has been devised to solve the above problems, and an object of the present invention is to accurately detect a pupil in an eyeball image despite reflected light reflected in the eye, an eyelid occlusion phenomenon, and a sudden change in pupil size, thereby improving the accuracy of pupil measurement. An object of the present invention is to provide an image-based pupil detection method that can be used for tracking, pupil inspection, and user authentication, and can reduce processing time for pupil detection of high-resolution images.

In order to achieve the above object, the present invention comprises the steps of: generating a filter bank including binarized images in which circles of different sizes are generated; receiving an eyeball image captured by a camera; extracting an expected pupil position (hereinafter, referred to as an “expected pupil region”) by calculating the eyeball image with the binarized images of the filter bank; and extracting boundary regions having an elliptical shape from the predicted pupil region and determining a final pupil position (hereinafter referred to as “actual pupil region”) by comparing it with a preset condition among the boundary regions. It provides an image-based pupil detection method, characterized in that.

In a preferred embodiment, the generating of the filterbank comprises: generating a plurality of image backgrounds having various image sizes; projecting circles of various sizes based on the respective center coordinates of the image backgrounds; and a first filter bank including images in which a pixel value of a region corresponding to the inside of the projected circle on the image backgrounds is “0” and a pixel value of a region other than “255” is binarized and the projection of the image backgrounds and generating a second filter bank including images in which the pixel value of the inner circle is set to “255” and the pixel value of the other region is set to “0”.

In a preferred embodiment, the step of extracting the predicted pupil region: generates first convolutional images by performing a convolution operation on the eyeball image with the binarized images of the first filter bank, respectively, and converts the eyeball image to the first generating second convolutional images by performing a convolution operation with each of the binarized images of two filter banks; inverting the second convolutional images; extracting multiplied images by multiplying the first convolutional image and the inverted second convolutional image using the binarized image of the same size in the image generating step of the convolutional images; The pixel coordinates (hereinafter referred to as "effective coordinates") of the multiplication image (hereinafter referred to as "effective multiplication image") having the highest brightness value among the multiplication images and the binarized image used to extract the effective multiplication image extracting the size of (hereinafter referred to as "effective binarized image"); and setting pixel coordinates corresponding to the effective coordinates in the eyeball image as central coordinates of the effective binarized image, and extracting an image size of the effective binarized image as an expected pupil region from the eyeball image.

In a preferred embodiment, the extracting of the actual pupil region comprises: extracting a contour image and a gradient image of the extracted expected pupil region; detecting boundary regions having an elliptical shape from the outline image; counting the number of pixels having a gradient value greater than or equal to a preset value in pixel coordinates of the gradient image corresponding to the boundary regions; extracting a boundary region having the largest number of counted pixels; and determining whether or not the extracted boundary region corresponds to a preset condition, extracting the boundary region as an actual pupil region if the boundary region meets the preset condition, and detecting the boundary regions if not suitable Repeat from step.

In a preferred embodiment, the gradient image includes a vertical gradient image differentiated in a vertical direction and a horizontal gradient image differentiated in a horizontal direction.

In a preferred embodiment, in the suitability determination step, the ratio between the major axis and the minor axis of the boundary area is less than or equal to a set ratio, the size falls within a predetermined range, and the change in the center coordinate and size of the boundary area within a certain frame is within an error range It is judged as suitable if applicable within the

In addition, the present invention further provides a computer program stored in a medium for performing the image-based pupil detection method in combination with a computer.

The present invention has the following excellent effects.

According to the image-based pupil detection method of the present invention, the predicted pupil region is primarily detected from the input eye image using a filter bank including binarized images projected with circles of various sizes, and then a contour image of the predicted pupil region Since the final pupil position is detected through the and gradient image, the pupil can be accurately detected regardless of the irregular shape of the pupil as well as the reflected light, which has the advantage of improving the reliability of pupil measurement.

In addition, according to the image-based pupil detection method of the present invention, since the operation for extracting the actual pupil region is performed after first detecting the expected pupil region, there is an advantage in that unnecessary computational work and erroneous detection can be minimized. .

1 is a flowchart of an image-based pupil detection method according to an embodiment of the present invention;
2 is a flowchart for explaining a process of extracting an expected pupil region according to an embodiment of the present invention;
3 is a schematic diagram for explaining a process of extracting an expected pupil region according to an embodiment of the present invention;
4 is a flowchart for explaining an actual pupil region determination process according to an embodiment of the present invention;
5 is a schematic diagram for explaining an actual pupil region determination process according to an embodiment of the present invention.

As for the terms used in the present invention, general terms that are currently widely used are selected as possible, but in certain cases, there are also terms arbitrarily selected by the owner. should be taken into account to understand its meaning.

Hereinafter, the technical configuration of the present invention will be described in detail with reference to preferred embodiments shown in the accompanying drawings.

However, the present invention is not limited to the embodiments described herein and may be embodied in other forms. Like reference numerals refer to like elements throughout.

1 is a flowchart of an image-based pupil detection method according to an embodiment of the present invention, FIG. 2 is a flowchart for explaining an expected pupil region extraction process according to an embodiment of the present invention, and FIG. 3 is an embodiment of the present invention A schematic diagram for explaining a process of extracting an expected pupil region according to , FIG. 4 is a flowchart for explaining a process for determining an actual pupil region according to an embodiment of the present invention, and FIG. 5 is an actual pupil region according to an embodiment of the present invention It is a schematic diagram for explaining the judgment process.

1 to 5 , the image-based pupil detection method according to an embodiment of the present invention is a method for accurately detecting a pupil position by receiving an eyeball image captured by a camera.

In addition, the image-based pupil detection method according to an embodiment of the present invention is performed by a computer, and the computer stores a computer program for operating the computer to perform the image-based pupil detection method.

In addition, the computer is not only a general personal computer, but also a server computer accessible through a communication network, a cloud system, a smart phone, a smart device such as a tablet PC, a security camera, etc. It is a computer in a broad sense including embedded systems.

In addition, the computer program may be provided by being stored in a separate recording medium, and the recording medium may be specially designed and configured for the present invention or may be known and available to a person skilled in the art of computer software. .

For example, the recording medium includes a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as a CD and DVD, a magneto-optical recording medium capable of both magnetic and optical recording, ROM, RAM, and flash memory. and the like, alone or in combination, may be a hardware device specially configured to store and execute program instructions.

In addition, the computer program may be a program composed of program instructions, a local data file, a local data structure, etc. alone or in combination, and may be executed by a computer using an interpreter as well as machine code such as generated by a compiler It can be a program written in a high-level language code

In addition, the image-based pupil detection method according to an embodiment of the present invention largely includes the steps of generating a filter bank (S1000), receiving an eyeball image (S2000), and a pupil position expected from the eyeball image (hereinafter, “expected”). Extracting the pupil region (referred to as “,”) (S3000) and determining the final pupil position (hereinafter, referred to as “actual pupil region”) from the expected pupil region (S4000).

The image-based pupil detection method first creates a filter bank (S1000).

The filter bank of the present invention means a set of binarized images necessary for extracting the predicted pupil region, which will be described below. In order to generate the filter bank, a plurality of image backgrounds having various sizes are first generated (S1100). ).

Next, one circle is projected for each image background based on the respective center coordinates of the plurality of image backgrounds (S1200).

In this case, circles of different sizes may be formed for each image background, and the size is not limited.

Next, a filter bank is generated by extracting and collecting an image obtained by binarizing the image backgrounds on which the circle is projected (S1300).

In detail, as opposed to the first filter bank including images in which the pixel value of the region corresponding to the inside of the circle projected on the image backgrounds is “0”, and the pixel value of the region is “255”, in contrast to the first filter bank A second filter bank including images in which the pixel value of the region corresponding to the inside of the circle projected on the image backgrounds is “255” and the pixel value of the region is “0” is generated.

That is, the filter bank has two filter banks, and each filter bank has binarized images having various image sizes as well as a circle size, and an inner region of the circle has a value of “0” or “255”.

Next, after the filter bank is created, an eyeball image captured by the camera is received (S2000).

The camera may be a CAM connected to a computer, a smart device, a medical camera, a head mount, etc., and the eyeball image may be extracted from a user's face detected through the camera through a separate eye detection algorithm.

Next, an expected pupil region is extracted by calculating the inputted eye image with the binarized images of the filter bank (S3000).

In order to extract the predicted pupil region, first convolutional operations are performed on the eyeball image with the binarized images of the first filter bank to generate first convolutional images, and the eyeball image is converted into the second filter bank. A convolution operation is performed on each of the binarized images to generate second convolutional images (S3100).

Accordingly, first convolutional images emphasizing the characteristic of the pupil position are generated in the generated first filter bank, and second convolution images in which the characteristics of the eye-guk region other than the pupil position are emphasized are generated in the second filter bank.

On the other hand, before performing the convolution operation, it is preferable to perform a padding operation on the eyeball image in order to output an image size in which the result of the convolution operation is the same as the size of the eyeball image.

Next, the second convolution images are inverted, and multiplication images are extracted by multiplying the first convolution images and the second convolution images ( S3200 ).

In detail, in the step of generating the convolutional image ( S3100 ), a first convolutional image using a binarized image of the same image size and an inverted second convolutional image are multiplied by each other.

On the other hand, by inverting the second convolutional image, the existing dark part becomes bright and the bright part becomes dark, and the characteristic of the pupil region is emphasized. There is a characteristic that an area appears brighter than other areas of the eyeball.

Next, to extract the pixel coordinates (hereinafter referred to as "effective coordinates") of the multiplication image (hereinafter referred to as "effective multiplication image") having the highest pixel value among the extracted multiplication images and the effective multiplication image The image size of the used binarized image (hereinafter, referred to as "effective binarized image") is extracted (S3300).

Next, pixel coordinates corresponding to the effective coordinates in the eyeball image are set as the center coordinates of the effective binarized image, and an image size of the effective binarized image is extracted as an expected pupil region from the eyeball image (S3400).

After the expected pupil region is extracted, the actual pupil region is determined (S4000).

In detail, an edge image and a gradient image of the predicted pupil region are extracted (S4100).

Here, for the outline image, it is preferable to use a Canny algorithm among various outline detection algorithms.

In addition, the contour image may be subjected to a morphology operation for removing noise such as unnecessary circles and short lines of a predetermined size in the image.

In addition, the gradient image includes a vertical gradient image obtained by differentiating the expected pupil area in a vertical direction and a horizontal gradient image obtained by differentiating the expected pupil area in a horizontal direction, respectively.

Also, the gradient image may be extracted using a Sobel algorithm, a Prewitt algorithm, a ROberts algorithm, a laplacian algorithm, or the like.

Next, boundary regions having an elliptical shape are detected from the outline image (S4200).

Here, as the algorithm for detecting the shape of the ellipse, various known techniques such as an algorithm based on hub transformation may be used, and the algorithm for detecting the ellipse is not limited.

Next, the number of pixels having a gradient value greater than or equal to a preset value in pixel coordinates of the gradient image corresponding to the boundary regions is counted (S4300), and a boundary region having the largest number of counted pixels is extracted (S4400).

Next, it is determined whether the extracted boundary region corresponds to a preset condition, and the actual pupil region is detected (S4500).

Here, preset conditions for determining suitability of the extracted boundary region are as follows.

1) The ratio of the major axis and the minor axis of the extracted boundary area is less than or equal to a preset ratio

2) The size of the extracted boundary area falls within a preset range

3) The change in the center coordinates and size of the boundary area extracted within a certain frame falls within a preset error range

That is, it is determined whether the extracted boundary region corresponds to a preset condition (S4510), and if all conditions are satisfied, it is determined to be suitable, and the extracted boundary region is extracted as an actual pupil region (S4520).

If at least one of the preset conditions is not met, the process of detecting the boundary regions is repeatedly performed.

Therefore, in the image-based pupil detection method of the present invention, unnecessary operation and erroneous detection can be minimized because the predicted pupil region is primarily detected and the operation to find the actual pupil region from the detected predicted pupil region is performed.

In addition, after detecting the elliptical shape from the outline image in the expected pupil region, the position corresponding to the actual pupil region is detected using the size and ratio of the elliptical shape, the number of pixels including a gradient value over a certain value, etc. However, it is possible to accurately detect the pupil regardless of the irregular shape of the pupil, which has the advantage of improving the reliability of pupil measurement.

In addition, when the size and state of the pupil is measured based on the actual pupil region detected through the image-based pupil detection method of the present invention, the accuracy of pupil detection is high, so that high reliability of the pupil measurement result can be obtained.

As described above, the present invention has been illustrated and described with reference to preferred embodiments, but it is not limited to the above-described embodiments, and those of ordinary skill in the art to which the present invention pertains within the scope not departing from the spirit of the present invention Various changes and modifications will be possible.

Claims (7)

generating a filter bank including binarized images in which circles of different sizes are generated;
receiving an eyeball image captured by a camera;
extracting an expected pupil position (hereinafter, referred to as an “expected pupil region”) by calculating the eye image with the binarized images of the filter bank; and
extracting boundary regions having an elliptical shape from the predicted pupil region, and determining a final pupil position (hereinafter referred to as “actual pupil region”) by comparing it with a preset condition among the boundary regions; An image-based pupil detection method characterized in that.
The method of claim 1,
Creating the filter bank:
generating a plurality of image backgrounds having various image sizes;
projecting circles of various sizes based on the respective center coordinates of the image backgrounds; and
A first filter bank including images in which the pixel value of a region corresponding to the inside of the projected circle on the image backgrounds is “0” and pixel values of other regions are “255” and the projected images of the image backgrounds Creating a second filter bank including images in which the pixel value inside the circle is set to “255” and the pixel value in the other region is binarized to “0”; Pupil detection method
3. The method of claim 2,
Extracting the predicted pupil region:
First convolution operations are performed on the eyeball image with the binarized images of the first filter bank, respectively, to generate first convolutional images, and convolution operation is performed on the eyeball image with the binarized images of the second filter bank, respectively. 2 generating convolutional images;
inverting the second convolutional images;
extracting multiplied images by multiplying the convolutional images by multiplying a first convolutional image using a binarized image of the same size and an inverted second convolutional image in the image generating step;
The pixel coordinates (hereinafter referred to as "effective coordinates") of the multiplication image (hereinafter referred to as "effective multiplication image") having the highest brightness value among the multiplication images and the binarized image used to extract the effective multiplication image extracting the size of (hereinafter, referred to as "effective binarization image"); and
setting pixel coordinates corresponding to the effective coordinates in the eyeball image as the center coordinates of the effective binarized image, and extracting an image size of the effective binarized image as an expected pupil region from the eyeball image. An image-based pupil detection method.
4. The method of claim 3,
Extracting the real pupil region:
extracting a contour image and a gradient image of the extracted predicted pupil region;
detecting boundary regions having an elliptical shape from the outline image;
counting the number of pixels having a gradient value greater than or equal to a preset value in pixel coordinates of the gradient image corresponding to the boundary regions;
extracting a boundary region having the largest number of counted pixels; and
Including; determining suitability whether the extracted boundary area corresponds to a preset condition;
An image-based pupil detection method, characterized in that if the extracted boundary region is suitable for a preset condition, it is extracted as an actual pupil region, and if not suitable, the step of detecting the boundary region is repeatedly performed
5. The method of claim 4,
The gradient image is an image-based pupil detection method, characterized in that it includes a vertical gradient image differentiated in a vertical direction and a horizontal gradient image differentiated in a horizontal direction
5. The method of claim 4,
In the suitability determination step, when the ratio of the major axis and the minor axis of the extracted boundary area is less than or equal to a set ratio, the size falls within a preset range, and the change in the center coordinate and size of the extracted boundary area within a certain frame falls within the error range Image-based pupil detection method, characterized in that it is judged as suitable
A computer program stored in a medium for performing the image-based pupil detection method of any one of claims 1 to 6 in combination with a computer.

Images