KR102583280B1 - method for detection pupil by using a visual-light camera and system using the method - Google Patents

Abstract

A pupil detection method and system using a visible light camera are disclosed. The disclosed method detects facial landmarks of the left and right eyes from facial images acquired from a subject using a camera, calculates the distance between the centers of the left and right eyes from the center coordinates of the left and right eyes extracted using the facial landmarks of the left and right eyes, and , Based on the distance between the centers of the eyes, a reference face image of the face area including the left and right eyes and the side parts of the face on both sides of the left and right eyes is selected, and the facial feature points of the left and right eyes are detected from the reference face image to create a region of interest ( It includes the steps of extracting an eye region (ROI), detecting both pupils in the eye region by a circular detection method for the eye region, and extracting center coordinates of the both pupils.

Description

Pupil detection method and system using a visible light camera {method for detection pupil by using a visual-light camera and system using the method}

The present disclosure relates to a pupil detection method and a system for applying the same, and more specifically, to a pupil detection method using an RGB visible light camera and a system for applying the same.

A person's gaze information is an objective indicator for determining where an individual focuses, and is used in marketing, games, and AR (Augmented Reality). It is widely used for various purposes such as market research, content development, and interface development in various fields such as VR (Virtual Reality) and HCI (Human-Computer Interaction).

As a commonly used visible light or RGB camera, the technology for extracting human gaze information through a small web camera is attracting a lot of attention as it can extract gaze information of many people even in a non-face-to-face environment and is suitable for commercializing eye tracking technology. I'm receiving it.

Existing pupil movement detection methods used special cameras such as infrared (IR) cameras. This method has a problem in that it cannot be applied to a general RGB camera, so it cannot be applied to portable devices such as PCs or smartphones equipped with an RGB camera.

Therefore, research is required on how to accurately detect the pupil using RGB cameras, which are widely used for general purposes.

KR 1021658310000 B KR 1020200019162 A KR 1017528730000 B KR 1019993180000 B KR 1020170120546 B KR 1020180095431 B

Beatty, J., and Kahneman, D. (1966). Pupillary changes in two memory tasks, Psychonomic Science, 5(10), 371-372. Beatty, J., and Lucero-Wagoner, B. (2000). The pupillary system, Handbook of Psychophysiology, 2, 142-162. Chen, S., Epps, J., and Chen, F. (2013). An investigation of pupil-based cognitive load measurement with low cost infrared webcam under light reflex interference, In Engineering in Medicine and Biology Society (EMBC), 2013 35th Annual International Conference of the IEEE, 3202-3205. Daugman, J. (2004). How iris recognition works, Circuits and Systems for Video Technology, IEEE Transactions on, 14(1), 21-30. Fotiou, F., Fountoulakis, K. N., Tsolaki, M., Goulas, A., and Palikaras, A. (2000). Changes in pupil reaction to light in Alzheimer's disease patients: a preliminary report, International journal of psychophysiology, 37(1), 111-120. Hakerem, G. A. D., and Sutton, S. (1966). Pupillary response at visual threshold, Nature, 212(5016), 485-486. Heller, P. H., Perry, F., Jewett, D. L., and Levine, J. D. (1990). Autonomic components of the human pupillary light reflex, Investigative Ophthalmology and Visual science, 31(1), 156-162. Hess, E. H., and Polt, J. M. (1964). Pupil size in relation to mental activity during simple problem-solving, Science, 143(3611), 1190-1192. Just, M. A., and Carpenter, P. A. (1993). The dimension of intensity of thought: Pupilometric indices of sentence processing. Canadian Journal of Experimental Psychology/Revue Canadienne de Psychologie Experimentale, 47(2), 310. Kahneman, D., and Beatty, J. (1966). Pupil diameter and load on memory, Science, 154(3756), 1583-1585. Klingner, J., Kumar, R., and Hanrahan, P. (2008). Measuring the task-evoked pupillary response with a remote eye tracker, In Proceedings of the 2008 symposium on Eye tracking research and applications, 69-72. Kojima, M., Shioiri, T., Hosoki, T., Kitamura, H., Bando, T., and Someya, T. (2004). Pupillary light reflex in panic disorder, European Archives of Psychiatry and Clinical Neuroscience, 254(4), 242-244. Loewenfeld, I. E., and Lowenstein, O. (1993). The pupil: Anatomy, physiology, and clinical applications (Vol. 2), Wiley-Blackwell. Partala, T., and Surakka, V. (2003). Pupil size variation as an indication of affective processing, International Journal of Human-Computer Studies, 59(1), 185-198. Steinhauer, S. R., Condray, R., and Kasparek, A. (2000). Cognitive modulation of midbrain function: task-induced reduction of the pupillary light reflex, International Journal of Psychophysiology, 39(1), 21-30. Verney, S. P., Granholm, E., and Dionisio, D. P. (2001). Pupillary responses and processing resources on the visual backward masking task, Psychophysiology, 38(01), 76-83. Bebis, G., Georgiopoulos, M., da Vitoria Lobo, N., and Shah, M. (1999). Learning affine transformations. Pattern recognition, 32(10), 1783-1799. Yuen, H. K., Princen, J., Illingworth, J., and Kittler, J. (1990). Comparative study of Hough transform methods for circle finding. Image and vision computing, 8(1), 71-77.

This disclosure proposes a method and system that can precisely detect and track the pupil using a visible light camera.

The present disclosure proposes a method and system that can detect pupil movement with high precision by accurately inferring the center coordinates of the pupil not only near but also at a distance based on face detection and facial feature point detection.

Pupil detection method according to the present disclosure:

Obtaining a facial image by photographing the subject's face using a camera;

Detecting facial landmarks of left and right eyes from the face image by an image processing unit;

extracting center coordinates of the left and right eyes using the facial feature points of the left and right eyes by an analysis unit;

calculating the distance between the centers of the left and right eyes from the coordinates of the centers of the left and right eyes, and selecting a reference face image of a face area including the left and right eyes and side parts of the face on both sides of the left and right eyes based on the distance between the centers of the eyes;

extracting an eye region, which is a region of interest (ROI), by detecting facial feature points of the left and right eyes from the reference face image;

detecting both pupils in the eye region by a circular detection method for the eye region; and

It includes: extracting center coordinates of both pupils.

According to one or more embodiments,

In the face image selection step, the tilt angle ( θ ) of the face is calculated using the coordinates of the pupil centers of both sides,

A facial image alignment step of aligning the facial area by rotating the facial image according to the tilt angle may be further included.

According to one or more embodiments,

The reference face image may be converted to an image of a predetermined size by scaling the reference face image to an image of a preset size based on the distance between the centers of the eyes, and then the region of interest (ROI) may be extracted.

According to one or more embodiments,

The reference face image is divided into 3 vertically and horizontally with an A:B:A ratio (A and B are the same or different positive numbers), and includes a unit area divided into 9 grids as a whole,

Among the 9 divided unit areas on the grid, the centers of both eyes may be located at the coordinates of the two corners on both upper sides of the center area at the exact center or inside the center area.

According to one or more embodiments,

The reference face image may be divided into 9 segments of 1:1:1 in the vertical and horizontal directions, so that the 9 segmented unit areas may have a square structure of the same size.

According to one or more embodiments,

The pupil detection method obtains a scaling factor based on the distance between the centers of the eyes, and scales the reference face image into an image of a preset size using the scaling factor to convert the reference face image into an image of a predetermined size.

According to one or more embodiments,

The Hough Circle detection method may be applied to detect the pupil.

A pupil detection system according to the present disclosure that performs the above method:

A visible light camera that captures images of the subject's face;

an image processing unit that detects facial landmarks of left and right eyes from the face image;

The center coordinates of the left and right eyes are extracted using the facial feature points of the left and right eyes, a reference face image of the face area is selected using this, an eye area of a region of interest (ROI) is extracted from the reference face image, and the eye area is extracted. It includes an analysis unit that detects the pupil and extracts center coordinates from the pupil.

According to one or more embodiments,

The analysis unit may calculate a tilt angle ( θ ) of the face using the center coordinates of both eyes, and align the facial area by rotating the reference face image according to the tilt angle.

According to one or more embodiments,

The analysis unit may extract the region of interest from a reference face image scaled to an image of a preset size based on the distance between the centers of the eyes.

In a pupil detection system according to one or more embodiments,

The reference face image is divided into 3 vertically and horizontally with an A:B:A ratio (A and B are the same or different positive numbers), and includes a unit area divided into 9 grids as a whole,

Among the 9 divided unit areas on the grid, the centers of both eyes may be located at the coordinates of the two corners on both upper sides of the central area in the exact center.

In a pupil detection system according to one or more embodiments,

The reference face image may be divided into 9 segments of 1:1:1 in the vertical and horizontal directions, so that the 9 segmented unit areas may have a square structure of the same size.

In a pupil detection system according to one or more embodiments,

The analysis unit may apply the Hough Circle detection method to detect the pupil.

Figure 1 shows a comparison of face images from an IR camera and an RGB camera.
Figure 2 (A) shows a state in which a square face area was detected in an upper body image taken from a subject.
Figure 2 (B) shows a state in which a landmark is detected in a square face area obtained from the subject's upper body image.
Figure 3a exemplarily shows an output image obtained by face alignment with respect to an input image in an embodiment of the present invention.
Figure 3b shows the result of scaling at a 1:1:1 ratio for aligned images in an embodiment of the present invention.
Figure 4a illustrates an aligned face image (input) and a face image (output) from which an ROI is detected, in one embodiment.
FIG. 4B is a partial enlarged photograph of a face image for illustrating extraction of an eye region, according to one embodiment.
Figure 5 schematically shows the entire process of extracting a pupil from an ROI, in one embodiment.
Figure 6 illustrates an example of a specific method for detecting a circle corresponding to a pupil from a ROI image, in one embodiment.
Figure 7 explains circularity detection by Hough Gridient among the circularity detection methods in one embodiment.
Figure 8 is a block diagram showing the schematic configuration of an RGB image-based pupil detection system according to an embodiment of the present disclosure.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the embodiments of the present invention concept may be modified into various other forms, and the scope of the present invention concept should not be construed as being limited to the embodiments described in detail below. It is preferable that the embodiments of the present invention be interpreted as being provided to more completely explain the present invention to a person with average knowledge in the art. Identical symbols refer to identical elements throughout. Furthermore, various elements and areas in the drawings are schematically drawn. Accordingly, the inventive concept is not limited by the relative sizes or spacing depicted in the accompanying drawings.

Terms such as first, second, etc. may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component, and conversely, a second component may be named a first component without departing from the scope of the present invention concept.

The terms used in this disclosure are merely used to describe specific embodiments and are not intended to limit the inventive concept. Singular expressions include plural expressions unless the context clearly dictates otherwise. In the present disclosure, expressions such as “comprises” or “has” are intended to indicate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the present disclosure, but are not intended to indicate the presence of one or more other features. It should be understood that it does not exclude in advance the possibility of the existence or addition of elements, numbers, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical terms and scientific terms, have the same meaning as commonly understood by those skilled in the art in the technical field to which the concept of the present invention pertains. Additionally, commonly used terms, as defined in dictionaries, should be interpreted to have meanings consistent with what they mean in the context of the relevant technology, and should not be used in an overly formal sense unless explicitly defined herein. It will be understood that this is not to be interpreted.

One or more embodiments described below present a method of detecting a pupil using a visible light camera and extracting pupil movement information through this, and a system for applying the same.

Unlike the conventional method of extracting pupil movement data by photographing the pupil using an existing IR camera, the present disclosure uses a visible light camera, that is, an RGB camera, to infer the center coordinates of the pupil and use these to determine the pupil's movement data. We present a method and system that can accurately track movement.

Figure 1 shows a comparison of face images from an IR camera and an RGB camera.

As shown in Figure 1, the eye image obtained by an RGB camera not only has more noise than the eye image obtained by an IR camera, but also the position and size of the eye are different for each individual, and the iris and The division of the pupil is not clear, making it difficult to track the pupil.

According to the present invention, as understood from the description of the embodiment below, a face image is acquired by an RGB camera, the center coordinates of the eye are extracted based on face detection and facial feature point detection, and the center coordinates of the pupil are inferred using this. In practice, it is possible to extract pupil movement information and apply it to various application fields. This method of the present invention is processed and analyzed by a system equipped with a vision system equipped with a web camera.

This system of the present invention was performed by a computer-based analysis system including imaging equipment including an RGB video camera, an image processing unit that processes the captured video, and an analysis unit that analyzes the video, and the analysis unit includes software including an analysis algorithm. It may include analysis tools provided by .

Figure 8 is a block diagram showing the schematic configuration of an RGB image-based pupil detection system according to an embodiment of the present disclosure.

As shown in Figure 8, the pupil detection system detects the upper body or face of the subject or user (P1) from a visible light camera 110, an image processor 120 that processes the image acquired from the camera 110, and an image processor. It is provided with an analysis unit 130 that extracts pupil information by detecting facial feature points, and a display 140 that shows the analyzed results to the user.

The analysis unit 130 is mainly composed of software including an image information analysis algorithm to be described later, and may include some hardware as a component.

An embodiment of the pupil detection method according to the present disclosure performed by the above system largely includes the following three steps.

I. Face Alignment: Find and align the face area in the subject's upper body image and create an aligned face image.

II. ROI Extraction: Extract the eye area, which is the region of interest, from the aligned face image.

III. Pupil Detection: Detects the pupil within the extracted eye area.

By extracting the center coordinates of the detected pupil for each frame, pupil movement information that can ultimately analyze where the user's gaze is looking and how long it stays is extracted.

Below, the three steps I, II, and III are described individually in detail.

I. Face Alignment

This is a process for detecting the user's face area from a video image acquired by a web camera, aligning the detected face area, and extracting it by fixing it at the same resolution. It proceeds in the following steps:

1. Facial Video Acquisition

The subject's upper body, including the face, is photographed at a predetermined frame rate, for example, 30 fps, using a visible light camera, for example, an RGB web camera.

2.Face Detection

Various existing methods can be applied to face detection. In this embodiment, a face area is detected from an image using an SVM (Support Vector Machine) learned using HOG (Histogram of Oriented Gradient) features. Figure 2 (A) shows a state in which a square face area was detected in an upper body image taken from a subject.

3. Facial Landmark Detection

Facial landmarks are detected within the detected face area. Here, for example, 68 feature points representing the positions of the user's eyes, nose, mouth, eyebrows, and facial outline are detected. Figure 2 (B) shows a state in which a landmark is detected in a square face area obtained from the subject's upper body image.

4. Face Alignment

Using the facial feature points detected in the previous step, the degree of rotation of the detected face is recognized and then aligned to create the correct frontal face. Additionally, the size of the sorted images is adjusted to maintain the same resolution. Figure 3a exemplarily shows an output image obtained by face alignment with respect to an input image.

The algorithm used for face alignment is Adrian's method (Face Alignment with OpenCV and Python), and creates an aligned image (Output) through the following specific process.

4.1 Detection Face Roll

In this embodiment, the head pose with the face turned is calculated to calculate the degree to which the face is turned (tilted) to the left and right, that is, the size of the roll.

The coordinates of n facial feature points of the left eye and n facial feature points of the right eye are averaged to obtain the center coordinates (Ex, Ey) of both eyes. Expressed as a formula, it is as follows, and the number of feature points (n) is set to "6".

After obtaining the center coordinates (Ex, Ey) of each eye using the two calculation formulas above, calculating the difference between the x-coordinates of the two eyes, dx, and the difference between the y-coordinates, dy , using the arctan operation, the angle of left and right head movement ( θ ), specifically, calculate the tilt angle ( θ ) of the other eye (Eye 2) with respect to one eye (Eye 1).

4.2 Obtaining the face resolution ratio

The aligned image obtained as a result of Face Alignment calculates a scaling ratio that satisfies the following conditions. Figure 3b shows the result of scaling at a 1:1:1 ratio for the aligned image.

As shown in Figure 3b, the distance (d) between the centers of the two eyes is set to the width of one side of a square unit area divided on a grid, and the width (w = Wpxn) and height (h = Hpxn) of the entire image are set to A. :A:A or A:B:A, for example, is divided vertically and horizontally at a ratio of 1:1:1 or 1:2:1 or 1.5:2:1.5 to obtain a total of 9 divided unit areas. At this time, in this embodiment, the sorted image size (Wpxn*Hpxn) can be set to 256*256 pixels, and at this time, the scale factor ( F _scale ) that satisfies the above conditions is calculated as follows. It can be obtained by

The equation below calculates the distance (d) between the centers of the left and right eyes in the face image.

Calculate the number of horizontal or vertical pixels (d _obj ) in the unit area when the face image below is divided into three at 1:1:1, and then calculate the scale factor ( F _scale ).

4.3 Affine Transformation

Through Affine Transformation, a transformation method that preserves the ratio of the face in the image, the image is rotated by the tilt angle ( Angle , θ ) obtained in steps 4.1 and 4.2, and the image is scaled using the scale factor (F _scale ). do.

The matrix for rotation according to the tilt angle and scale conversion according to the scale factor can be obtained using the cv2.getRotationMatrix2D() function provided by Opencv.

Since the angle standard in Opencv is counterclockwise, the affine transformation is performed using the obtained slope angle ( θ ). At this time, while rotating the image by the tilt angle ( θ ), image scaling is performed according to the scale factor, and for this, a rotation transformation matrix ( M _rotation Angle ), a scale transformation matrix (M scale ), and an affine transformation matrix (M _affine ) can be defined as follows.

( θ = - Angle )

(S _x = S _y = Fscale)

( t _x , t _y are translation matrix values for performing rotation transformation based on the center of the image)

The face images sorted through various processes for facial alignment as above are used as input for the next region of interest extraction step.

II. Region of Interest Extraction (ROI Detection)

In this step, the eye area, which is the region of interest, is extracted from the aligned face image. The eye area is extracted using facial feature points and proceeds in the following steps. Figure 4a illustrates the aligned face image (input) obtained in the previous step and the face image (output) from which ROI was detected, and Figure 4b is a partial enlarged photograph of the face image to explain the extraction of the eye region.

1.Face Detection

The face area is detected from the face image (input) sorted by a SVM (Support Vector Machine) trained in advance using the HOG (Histogram of Oriented Gradient) feature, which is one of the face detection methods.

2. Facial Landmark Detection

Within the detected face area, 68 feature points representing the positions of the user's eyes, nose, mouth, eyebrows, and facial outline are detected.

3. Eye Region Extraction

In this step, extraction of the eye area using facial feature points is performed. According to one embodiment, in this embodiment, an eye region is extracted from an aligned eye image using all or part of 12 feature points that represent the eye or are formed along the border of the eye.

Extraction of the eye region will be explained with reference to Figure 4b.

In this embodiment, among the 12 feature points representing the eyes, the x-coordinate of the largest value, the coordinate of the smallest x-value, the y-coordinate of the largest value, and the y-coordinate of the smallest value are used as standards, and the reference x, y The area created after increasing or decreasing a fixed value in the coordinate value, for example, 5 pixels, was extracted. In Figure 4b, a pair of vertical and horizontal lines are arranged on the top, bottom, and left sides of each of the left and right eyes.

In Figure 4b, VL1 and HL1 expressed in dotted lines are vertical lines passing through the highest and lowest x and y reference coordinates among the feature points of each eye, and VL2 and HL2 expressed in solid lines are the fixed values of VL1 and HL1, In this example, it is a vertical line passing through coordinates increased vertically and horizontally by 5 pixels in the direction away from the eye.

And, in Figure 4b, the rectangular ROI surrounding each eye is an area confined by the left and right VL2 and the upper and lower HL2 of both eyes. That is, the ROI has an area of an arbitrary value, for example, an area expanded by 5 pixels up, down, left and right in terms of x and y for both eyes.

The ROI detected as above is used as an input for the next pupil detection.

III. Pupil Detection

The circle forming the pupil and its center coordinates are extracted from the ROI obtained in the above process. Here, a circle detection algorithm that finds the circle through mathematical modeling, for example, Hough Circle Transform (HCT), can be used.

To perform HCT, the cv2.HoughCircles function provided by Opencv in library form was used. Figure 5 schematically shows the entire process of extracting a pupil from an ROI, Figure 6 illustrates an example of a specific method for detecting a circle corresponding to a pupil from an ROI image, and Figure 7 shows one of the circle detection methods. This explains circularity detection using Hough Gridient.

1. Gray Scale

Convert the RGB eye area image to a grayscale image.

Here, a raw image (RAW IMAGE) can be acquired in gray scale at the time of video capture, and in this case, separate gray scale conversion does not need to be performed.

2. Gaussian Blur

A Gaussian filter is applied to blur and smooth the grayscale image.

3.Circle Detection

In this step, a circle corresponding to the pupil border is detected. At this stage, for example, Hough Circle Transform is performed, and at this time, canny edges can be detected (Edge detection) using the cv2.HoughCircles function, and circles can then be detected (Circle Detection) using the Hough gradient.

As shown in Figure 7, there may be countless circles passing through one point in one image, and among these circles, there may be circles that also include other points. The Hough Rradient algorithm is an algorithm that determines a circle based on the number of points. If the number of points included in one circle among the points in the Canny edge result exceeds the set threshold, it is recognized as a circle. According to this circle detection, as shown in Figure 6, (A) no circle is detected (Detected circle=0) or (B) one circle is detected (B, Detected circle=1) from one ROI. Or (C) multiple circles can be detected (Detected circle=2).

The processing for each case (A), (B), and (C) above can be done as follows.

(A) When no circle is detected

This is a case where the characteristics of the pupil circle are not clearly visible due to noise such as light reflection and eyebrows in the extracted eye area. Preprocessing is performed through opening operation, one of the morphology calculation techniques, so that the characteristics of the pupil circle appear more clearly, and then HCT (Hough Circle Transform) is performed again.

(B) When there is only one circle detected

This is a case where the pupil was correctly detected through HCT. The center coordinates of the extracted circle are returned as the pupil center coordinates.

(C) When there are two or more detected circles

This is a case where two or more circles are detected in the extracted eye area due to noise due to light reflection or calculation error of the Hough Circle Transform. The value obtained by averaging the center coordinates (x, y) of the detected circles is continuously returned as the pupil center coordinate and obtained as the final output.

The final continuous output of these central coordinates can determine the subject's gaze direction.

The pupil center coordinates extracted through the above process can be used as an indicator to determine where the user's gaze is looking within the image and how long it stays there.

According to this invention, it becomes possible to track pupil movement or gaze using a general-purpose web camera. This present invention can improve the problem of the conventional technology in which detection of the iris area was difficult compared to an infrared camera.

In order to help understand the pupil tracking method according to an embodiment of the present invention and the system for applying the same, it has been described with reference to the embodiment shown in the drawings, but this is merely illustrative, and those skilled in the art will It will be understood that various modifications and equivalent other embodiments are possible. Therefore, the true technical protection scope of the present invention should be determined by the appended claims.

Claims (15)

Obtaining a facial image by photographing the subject's face with a camera;
Detecting facial landmarks of left and right eyes from the face image by an image processing unit;
extracting center coordinates of the left and right pupils using facial feature points of the left and right eyes using an analysis unit;
The analysis unit calculates the distance between the centers of the left and right eyes from the coordinates of the centers of the left and right eyes, and selects a reference face image of the face area including the left and right eyes and the side parts of the face on both sides of the left and right eyes based on the distance between the centers of the eyes. step;
extracting an eye region, which is a region of interest (ROI), by detecting facial feature points of left and right eyes from the reference face image with the analysis unit;
detecting both pupils in the eye area using the analysis unit using a circular detection method for the eye area; and
A pupil detection method based on facial feature points, comprising: extracting center coordinates of both pupils by the analysis unit. According to paragraph 1,
The analysis unit calculates the tilt angle ( θ ) of the face using the coordinates of the centers of both eyes, and
A facial image alignment step of aligning the facial area by rotating the facial image using the analysis unit according to the tilt angle. According to paragraph 1,
The reference face image is scaled to an image of a preset size based on the distance between the centers of the eyes, converts the reference face image into an image of a preset size, and then extracts the region of interest (ROI) based on facial feature points. Pupil detection method. According to paragraph 1,
The reference face image is divided into 3 vertically and horizontally with an A:B:A ratio (A and B are the same or different positive numbers), and includes a unit area divided into 9 grids as a whole,
A pupil detection method using facial feature points, wherein the centers of both eyes are located at the coordinates of the two corners on both upper sides of the center area of the 9-divided unit area on the grid. According to paragraph 4,
The reference face image is divided into 9 segments of 1:1:1 in the vertical and horizontal directions, and the 9 divided unit areas have a square structure of the same size. A pupil detection method using facial feature points. According to clause 4 or 5,
A pupil based on facial feature points is configured to obtain a scaling factor based on the distance between pupil centers and scale the reference facial image into an image of a preset size using the scaling factor to convert the reference facial image into an image of a predetermined size. Detection method. According to clause 6,
A pupil detection method using facial feature points, where the Hough Circle detection method is applied to detect the pupil. According to any one of paragraphs 1 to 5,
A pupil detection method using facial feature points, where the Hough Circle detection method is applied to detect the pupil. In the pupil detection system performing the method described in claim 1,
A visible light camera that captures images of the subject's face;
an image processing unit that detects facial landmarks of left and right eyes from the face image;
Using the facial feature points of the left and right eyes, the center coordinates of the left and right eyes are extracted, a reference face image of the face area is selected using this, an eye area of a region of interest (ROI) is extracted from the reference face image, and a pupil is drawn from the eye area. A pupil detection system based on facial feature points, including an analysis unit that detects and extracts center coordinates from the pupil. According to clause 9,
The analysis unit calculates a tilt angle ( θ ) of the face using the center coordinates of both pupils, and aligns the facial area by rotating the reference face image according to the tilt angle. A pupil detection system based on facial feature points. According to clause 9,
A pupil detection method based on facial feature points, wherein the region of interest is extracted from a reference face image scaled to an image of a preset size based on the distance between pupil centers. According to any one of claims 9 to 11,
The reference face image is divided into 3 vertically and horizontally with an A:B:A ratio (A and B are the same or different positive numbers), and includes a unit area divided into 9 grids as a whole,
A pupil detection system using facial feature points, wherein the centers of the pupil on both sides are located at the coordinates of the two corners on both upper sides of the center area among the 9 divided unit areas on the grid. According to clause 12,
The reference face image is divided into 9 segments of 1:1:1 in the vertical and horizontal directions, and the 9 segmented unit areas have a square structure of the same size. A pupil detection system using facial feature points. According to clause 12,
A pupil detection system using facial feature points that applies the Hough Circle detection method to detect the pupil. According to any one of claims 9 to 11,
A pupil detection system using facial feature points that applies the Hough Circle detection method to detect the pupil.