TW201624200A - Eyeball locating method and system - Google Patents

Abstract

The present disclosure provides an eyeball locating method and system to track the eyes. The eyeball locating method and system is better than the present commercialized products. Tracking the eyes is an important evidence to detect whether the user is looking at the screen, and the eyeball locating method and system is widely used in electronic industrial. Therefore, The eyeball locating method and system has its potential market.

Description

Eyeball positioning method and system

The present invention relates to an eyeball positioning method and system, and more particularly to a method for positioning an iris center, thereby achieving the industrial purpose of applying the eyeball positioning method and system.

In recent years, Internet technology has been changing with each passing day. New generations of young people have shifted their entertainment from TV to the Internet. Relaxing with a tablet or smart phone has become a daily business, and advertising on the Internet has become mainstream. one. In the space around the forums, we will see a lot of billboards appearing. New ads will be played in rotation every other time. When watching YouTube, there will be ads appearing in front of the selected videos, using the tablet. When the computer's APP (Application) online watch movies or national serials, there will also be advertisements before the broadcast. In addition to watching the videos on the videos or forum websites, some of the free-to-pay apps will also have a wide range. Appears in the corners, developers can use these ads to have income, just as traditional advertisements will be published in newspapers and magazines, newspaper manufacturers or magazine manufacturers can use advertising to increase revenue, which is one of the main income. Advances in technology have also made it possible for shoppers who want to advertise to choose not only TV or newspapers, but also online platforms, but how to make sure that users are paying attention to these ads, this can be used to study Let users of various platforms give back to the vendors the data for benefit analysis.

There may be some obstacles in the process of children learning, such as difficult vocabulary, will stay on these words for a long time, or encounter some difficulties in the process of foreign language learning, you can use this study to analyze learning What is the difficulty of learning, and what is different from the students in the same course, through the comparison of the number of gaze and gaze time through the location of the placement, to improve their learning, in order to achieve better learning result.

In the case of getting some diseases, human beings are difficult to communicate effectively with the outside world, such as patients with gradual freezing. These patients are degenerative of the central nervous motor neurons in the brain. The patient's performance is that the whole body muscles are slowly shrinking, including speechlessness, and even The respiratory function will also be completely degraded. In the end, only the eyes can be turned. These people with physical and mental disabilities are more inconvenient to interact with people, so some external auxiliary mechanisms are needed to help. In the future, if there is By using this research and related equipment, manufacturers can help these people communicate externally, understand their inner thoughts or desires, and even help them to recover psychologically and build confidence. In addition, there are many development directions, such as driving safety analysis. Does the driver notice the lanes when driving the steam locomotive, whether there is attention to the left or right rear mirror when changing lanes, whether it is because of listening to radio or music. It affects the visual attention, and may also be scored for reference in the future when the driver's license is obtained, so that the driver's license can be surely delivered to the safe driving of the driving, to avoid the damage caused by the accident in the future.

The human eye is a very delicate and complex organ. The following is a brief introduction to some of the more commonly seen structures. The first picture is the eye structure. Sclera: It is the part of the eye white, which is the outermost layer of the eyeball. It is tough and opaque. It protects the inside of the eyeball and maintains the shape of the eyeball. Iris: Contains muscles and pigments. The center of the iris has a circular opening called the pupil. The pupil is enlarged and reduced by the intensity of the light to control the incoming light. Cornea: The cornea is the transparent part of the front of the eye that covers the iris, pupil, and anterior chamber. It is normally colorless and transparent, and the color of the iris can be seen through the cornea. Lens: The flat crystal oval shape behind the pupil after the iris is placed behind the iris. It has a fine pocket to prevent water from entering the water chamber. The thickness can be changed by the elasticity of the ligament to adjust the refraction. . Limbus: located at the junction of the cornea and the sclera, as the transparent cornea gradually breaks into the sclera, In the absence of a clear dividing line on the surface of the eye, the younger the person will be more obvious, and there is no certain width, the more obvious the person will be more attractive.

Visual gaze research can be roughly divided into contact systems and non-contact systems, which are classified as contact systems when they come into contact with the human body. Some physical damage or potential risks are caused by contact with the human body. Relatively not recommended, non-contact systems are relatively safe, and have been adopted by many research units for visual gaze tracking. In addition, in terms of comfort, the non-contact system is also more comfortable for the subject. The various systems developed at home and abroad are further described below.

Electro-Oculography (EOG), the electro-optic method is to attach the upper and lower skin of the subject's eyes to the electrode patch, and receive the voltage difference between the cornea and the retina. The principle of the eye is the human eye. For a circular battery, the middle of the cornea is the positive pole, the retina is the negative pole, and the eye movement is calculated by using the difference of the received signals to achieve the purpose of tracking the eye movement. The second picture is a schematic diagram of the electro-oculogram method. A total of five patches are used. The patch for measuring the horizontal movement of the eyeball is attached to the left and right sides of the eye, and the patch for measuring the vertical movement of the eyeball is attached to the upper and lower sides of one of the eyes, and the last one is Sticking to the forehead position as a reference point, when the eyeball is turned to the right, there will be a positive voltage difference between the electrodes. When turning to the left, a negative voltage difference will be generated between the electrodes, and the voltage difference is judged by the magnitude of the voltage difference. Eyeball up and down The angle of movement to the left and right. The advantage of the electro-oculogram method is that the price is not high, but the disadvantage is that the resistance of the human body will be different due to environmental changes, such as the secretion of keratin, which will cause voltage instability. In recent years, Andreas Bulling and others have used this method to capture three kinds of eye features, squint, gaze and close the eyes, and then classify the repeated signals into models. Analyze the common actions in the five offices, including copying articles, reading documents, and writing. Words, movies, free reading, and nothing, the average accuracy rate is 76.7%.

Search Coil, search coil method is to let the subject wear a specially designed soft lens. As shown in the third figure (a), the induction coil is mainly placed in the middle of the two soft lenses, and a fixed magnetic field is applied around the eyeball in advance. When the eyeball of the subject moves, the lens is driven, and the induction coil changes due to the magnetic flux. The induced electromotive force is generated, and the magnitude of the induced electromotive force represents the angle of deflection of the eyeball, and the signal of the eyeball can be recorded after the signal is amplified. The advantage is high accuracy, so it will be used in medical and psychological research, but the measurement method of the search coil method is very susceptible to the eye condition of the subject at the time, such as the secretion of the eyeball, etc., and because it is invasive. The method is not suitable for long-term wear, and the soft lens has a two-layer structure, which affects the user's vision, so it is rarely used clinically. This search coil lens is divided into 2D and 3D forms. 2D is composed of 9 circles of enameled wire for measuring horizontal and vertical data. 3D has two rings, each ring is composed of 8 circles of enameled wire, except for measuring horizontal and vertical. In addition, you can measure the twist The data.

Purkinje image tracking method (Dual-Purkinje-image), Purkinje image tracking method (Dual-Purkinje-image) is used to generate a reflection image of the light source when the light passes through the eyeball tissue due to the difference in refractive index of each tissue of the eye. The image is called Purkinje-image and usually four images can be observed. The first type of Purkinje-image is the surface of the light reflected from the cornea and the air in front of the eye. The fourth type of Purkinje-image is the surface that is reflected in the crystal and the inside of the eye. The other two types of Purkinje-image are usually ignored because the amount of reflection is too small. Excluding. This method detects the first and fourth Purkinje-image to detect eye movements. The image is captured by a black-and-white camera. The light source is directly illuminated by two near-infrared LED lights. The image as shown in the third figure (b) can be derived by using the relationship between the Purkinje-image and the center position of the pupil in the image. The possible position of the eyeball sight.

Pupil Tracking, if the system uses infrared or near-infrared light to illuminate the eye, the pupil is less reflective to infrared light, and the iris is more reflective to infrared light, so it will cause image The difference in brightness between the pupil and the iris becomes larger, and the difference in brightness between the iris and the white of the eye becomes smaller. This is different from the general white light source, which makes it easier to remove the contour of the pupil. This tracking technique that uses the position of the pupil to determine the direction of the line of sight is called Pupil Tracking. Since the inner circumference of the iris (ie, the pupil contour) is clearer than the outer periphery of the iris, It is less easily obscured by the eyelids, so the tracking of the position has a higher resolution than the tracking of the heterochromatic boundary zone. However, infrared rays have certain damage to the eyes and skin under long-term exposure. The main damage to the eyes is cataract, retinal and corneal burns, and the hot pressure generated by heat radiation under low-intensity light sources. If the exposure is greater than a critical value The value will cause damage to the eyeball.

Infrared Oculography (IROG), which is a method of arranging a row of infrared light source and infrared receiver on the frame, and is placed around the iris at a fixed angle, because of the reflection effect of infrared rays on the black pupil and the iris. It is relatively low, so most of the light source will be absorbed, and the white part of the eye will almost completely reflect the infrared rays, so it can be traced by the difference of the reflection of the infrared light source at the edge of the cornea, and the infrared receiver will convert the reflected infrared light. After the current signal is obtained, the angle of the eyeball rotation can be judged by the magnitude of the signal after differentially amplifying the up, down, left and right signals. The advantage of this technology is that it is cheap and convenient to use, and is not affected by the intensity of external light, but the disadvantage is that the infrared radiation to the eyes at close range can make the eyes dry and uncomfortable, and is easily affected by the background light. Eye tissue is hurting.

Limbus Tracking This type of system is mainly used to reflect the image reflected by the light source after entering the eyeball. If the background light or the general white light source is used, the natural difference between the white eye and the black eyeball can be used to detect the iris boundary, which is called the heterochromatic boundary. With Limbus Tracking. The advantage of using this method is that it does not need to be illuminated by the infrared light source to avoid physical damage caused by light to the eyes, but the disadvantage is that it is not easy to clearly take out the periphery of the iris, and the upper and lower boundaries are easily covered by the upper and lower eyelids, so the range is limited. The heterochromatic boundary zone processing method needs to manually select six iris boundaries, let the system know the parameters of the boundary and then take the maximum value as the threshold. The pixels below the threshold may be iris or pupil, and also After the six points of the iris are taken, the minimum parameter is used as the threshold. The value above the threshold may be the white of the eye or the skin part. Each subject needs to be selected, which may cause inconvenience to the subject. The method of searching the center of the pupil is to calculate the position of the center of gravity of the pixel coordinates that may be the iris and the pupil. However, in the iris and the pupil, sometimes the pixel coordinates are listed as non-iris or pupils due to the reflection of the center of gravity. It was calculated, which also affected the final coordinates of the center of gravity, so the author also mentioned that this center of gravity is not necessarily the center of the pupil, but the author believes that it is quite close to the center of the pupil.

There are also many methods of eye tracking that use the Circle Hough Transform to find the pupil position in the eye, but most of these methods are used with infrared LED lights because the pupils in the infrared light will absorb large. Part of the infrared light, the iris and the white part of the eye will reflect more light, resulting in a clear outline of the junction between the pupil and the iris, so the use of Hough circle detection will have a good effect. If you use Hough Circle Detection to find a different color boundary zone, it will be more difficult to find the right one. Iris, because the circle detected by the Hough circle depends on the setting range on the parameter, and the iris size of each person is different, so you need to use several parameters in the parameter of setting the radius of the circle, so that it will not be because The difference between the subjects will not be detected, but it will cause many detected circles in the image. Although the iris boundary is found, the system will be misjudged as a circle at other non-iris boundaries. The fourth picture shows the results of using the Hough circle detection. In addition to detecting the iris boundary, other parts that are obviously not representative of the iris circle are detected.

The above methods have the disadvantages of high price, physical damage that the test subject may be exposed to, the excessively large volume of the instrument itself, or the use of infrared rays to make the eyes feel dry and uncomfortable, and still be an industry improvement in the current industry. Demand.

In addition, the relevant knowledge required by the present invention, such as grayscale, spatial filtering, binarization, and inner, outer and center of gravity of the triangle, is briefly explained as follows.

Gray-scale, the more common image color models are as follows: RGB, YIQ, HSI, YUV, and YCrCb. Different color models will behave differently in different applications. In some applications, gray image (Gray Image) is used instead of the original RGB color image for some processing, because it can effectively avoid the influence of color change, and to some extent reduce the amount of data and processing time. The conversion relationship between the RGB color model and the YIQ color model can be described by a matrix type, as shown in (1-1): Where Y stands for Luminance, I is a color element (Inphase), and Q is also a color element (Quadrature). The benefit of using the YIQ model is that the luminance Y and the I and Q associated with the color elements can be separated.

Spatial filtering, in the image processing process, often uses some spatial filters to image the image, and the most basic principle of the spatial filter is to use a mask to process the image point by point. Commonly used mask sizes are 3×3 (as shown in the fifth figure), 5×5 or 7×7, and common spatial filters include enhancement filters and smoothing filters. The sharpening filter is essentially a high-pass filter, which is mainly used to make the detail or edge of the image more prominent, and achieve image enhancement. There are three common enhancement filters: the first is a basic high-pass spatial filter, which uses the mask operation of the sixth figure (a) and returns the operation result to P ₅ as shown in the fifth figure. The characteristic of such a filter is that if the center point of the mask corresponds to a pixel value having a large gray scale, after this filtering, the gray scale difference between the pixel and the adjacent pixel will be enlarged, otherwise, the mask The cover will have a very small output for smooth areas where the grayscale changes quite slowly. In the more extreme case, when the gray scales in the coverage range are the same, regardless of the original gray scale value, the output will be zero, which means that the mask has a lower average image. The shortcomings of the overall darkening of the image, and the actual output value will have a negative possibility, so the size must be adjusted. The second type is a high-frequency enhancement filter, which uses the mask operation of Fig. 6(b) (where α is the magnification) and returns the operation result to P ₅ . This filtering is to multiply the original image by a magnification greater than one and subtract the low-pass filtered result of the image. The third type is a differential type filter, which uses the masks of (a) and (b) of the seventh figure to calculate the gradient components of the rows and columns, respectively. The main purpose of this filter is to detect the edge of an object in image segmentation because it magnifies the grayscale between the edge of the image object and its neighboring pixels to achieve an image enhancement that highlights the edge contour of the object. The smoothing filter is essentially a low-pass filter whose primary purpose is to blur the image or reduce high-frequency noise. For the purpose of image recognition, image blurring caused by the use of a smoothing filter removes small details that impede the capture of important features and connects the broken lines. For pulsed high frequency noise, the smoothing filter will alleviate the effects of this noise. There are two main types of smoothing filters: the first is an averaging filter, which combines the pixel values in the mask and averages them back to P ₅ . The calculation mask is shown in the eighth figure, which shows a 3×3 mask. In general, the larger the mask blur effect, the lower the cutoff frequency of the filter, and the more the high frequency portion is filtered out. The second is a median filter, which is to arrange the pixel values covered in the mask from small to large, and take out the value arranged in the middle and pass it back to P ₅ instead.

Binarization, first set a gray value in binarization, any image whose gray level is larger than it will make it a bright spot (pixel value 255), and if the gray value is lower than the set value, it will be dark. Point (pixel value 0), this gives a binary image. As shown in the ninth figure, the valley between the two peaks is the Threshold value, T ₀ , and then the binarization value is greater than 255 for the grayscale value greater than the threshold value, and less than or equal to 0. Equation (1-2) defines the formula for binarization: Where B _b is the binarized signal, B _a is the signal before binarization, T ₀ is the threshold, and when the value of B _a is less than T ₀ , all the qualified B _a signals are regarded as 0. When the value of B _a is greater than or equal to T ₀ , all eligible B _a signals are treated as 255, making the picture black and white only.

The three-point circle method, the outer core, all triangles also have circumscribed circles, and the center of the circle is called the outer core. Making a vertical bisector on each side of the triangle gives a junction. This is the outer core. As shown in the tenth figure, the distance from the center of the circle to the three vertices is equal to the radius of the circle, and the outer centroid is O ( x, y ), where , .

In view of the above-mentioned background of the invention, in order to meet the needs of the industry, the present invention provides an eyeball positioning method and system for solving the problems currently faced by the industry, and at the same time improving the technique of the eyeball positioning method and system.

The present invention is suitable for all situations in an eye tracker, and the eye tracker to be used is designed according to specific needs, so the system sets certain fixed conditions, such as the intensity of the light source, the position of the screen, and the camera pendulum. In the position, this system does not use infrared LED lights to strengthen the difference between the iris and the white-white junction, so the symmetry of some eyeballs is used later to make up for the system's lack of eyeballs. The system directly sets the region of interest (ROI) at the beginning, so that there is no need to find other parts of the eye to set the ROI, which causes instability on the system, and then the captured image. Gray-scale processing is performed to simplify the image information at the ROI. After obtaining the gray-scale information, the gray-scale image is filtered to reduce unnecessary noise, and finally filtered by means of image partitioning (Thresholding). Unnecessary information, then use the characteristics of three points into a single circle to draw a circle in the iris boundary, and judge whether the circle drawn by these three points is reasonable, if not reasonable to leave.

According to an aspect of the present invention, an eyeball positioning system is provided. The eyeball positioning system includes: a display screen for providing a subject for viewing; and a camera having an adjustable support rod attached to the bracket. And an arithmetic processing device for receiving a recorded image by the camera, and then performing a setting of the attention area The field is obtained by extracting a right captured image and a left captured image from the recorded image, and performing grayscaled to change the right captured image and the left captured image into a right simple image and a left a simple image, performing a filtering to remove the noise of the right simple image and the left simple image into a right filtered image and a left filtered image, performing a binarization, and converting the right filtered image and the left filtered image into a right second The value image and the left binary image are subjected to an iris center estimation, wherein the right binary image and the left binary image are taken from three points to form a triangular circumcircle, and a right iris center is estimated. At the left iris center, a screen fixation point estimate is performed, and the position of the gaze point is estimated by at least one of the right iris center and the left iris center.

According to another aspect of the invention described above, the eyeball positioning system comprises: a bracket for stabilizing the head of the subject.

According to another aspect of the invention described above, the display screen is for receiving and displaying the information provided by the arithmetic processing device, and the resolution of the camera is 1280 x 720 pixels.

According to an aspect of the present invention, an eyeball positioning method includes: performing an open video to receive a recorded image; and performing a setting of a focus area by extracting a right image from the recorded image according to the set region. Taking the image and capturing the image from the left; performing a grayscale transformation, converting the right captured image and the left captured image into a right simple image and a left simple image; performing a filtering, the right simple image and the left The noise removal of the simple image becomes a right filtered image and a left filtered image; Performing a binarization, converting the right filtered image and the left filtered image into a right binary image and a left binary image; and performing an iris center estimation by using the right binary image and the left binary image Three points are taken to form a triangular circumcircle, and a right iris center and a left iris center are estimated.

According to another aspect of the present invention, the eyeball positioning method includes: performing a gaze point estimation, and estimating a position of the gaze point by at least one of the right iris center and the left iris center .

According to another aspect of the present invention, the received image received in the above-mentioned open video is a resolution of (2000 to 800) x (1500 to 300) pixels, wherein a set of embodiments is 1280 x 720 pixels, and The included image is a range of fixed faces.

According to another aspect of the present invention, the setting of the attention area is to directly set a size of a right focus area and a left focus area from the recorded image, and then the position between the right focus area and the right focus area. The size of the left focus area and the position between the left focus area and the left capture image, wherein the right focus area and the left focus area are set to a fixed size of (600~120) x (300~60) pixels. Then, in accordance with the distance of the lens from the face, adjust the lens to the corneal edge according to the difference of each person. This size just does not contain the eyebrows, and the corners are not framed at the periphery. There is also a certain distance between the areas of interest to exclude the bridge of the nose from the area of interest. During the setting process, the head is fix.

According to another aspect of the invention described above, the above-mentioned region of interest is 240 x 120 pixels.

According to another aspect of the present invention, the right captured image and the left captured image become the right simple image and the left simple image. The recorded image information is a full-color image, which is converted into gray-scale information by Y = R × 0.299 + G × 0.587 + B × 0.114, leaving only the brightness information, where Y represents brightness and RGB represents red, green and The chromaticity of blue, in which a set of embodiments is the information of the converted brightness from 0 to all black 255.

According to another aspect of the present invention, the noise of the right simple image and the left simple image is versus The mode removal becomes the right filtered image and the left filtered image.

According to another aspect of the present invention, the right filtered image and the left filtered image are represented by a threshold value greater than a binarization threshold, and a threshold value less than the binarization threshold is presented. Points are respectively converted into the right binary image and the left binary image.

According to another aspect of the invention described above, wherein the two values are The threshold value is 20.

According to another aspect of the present invention, the method for finding the center of the iris in the right binary image is as follows: Step 1: Find the vertical line with the most black points and the most nose from the right binary image. The first vertical line; step two, starting from the right first vertical line to the nose side to find the first vertical line of the white point is a right second vertical line; step three, the right second vertical line to the right The ear side finds the first black point as a right first certain point; in step 4, the lowest black point in the right binary image is a right lowest point; in step 5, the right first determined point and the right lower point are utilized The middle point of the connection finds the horizontal line as a right first horizontal line; in step 6, the intersection of the right first horizontal line and the right first vertical line is found along the right first horizontal line toward the nose side, and the end point is the right first The intersection of the horizontal line and the second right vertical line, the last black point on the line is a right second certain point; in step 7, the right first horizontal line moves to the right ear side of the right binary image to find the first line The vertical line with the most white points is a right Vertical line; step eight, from the right third vertical line to the right side of the right binary image to find the first black point is a right third determination point; step nine, the right first determination point, the right second A right triangle circumscribed circle formed by the determined point and the right third determined point, the center of which is a right iris center.

According to another aspect of the present invention, the method for finding the iris center in the left binary image is as follows: Step 1: Find the left first vertical line with the black point and the most nose from the left binary image. ;step In the second step, the vertical line from the left first vertical line to the nose side to find the first white point is the second second vertical line; in the third step, the left second vertical line finds the first to the left ear side. The black point is a left first certain point; in step 4, the lowest black point in the left binary image is a left lowest point; in step 5, the middle of the left first determined point and the left lowest point is used. Point to find the horizontal line is a left first horizontal line; in step 6, the intersection of the left first horizontal line and the right first vertical line is found along the left first horizontal line toward the nose side, and the end point is the left first horizontal line and the left The intersection of the second vertical line, the last black point on the line is the second second determination point; in step 7, the left first horizontal line moves to the left ear side of the left binary image to find the first white point. The vertical line is a left third vertical line; in step 8, the first black point is found from the left third vertical line to the nose side of the left binary image, which is a left third determined point; a left first determined point, the left second determined point, and the left third determined point Left into a triangle circumscribed circle whose center is the center of a left iris.

According to another aspect of the present invention, the radius of the right triangle circumscribed circle and the left triangle circumscribed circle is larger than one third of the width of the region of interest (600 to 120), and then the attention area is set.

According to another aspect of the invention described above, wherein the radius of the right triangle circumscribed circle and the left triangle circumscribed circle is less than one third of the width of the region of interest (600-120), and then a screen fixation point estimation is performed.

According to another aspect of the present invention, the radius of the right triangle circumscribed circle is less than one third of the width of the region of interest (600-120), and the radius of the left triangle circumscribed circle is greater than the width of the region of interest (600) ~120) one-third of the pixel, and then performing a left mapping compensation; the left mapping compensation step is as follows: Step one, calculating the vertical direction position of the right iris center point and the vertical direction position of the right first determining point The difference is a right vertical gap; in step 2, the difference between the horizontal direction position of the right iris center point and the horizontal direction position of the right first determined point is calculated as a right horizontal difference; and step 3 is determined by the left first The vertical position of the point moves the right vertical gap from the starting point, and then moves the right horizontal gap to the left ear side to obtain a estimated left iris center.

According to another aspect of the present invention, the radius of the left triangle circumscribed circle is less than one third of the width of the region of interest (600-120), and the radius of the right triangle circumscribed circle is greater than the width of the region of interest (600) ~120) one-third of the pixel, and then performing a right map compensation; the right map compensation step is as follows: Step one, calculating the vertical direction position of the left iris center point and the vertical position of the left first determination point The difference is a left vertical gap; in step 2, the difference between the horizontal position of the left iris center point and the horizontal position of the left first determined point is calculated as a left horizontal difference; and step 3 is determined by the right first The vertical position of the point moves the left vertical gap from the starting point, and then moves the left horizontal gap to the right ear side to obtain a estimated right iris center.

According to another aspect of the invention described above, wherein the screen fixation point estimation is performed, the position of the gaze point of the screen is estimated by at least one of the center of the right iris and the center of the left iris by a linear amplification method.

According to another aspect of the present invention, the linear amplification method is to determine the relationship between the position of the center of the iris and the position of the four sides of the screen by pre-measuring the center of an iris to view the fixed points on the four sides of the screen. Thereby, the position of the viewing screen is estimated by the position of the other iris center.

100‧‧‧ eye positioning method

110‧‧‧Open video

120‧‧‧Set Area of Interest (ROI)

130‧‧‧ Grayscale

140‧‧‧ Filter

150‧‧‧ Binarization

160‧‧‧Iris Center Estimate

164L‧‧‧left mapping compensation

164R‧‧‧right mapping compensation

170‧‧‧ Screen gaze estimation

210‧‧‧Included images

120R‧‧‧right area of interest

120L‧‧‧Left area of interest

220R‧‧‧Image captured right

220L‧‧‧ Left capture image

230R‧‧‧right simple image

230L‧‧‧ Left Simple Image

240R‧‧‧right filtered image

240L‧‧‧left filter image

250R‧‧‧Right binary image

250L‧‧‧left binary image

311R‧‧‧right first vertical line

312R‧‧‧Right second vertical line

313R‧‧‧Right third vertical line

321R‧‧‧Right first horizontal line

331R‧‧‧The first fixed point on the right

332R‧‧‧ second right point

333R‧‧‧ third right point

334R‧‧‧Right Iris Center

335R‧‧‧Right triangle circumscribed circle

336R‧‧‧ Estimated right iris center point

341R‧‧‧The lowest point

311L‧‧‧Left first vertical line

312L‧‧‧ second vertical line

313L‧‧‧Left third vertical line

321L‧‧‧Left first horizontal line

331L‧‧‧Lead first point

332L‧‧‧ second second determination point

333L‧‧‧Left third fixed point

334L‧‧‧Left Iris Center

335L‧‧‧Left triangle circumscribed circle

336L‧‧‧ Estimated left iris center point

341L‧‧‧Last lowest point

351R‧‧‧Right vertical gap

352R‧‧‧Right level gap

351L‧‧‧ Left vertical gap

352L‧‧‧Left level gap

400‧‧‧Eye positioning system

405‧‧‧ bracket

407‧‧‧ display screen

410‧‧‧ camera

420‧‧‧Operation processing device

The first figure is a schematic diagram of the limbus (Limbus); the second figure is a schematic diagram of the electro-oculogram device; the third figure (a) is a schematic diagram of the search coil method; the third figure (b) is a near-infrared LED light reflection. The image presented later; the fourth image is the result of iris boundary detection using Hough circle detection, (a) the right eye (b) the left eye; the fifth picture is the mask schematic; the sixth picture is The arithmetic mask of the enhancement filter, (a) the mask of the basic high-pass spatial filtering (b) the mask of the high-frequency enhancement filter; the seventh diagram is the operation mask of the gradient component, (a) the row direction, (b ) column direction; The eighth picture is the arithmetic mask of the averaging filter; the ninth picture is the binarization concept picture; the tenth picture is the circumscribed circle and the outer center of the triangle; the eleventh figure is a preferred implementation of the present invention A flow chart of the steps of the method; a twelfth figure is a preferred embodiment of the present invention, which directly sets a fixed ROI diagram in the captured image; and a thirteenth figure is a preferred embodiment of the present invention. Filter used, (a) right-eye filter (b) left-eye filter; fourteenth embodiment is a preferred embodiment of the present invention, using the effect of the filter, (a) gray-scale image of the right eye (b) Filtered image; The fifteenth figure is a preferred embodiment of the present invention, the effect of binarization. (a) filtered image (b) binarized image; FIG. 16 is a schematic diagram of a method for finding the center of the iris in the right binary image according to a preferred embodiment of the present invention; A preferred embodiment of the present invention is a schematic diagram of a right eye iris center estimation point formed by a method for finding an iris center from a right binary image; and an eighteenth diagram is a preferred embodiment of the present invention, the left binary value A schematic diagram of a method for finding an iris center in an image; a nineteenth diagram is a schematic diagram of a method for estimating an iris center formed by a method for finding an iris center from a left binary image according to a preferred embodiment of the present invention; FIG. 20 is a schematic diagram of a method for compensating left mapping according to a preferred embodiment of the present invention; FIG. 11 is a schematic diagram of a preferred embodiment of the present invention, and a method for compensating right mapping; 2 is a schematic diagram of a preferred embodiment of the present invention, a schematic diagram of a boundary point diagram; and a twenty-third diagram is a preferred embodiment of the present invention, and the distribution position of the five-point diagram on the screen of the success rate of the test is tested. The schematic diagram of the twenty-fourth embodiment is a schematic diagram of a preferred embodiment of the present invention.

The twenty-fifthth diagram is a schematic view of a preferred embodiment of the present invention.

The present invention is directed to an eyeball positioning method and system. In order to thoroughly understand the present invention, detailed materials, steps, and applications will be set forth in the following description. Obviously, the practice of the invention is not limited to the specific details that are apparent to those skilled in the art. On the other hand, well-known materials or steps are not described in detail to avoid unnecessarily limiting the invention. The present invention will be described in detail below, but the present invention may be widely practiced in other examples, and the scope of the present invention is not limited by the scope of the following patents.

According to a preferred embodiment of the present invention, as shown in FIG. 11 , the present invention provides an eyeball positioning method 100. The eyeball positioning method 100 includes: performing an open video 110 to receive a recorded image 210, and then performing a setting. The area of interest (ROI) 120 is obtained by capturing a right captured image 220R and a left captured image 220L from the recorded image 210, and performing a grayscaled 130 right captured image 220R and a left captured image 220L. The image becomes a right simple image 230R and a left simple image 230L, and a filtering 140 is performed to remove the noise of a right simple image 230R and a left simple image 230L into a right filtered image 240R and a left filtered image 240L, and a binary value is performed. 150 turns a right filtered image 240R and a left filtered image 240L into one The right binary image 250R and the left binary image 250L. Performing an iris center estimation 160, respectively, by taking three points in the right binary image 250R and the left binary image 250L, respectively forming a triangular circumcircle, and estimating a right iris center 334R and a left iris center 334L, A screen gaze point estimate 170 is performed to estimate the gaze point by at least one of a right iris center and a left iris center.

According to the above embodiment, the recorded image 210 received in the open video 110 is a resolution of (2000~800) x (1500~300) pixels, wherein one set of embodiments is 1280x720 pixels. Moreover, the recorded image 210 is a range of fixed faces.

According to the above embodiment, the set focus area 120 is configured to directly set the size of a right focus area 120R and a left focus area 120L and the position between the left and right focus areas 120L from the recorded image 210, and then follow the right focus area 120R and the left focus. The size of the area 120L and the position between the right and left capture images 220R and the left focus image 120L are set to be fixed (600~120) x (300~60). ) Pixel, as shown in the twelfth figure, and then match the distance of the lens from the face, and adjust the lens to grab the limbus according to each person's difference. This size is just not included in the eyebrows, and the corners are not framed at the periphery. There is also a certain distance between the two ROIs to exclude the nose from the ROI. Since the head is fixed during the setting process, it is possible to determine only the surrounding area where the eyes are fixed by placing the eye area into the ROI. Do the processing. Among them, a set of implementation examples is 240x120 pixels.

According to the above embodiment, a grayscale 130 is performed to change the right captured image 220R and the left captured image 220L into a right simple image 230R and a left simple image 230L. The image information captured by the camera is a full-color image, that is, an image composed of three colors of red, blue and green, but in order to reduce the calculation time and the amount of calculation, the method converts the color information into gray scale via (2-1). Information, that is, only the information of the brightness, the converted intensity value is from 0 to all black 255.

Y = R × 0.299 + G × 0.587 + B × 0.114 (2-1)

According to the above embodiment, a filtering 140 is performed to remove the noise of the right simple image 230R and the left simple image 230L into a right filtered image 240R and a left filtered image 240L, as shown in the thirteenth figure, wherein the thirteenth Figure (a) Right eye filter (b) Left eye filter. Everyone's iris shape is close to a circle, so this filter is suitable for most people. Because the camera is positioned in the middle of the eye near the front of the eyebrow, the filter is designed to remove the skin and enhance the integrity of the inner edge of the iris. The image before filtering is as shown in the fourteenth (a) picture, and the filtered image is as shown in the fourteenth (b) picture, in which the outer edge of the iris and the outer edge of the part are retained.

According to the above embodiment, a binarization 150 is performed to convert the right filtered image 240R and the left filtered image 240L into a two-dimensional coordinate image, a right binary image 250R and a left binary image 250L, respectively. It can be seen that most of the skin is filtered by the filter, only the iris part is left, and some of the image is damaged due to the reflective relationship. Therefore, the binarization threshold T ₀ is used in the method. A threshold value T ₀ greater than the binarization threshold exhibits a black dot, and a threshold value smaller than the binarization threshold T ₀ presents a white point, wherein a set of embodiments is to use the following formula (2-2) to treat all the pixels that are not pure white. The pixels are highlighted in full black, as shown in Fig. 15 (a) filtered image (b) binarized image, wherein a set of embodiments is a binarization threshold of 20.

According to the above embodiment, an iris center estimation 170 is performed, and three points of the right binary image 250R and the left binary image 250L are used to form a triangular circumcircle, and an iris center is estimated.

According to the above embodiment, as shown in the sixteenth and seventeenth views, the method of finding the center of the iris by the right binary image 250R is as follows:

Step one: Find the black point from the right binary image 250R (the black point is at most the sum of the pixel values) and the right first vertical line 311R of the nose.

Step 2, starting from the right first vertical line 311R and finding the right second vertical line 312R with the largest white point on the nose side, wherein one set of embodiments is a right second vertical line with a pixel value of 255. 312R.

Step three, found by the right second vertical line 312R to the ear side The first black dot is a right first certain point 331R.

In step four, the lowest black point in the right binary image 250R is a right lowest point 341R.

In step five, the horizontal line and the right first horizontal line 321R are found by using the middle point of the line connecting the right first determining point 331R and the right lowest point 341R.

Step 6. The intersection of the right first horizontal line 321R and the right first vertical line 311R is found along the right first horizontal line 321R toward the nose side, and the end point is the intersection of the right first horizontal line 321R and the right second vertical line 312R, the last line on the line. The black dot is a right second determination point 332R.

In step 7, the right first horizontal line 321R moves to the right side of the right binary image 250R to find the first vertical line and the right third vertical line 313R. One set of embodiments is that the pixel value is 255. The vertical line is a right third vertical line 313R.

Step 8: Finding the first black point from the right third vertical line 313R to the nose side of the right binary image 250R is a right third determining point 333R.

Step IX, a right triangle circumcircle 335R formed by the right first determining point 331R, the right second determining point 332R and the right third determining point 333R, the center of which is a right iris center 334R.

According to the above embodiment, as shown in FIG. 18 and FIG. 19, the method of finding the center of the iris by the left binary image 250L is as follows:

In step one, the all-black (all black is the sum of the pixel values) and the left first vertical line 311L of the nose are found from the left binary image 250L.

Step 2, starting from the left first vertical line 311L and finding the first left second vertical line 312L with the largest white point on the nose side, wherein one set of embodiments is a left second vertical line with a pixel value of 255. 312L.

In step three, the first black point is found by the left second vertical line 312L toward the ear side as a left first certain point 331L.

In step four, the lowest black point in the left binary image 250L is a left lowest point 341L.

In step five, the horizontal line and the left first horizontal line 321L are found by using the middle point of the line connecting the left first determining point 331L and the left lowest point 341L.

Step 6. The intersection of the left first horizontal line 321L and the right first vertical line 311L is found along the left first horizontal line 321L toward the nose side, and the end point is the intersection of the left first horizontal line 321L and the left second vertical line 312L, and the last line on the line. The black dot is a left second determination point 332L.

Step 7: moving from the left first horizontal line 321L to the left binary image 250L to the ear side to find the first vertical line and the left third vertical line 313L, wherein one set of embodiments is that the pixel value is all 255. The vertical line is a left third vertical line 313L.

Step 8: Finding the first black point from the left third vertical line 313L to the nose side of the left binary image 250L is a left third determining point 333L.

Step 9: A left triangle circumcircle 335L formed by the left first determined point 331L, the left second determined point 332L, and the left third determined point 333L, the center of which is a left iris center 334L.

According to the above embodiment, wherein the radius of the right triangle circumscribed circle 335R and the left triangle circumscribed circle 335L is larger than one third of the width (600 to 120) pixels of the region of interest, and then the region of interest (ROI) 120 is set.

According to the above embodiment, a set of embodiments is such that the radius of the right triangle circumscribed circle 335R and the left triangle circumscribed circle 335L is greater than one third of the width of the region of interest 240 pixels, and then the region of interest (ROI) 120 is set.

According to the above embodiment, wherein the radius of the right triangle circumcircle 335R and the left triangle circumcircle 335L is less than one third of the width (600 to 120) pixels of the region of interest, then a screen gaze point estimate 170 is performed.

According to the above embodiment, a set of embodiments is such that the radius of the right triangle circumscribed circle 335R and the left triangle circumscribed circle 335L is less than one third of the width of the region of interest 240 pixels, and then a screen gaze point estimate 170 is performed.

According to the above embodiment, wherein the radius of the right triangle circumcircle 335R is less than one third of the width of the region of interest (600-120), and the radius of the left triangle circumcircle 335L is greater than the width of the region of interest One-third of the (600~120) pixels, then a left-mapped compensation of 164L.

According to the above embodiment, wherein a set of embodiments is such that the radius of the right triangle circumcircle 335R is less than one third of the width of the region of interest 240 pixels, and the radius of the left triangle circumcircle 335L is greater than the width of the region of interest of 240 pixels. One third, then perform a left mapping compensation of 164L.

According to the above embodiment, wherein the radius of the left triangle circumcircle 335L is less than one third of the width of the region of interest (600-120), and the radius of the right triangle circumscribed circle 335R is greater than the width of the region of interest (600-120) One-third of the pixel is then subjected to a right mapping compensation 164R.

According to the above embodiment, wherein a set of embodiments is such that the radius of the left triangle circumcircle 335L is less than one third of the width of the region of interest 240 pixels, and the radius of the right triangle circumcircle 335R is greater than the width of the region of interest of 240 pixels. One third, then perform a right map compensation 164R.

According to the above embodiment, wherein, as in the twentieth diagram, the steps of the left map compensation 164L are as follows:

In step 1, the difference between the vertical direction position of the right iris center point 334R and the vertical direction position of the right first determination point 331R is calculated as a right vertical gap 351R.

Step 2, calculating a difference between the horizontal direction position of the right iris center point 334R and the horizontal direction position of the right first determination point 331R is a right water Flat gap 352R.

In step 3, the right vertical gap 351R is moved with the vertical position of the left first determined point 331L as a starting point, and the right horizontal gap 352R is moved to the left ear side to obtain a estimated left iris center point 336L.

In step four, the estimated left eye iris boundary is obtained by estimating the distance from the left iris center point 336L to the left first determining point 331L as a radius.

According to the above embodiment, wherein, as in the twenty-first figure, the steps of the right mapping compensation 164R are as follows:

In step 1, the difference between the vertical direction position of the left iris center point 334L and the vertical direction position of the left first determination point 331L is calculated as a left vertical gap 351L.

In step two, the difference between the horizontal direction position of the left iris center point 334L and the horizontal direction position of the left first determination point 331L is calculated as a left horizontal difference 352L.

In step 3, the left vertical gap 351L is moved from the vertical position of the right first determined point 331R as the starting point, and the left horizontal gap 352L is moved to the right ear side to obtain a estimated right iris center point 336R.

In step four, the estimated right iris boundary is obtained by estimating the distance from the right iris center point 336R to the right first determined point 331R as a radius.

According to the above embodiment, wherein a screen fixation point is performed Estimate 170, using a linear amplification method, using the position of the center of the iris to estimate the gaze point of the screen.

According to the above embodiment, the linear amplification method is a relationship between the position of the center of the iris and the fixed point of the four sides of the screen when the center of the screen is viewed by an iris center in advance. The location of the other iris center is estimated to be in the position of the viewing screen.

According to the above embodiment, wherein the above linear amplification method is to display several gaze sampling points as shown in the twenty-second diagram on the screen, a set of embodiments assumes that the screen size considered is 1280 x 720 pixels. When the uppermost boundary (position 8) and the lowermost boundary (position 2) are viewed by both eyes, the central coordinates of the iris are recorded as P _{p 8} ( x _{p 8} , y _{p 8} ) and P _{p 2} ( x _{p 2} , y, respectively). _{p 2} ), and calculate the iris center pixel difference between the 8th and 2nd y coordinates, which is called Δ y ; then, when viewing the leftmost boundary (position 4) and the rightmost boundary (position 6), the iris is recorded. The central coordinates are P _{p 4} ( x _{p 4} , y _{p 4} ) and P _{p 6} ( x _{p 6} , y _{p 6} ), respectively, and calculate the iris center pixel difference between the 4th and 6th x coordinates, which is called Δ x ; finally at the middle point of viewing The iris center coordinates P _c ( x _c , y _c ) are also recorded. Then calculate the conversion ratio using equations (2-3) and (2-4), and then convert the eye line of sight P' ( x', y' ) and (2-6) and (2-6) respectively. The relationship between the x coordinate and the y coordinate between the corresponding iris centers P ( x, y ).

Where R _x is the conversion ratio in the horizontal direction, and R _y is the conversion ratio in the vertical direction.

The success rate of iris boundary detection according to one of the above embodiments is as follows.

According to another preferred embodiment of the present invention, as shown in FIG. 24, the present invention provides an eyeball positioning system 400. The eyeball positioning system 400 includes a bracket 405 for stabilizing the head of the subject. . A display screen 407 is provided for viewing by a subject. A camera 410 is attached to the bracket 405 with an adjustable support bar. An arithmetic processing device 420 is configured to receive a recorded image 210 by the camera 410, and then perform a set focus area (ROI) 120, which is obtained from the recorded image 210 according to the set area. Right capture image 220R and left capture image 220L, perform a grayscale 130. Turn right capture image 220R and left capture image 220L into a right simple image 230R and a left simple image 230L, perform a filtering 140 to a right The noise removal of the simple image 230R and the left simple image 230L becomes a right filtered image 240R and a left filtered image 240L, and a binarization 150 is performed. The filtered image 240R and the left filtered image 240L become a right binary image 250R and a left binary image 250L. An iris center estimation 170 is performed, and three points of a right binary image 250R and a left binary image 250L are respectively formed to form a triangular circumcircle, and a right iris center 334R and a left iris center 334L are estimated. A screen gaze point estimate 170 is performed to estimate the gaze point by at least one of a right iris center and a left iris center.

According to one of the above embodiments, the bracket 405 adjusts the height of the four support legs below the bracket 405 by using a level placed at the chin position, so that the crossbar supporting the head can be level, as shown in Fig. 25. Before the experiment, it can be adjusted according to the height that the subject feels comfortable with the chair and the screen.

The detection success is defined as the iris boundary of both eyes, including one eye successfully taking another eye mapping compensation method to estimate, that is, the coordinate information of the center point of both eyes. The formula (2-7) is used to calculate the success rate S ₁ .

The testee is five people, watching five points on the screen at a distance of 50 cm from the screen. Each person grabs 10 frames per second, grabs 10 seconds per point, and has 500 frames at 5 points. The experimental calculation method will be excluded. The image of the subject when the subject is closed, so the total number of valid images may be less than 500 frames. Five points on a 1280x720 screen, as shown in Figure 23, each subject is The success rate of each point is presented in Table 1. The experimental results showed that two of the five subjects had data on both eyes in all of the images in the effective image, which is 100% detection success rate, including the captured eye and the mapped compensation eye. Among the other three subjects, images with no data were counted as invalid images, and images without data occurred at the left and right points, but the success rate was over 96%. In addition, the average success rate of each subject's attention to each point is also above 99%. If analyzed in units of each point, each point has a probability of success of more than 97%. Calculated by the whole, it is calculated by all effective images, and the success rate is also 99.75%. The possible cause of the detection failure is that a slight amount of residual image is caused during the process of closing the eye, and the eyelids are opened more in the captured image, and the iris is more damaged after the residual image is processed, resulting in detection failure.

The success rate of both eyes is obtained. In the experiment, there will be three results. The first one is that each of the two eyes successfully captures the iris boundary. This is the result of this paragraph. The second one is a The eye succeeded, and the second eye used map compensation to estimate where the iris boundary was. The third one did not capture any of the eyes. The experiment is based on the five-point chart used in the previous section, and the success rate is calculated by (8). Table 2 shows the test results of five subjects.

It can be seen that the five subjects have some failures in different positions. In general, the failure focuses on the left and right points, but the lowest is 87% success rate, and the highest is 100%. Each of the subjects has an average success rate of more than 96% at five points. Each point has a success rate of more than 95%, and the overall calculation can reach 98.39%. rate. The possible cause of failure of one eye is that the subject's eyes are not level, causing the shadow of one eyelash and the noise of the inner or outer corner of the eye to be connected together, causing unreasonable conditions and causing detection failure.

The probability of mapping compensating eyes appears. This section presents one eye successfully and the other eye uses the map compensation method to estimate the probability of the iris boundary. The subjects were five, and the five-point chart was used for the experiment. The probability P was calculated as (2-9), and the results are shown in Table 3.

This section is the probability of the map compensating eye, which can echo the two subsections above. When one eye is successfully captured and the other eye is not captured, a mapping compensation eye is generated. The data display is also concentrated on the left and right points. The highest probability of 13% occurs when the 4th subject sees the following point. This is because the subject's eyelashes are longer and the position of the lens is placed. When looking down, the upper eyelids will naturally move down, resulting in more shadows and an unreasonable size of the iris, so it is replaced by mapping compensation. The probability that each subject produces a map compensation eye is below 4%. At each point of view, after deducting the 4th and 5th subjects, the highest and second highest chances occur in the left and right eyes. This is because the camera is placed in front of the eyebrow and when looking left or right. There must be one eye away from the camera. The eye that is far away from the camera is more likely to have a map compensation effect, but the highest chance of each point is below 4%. Calculated by the total number of valid frames, a probability of 1.35% will result in a map compensation eye.

The accuracy of the line-of-sight estimation, the estimation of the line-of-sight point is obtained by adding the coordinate information of the binocular ROI, dividing it by two and then mapping it to the screen by linear amplification, and using the five-point chart to do the experiment, and The horizontal and vertical errors of the mapped point coordinates and the test point coordinates are calculated using equations (2-12) and (2-13). Table 4 shows the average error of five test points viewed by five subjects. Each test point records the coordinates of 10 observations after the subject himself determines to concentrate on watching, and then calculates the data with 10 data.

Where H ₆ is the horizontal error and V ₆ is the vertical error. The results are shown in Tables 4 and 5, respectively.

Table 4 shows the error in the horizontal direction. It can be found that the error of the left and right points is relatively large, and the average is close to 7%, while the error at the upper and lower points is about 4%. When the eye looks left or right, the change of the x coordinate is More dramatic, the y coordinate will not change, so it will be larger when calculating the horizontal error. Table 5 shows the error in the vertical direction. It can be found that most of the errors of the upper and lower points are relatively large, with an average of 4% to 6%, and the left and right points are within 3%. When the eyes are up or down, the change of the y coordinate is more severe. The x coordinate comparison will not change, so the vertical error will be larger when looking at the up and down points. The average error in the overall horizontal direction is 4.5%, and the average error in the vertical direction is 3.5%.

The next line of sight is based on the information of the single-eye ROI and is divided into the left and right eyes. The rectangular five-point diagram of 1280x720 and the five-point diagram of 720x720 are used to compare the horizontal and vertical errors. . The results are shown in Tables 6 to 13, respectively.

The experimental difference between Table 6 and Table 7 is that the observed window is rectangular or square. The coordinates of the right eye are used for the line-of-sight experiment and the horizontal error is calculated. It can be found that the error is significantly larger when viewing the right point in Table 6. This is because the outer edge of the right eye is the farthest when looking at the right point, resulting in more deformation on the outer edge, and the error mapped to the screen is also The change is relatively large, and the smallest error occurs when looking at the intermediate point. This is because when doing the linear amplification method, it is necessary to calculate the intermediate point as the base point to the periphery. When correcting, there is a record at the intermediate point, so the error is better. small. Table 7 is an experiment with a square five-point diagram. It is also a large horizontal error when looking at the right point, but it is also lower than the error of viewing the right point of the rectangular five-point diagram. This is because the right side of the square is compared with The right side of the rectangle is coming Close to the left, that is, the information captured by the camera on the outer edge of the iris will be more complete. The smallest error is also present when viewing the intermediate point. This is also because the correction is performed when the intermediate point is corrected, and the error in viewing the intermediate point in the experiment is smaller.

The difference between Table 8 and Table 9 is that the observed window is rectangular or square. The experiment uses the right eye information to make the line of sight detection and calculate the vertical error. In the rectangular five-point diagram, we can see that the maximum error value falls on the upper point and the lower point. The square five-point diagram also has the same result. This is because the two points in the rectangle and the square five-point diagram are in the same position. It has nothing to do with width, but the vertical error of the left and right points can observe that the rectangle has a similar result to the square five-point diagram. This is because the horizontal coordinate changes will be larger and vertical when looking at the left and right points. The change in coordinates is relatively small, so the gap is not large. The minimum error is at the middle point, because the rectangle and the square five-point map should be corrected at the middle point, and the point where the drop point error is the smallest.

Tables 10 and 11 show the horizontal error of the rectangular and square five-point graphs with the coordinate information of the left eye. The maximum error in Tables 10 and 11 occurs when the left point is seen, but the square five-point map is The error of the rectangular five-point diagram is 1% smaller. These changes are because the left point of the square five points is closer to the middle than the left point of the rectangular five-point diagram. When using the right eye coordinate information to do the drop point experiment There are also the same inferences. Some of the smallest errors fall on the upper or lower point of view. This is because the amount of movement in the horizontal direction is small when viewing these two points, but the smallest average error falls when viewing the intermediate point.

Table 12 and Table 13 are the vertical errors of the gaze point on the rectangular five-point diagram and the square five-point diagram. It can be observed that the maximum error of the two tables appears when viewing the upper point and the lower point, because these two points are The positions of the two five-point diagrams are the same, and the vertical coordinate movements when viewing the upper point and the lower point are more severe, the horizontal coordinates are hardly changed, and the results of the vertical errors of the left and right points are observed. It’s about the same, because when you watch these two points, the horizontal coordinates are more dramatic, and the vertical coordinates change less. The smallest error occurs when viewing the intermediate point.

Table 14 is a table of error comparisons between the heterochromatic boundary method and the three-point circle method. The heterochromatic boundary method needs to be manually applied to the iris at the heterochromatic boundary. Partially select six pixels and take the largest pixel value as a parameter. Then, manually select six pixels in the part of the heterochromatic border near the white of the eye and take the pixel value as the minimum parameter, and then set the two parameters as the valve. Values are used to make the color boundary and the watershed of the eye white and the iris. Finally, after finding the iris area, the coordinate center of gravity is taken as the center point of the iris, and each subject needs to click to generate its own parameters. The three-point circle method proposed by the invention can detect the limbus completely in an automatic manner, and the three-point circle method can be used for detecting the iris of different subjects. In the paper [Wang Junxiong, the rapid visual detection system using CCD image and the application of the human interface of the disabled, the master's thesis of the Institute of Electrical Engineering, National Tsinghua University, 1998], the experimental results obtained by the heterochromatic boundary method show the average horizontal error. At 19.1%, the average vertical error is 18.3%, and the experimental results obtained by the three-point circle method show an average horizontal error of 4.5% and an average vertical error of 3.5%. In the experimental method, the heterochromatic boundary method and the three-point circle method directly capture the facial image by the camera. In addition, the heterochromatic boundary method is performed after the body is fixed against the back of the chair, and the three-point circle method uses the shelf. After the head is fixed, the linear magnification method is used to estimate the gaze drop point. However, it should be noted that the images used in the experiments are not the same, and the screen size is different. Therefore, the results are for reference only and do not represent a fair actual gap. Nevertheless, the excellent performance of the proposed method can be seen roughly.

Table 14. Comparison of the average error obtained by the method proposed herein and another method.

In summary, the eyeball positioning method and system of the present invention has higher detection rate than the current product on the market, and the invention improves the shortcomings of the old eyeball positioning method and system, and has obvious market value. Obviously, many modifications and differences may be made to the invention in light of the above description. It is therefore to be understood that within the scope of the appended claims, the invention may be The above are only the preferred embodiments of the present invention, and are not intended to limit the scope of the claims of the present invention; all other equivalent changes or modifications which are not departing from the spirit of the present invention should be included in the following claims. Within the scope.

100‧‧‧ eye positioning method

110‧‧‧Open video

120‧‧‧Set the area of interest

130‧‧‧ Grayscale

140‧‧‧ Filter

150‧‧‧ Binarization

160‧‧‧Iris Center Estimate

164L‧‧‧left mapping compensation

164R‧‧‧right mapping compensation

170‧‧‧ Screen gaze estimation

Claims (20)

An eyeball positioning system, comprising: a display screen for providing a subject for viewing; a camera, the camera is connected with an adjustable support rod on the support; and an arithmetic processing device, The processing device is configured to receive a recorded image by the camera, and then perform a setting of the attention area, by extracting a right captured image and a left captured image according to the set region from the recorded image, and performing a gray The grading transforms the right captured image and the left captured image into a right simple image and a left simple image, and performs a filtering to remove the right simple image and the left simple image noise into a right filtered image and a left Filtering the image, performing a binarization, and converting the right filtered image and the left filtered image into a right binary image and a left binary image, and performing an iris center estimation by using the right binary image and the left binary value Three points are taken from the image to form a circumscribed circle of the triangle, and a right iris center and a left iris center are estimated to perform a screen fixation point estimation, by the right iris center and the left rainbow Both the center of the screen at least one of the gaze point position conjecture. The eyeball positioning system according to claim 1, wherein the eyeball positioning system comprises: a bracket for stabilizing the head of the subject. According to the eyeball positioning system of claim 1, wherein the display screen is for receiving and displaying the information provided by the arithmetic processing device, and the resolution of the camera is 1280 x 720 pixels. An eyeball positioning method includes: performing an open video to receive a recorded image; and performing a setting of the attention area by extracting a right captured image and a left capture from the recorded image Image; performing a grayscale transformation, converting the right captured image and the left captured image into a right simple image and a left simple image; performing a filtering to remove the right simple image and the left simple image noise into a right filtered image and a left filtered image; performing a binarization, converting the right filtered image and the left filtered image into a right binary image and a left binary image; and performing an iris center estimation by using the The right binary image and the left binary image take three points, respectively forming a triangular circumcircle, and estimating a right iris center and a left iris center. According to the eyeball positioning method of claim 1, wherein the eyeball positioning method comprises: performing a gaze point estimation, and estimating the gaze by at least one of the position of the right iris center and the left iris center Point location. According to the eyeball positioning method of the first and fourth aspects of the patent application scope, the received image received in the above-mentioned open video is a resolution of (2000~800) x (1500~300) pixels, wherein a set of implementation examples is 1280x720 pixels, and the included image is a range of fixed faces. According to the eyeball positioning method of the first and fourth aspects of the patent application, wherein the setting of the attention area is to directly set the size of a right focus area and a left focus area from the recorded image, and then the position The size of the right focus area and the left focus area and the position between the right focus area and the left view image are captured, wherein the right focus area and the left focus area are set to a fixed size of (600~120) x ( 300~60) pixels, and then the distance from the lens to the face, according to each person's different adjustments to the lens to grab the limbus, this size just does not contain the eyebrows inside, will not be the corners Framed, there is also a certain distance between the two areas of interest to exclude the bridge of the nose from the area of interest, and the head is fixed during the set process. According to the eyeball positioning method of claim 7, wherein the above-mentioned area of interest is 240 x 120 pixels. According to the eyeball positioning method of the first and fourth aspects of the patent application, the right captured image and the left captured image become the right simple image and the left simple image. The recorded image information is a full-color image, which is converted into gray-scale information by Y = R × 0.299 + G × 0.587 + B × 0.114, leaving only the brightness information, where Y represents brightness and RGB represents red, green and The chromaticity of blue, in which a set of embodiments is the information of the converted brightness from 0 to all black 255. According to the eyeball positioning method of the first and fourth aspects of the patent application, wherein the right simple image and the left simple image noise are used by versus The mode removal becomes the right filtered image and the left filtered image. According to the eyeball positioning method of the first and fourth aspects of the patent application, wherein the right filtered image and the left filtered image are represented by a threshold value greater than a binarization threshold, which is smaller than the binarization threshold. The value presents a white point, which is converted into the right binary image and the left binary image, respectively. According to the eyeball positioning method of claim 11, wherein the above-mentioned binarization threshold is 20. According to the eyeball positioning method of the first and fourth aspects of the patent application, the method for finding the center of the iris by the right binary image is as follows: Step 1: Find the most black point and the most vertical of the nose from the right binary image. The line is a right first vertical line; in step 2, the vertical line from the right first vertical line to the nose side to find the first white point is a right second vertical line; step three, by the right The second vertical line finds the first black point as the right first certain point on the right ear side; in step 4, the lowest black point in the right binary image is the right lowest point; step 5, using the right first certain point The middle point of the line connecting the right lowest point is found to be a right first horizontal line; in step 6, the intersection of the right first horizontal line and the right first vertical line is found along the right first horizontal line toward the nose side, and the end point For the intersection of the right first horizontal line and the right second vertical line, the last black point on the line is a right second certain point; in step 7, the right first horizontal line goes to the right binary image right ear side Move to find the first white point The straight line is a right third vertical line; in step 8, the first black point is found from the right third vertical line to the right side of the right binary image as a right third determining point; step nine, the right first A right triangle circumscribed circle formed by the determined point, the right second determined point and the right third determined point, the center of which is a right iris center. According to the eyeball positioning method of the first and fourth aspects of the patent application, the method for finding the center of the iris by the left binary image is as follows: Step 1: Find the one with the most black spots and the most nose from the left binary image. The first vertical line on the left; step two, starting from the first vertical line on the left, find the first line on the side of the nose The vertical line with the most white point is a left second vertical line; in step 3, the first black point is found by the left second vertical line to the left ear side as a left first certain point; step four, in the left two The lowest black point in the value image is the left lowest point; in step 5, the horizontal line is used as the left first horizontal line by using the middle point of the left first determined point and the left lowest point connection; step six, by the left The intersection of a horizontal line and the right first vertical line is found along the left first horizontal line toward the nose side, and the end point is the intersection of the left first horizontal line and the left second vertical line, and the last black point on the line is one left Two determining points; step seven, and then moving from the left first horizontal line to the left ear side of the left binary image to find the first vertical line of the white point is a left third vertical line; step eight, from the left The third vertical line finds the first black point on the nose side of the left binary image as a left third determination point; step IX, the left first determination point, the left second determination point, and the left third determination point A left triangle circumscribed by a point, the center of which is a left iris . According to the eyeball positioning method of the first and fourth aspects of the patent application scope, wherein the radius of the circumscribed circle of the right triangle and the radius of the circumscribed circle of the left triangle is greater than one third of the width of the region of interest (600 to 120), and then setting attention region. According to the eyeball positioning method of the first and fourth aspects of the patent application scope, wherein the radius of the right triangle circumscribed circle and the left triangle circumscribed circle is smaller than the attention area One-third of the width (600~120) pixels, and then a screen gaze point estimate. According to the eyeball positioning method of claim 13, the radius of the circumscribed circle of the right triangle is less than one third of the width of the region of interest (600~120), and the radius of the circumscribed circle of the left triangle is larger than the focus. The width of the region (600~120) is one-third of the pixel, and then a left map compensation is performed; the left map compensation step is as follows: Step 1: Calculate the vertical direction position of the right iris center point and the right first certain point The difference between the vertical position is a right vertical gap; in step 2, the difference between the horizontal position of the right iris center point and the horizontal position of the right first determined point is calculated as a right horizontal difference; The vertical position of the left first determined point moves the right vertical gap as a starting point, and then moves the right horizontal gap to the left ear side to obtain a estimated left iris center. According to the eyeball positioning method of claim 13 and 14, wherein the radius of the left triangle circumscribed circle is less than one third of the width of the region of interest (600-120), and the radius of the right triangle circumscribed circle is larger than the focus The width of the region (600~120) is one-third of the pixel, and then a right map compensation is performed; the right map compensation step is as follows: Step 1: Calculate the vertical direction position of the left iris center point and the left first determination point The difference between the vertical position is a left vertical gap; in step two, the left iris center point is calculated The difference between the horizontal direction position and the horizontal position of the left first determined point is a left horizontal difference; in step 3, the left vertical gap is moved with the vertical direction position of the right first determined point as a starting point, and then to the right ear Move the left horizontal gap to the side to get a estimated right iris center. According to the eyeball positioning method of the first and fourth aspects of the patent application, wherein the above-mentioned screen fixation point estimation is performed by using a linear amplification method, using at least one of the right iris center and the left iris center to estimate the screen annotation. Viewpoint location. According to the eyeball positioning method of the first and fourth aspects of the patent application, wherein the above linear amplification method is to measure the position of the center of the iris and the four sides of the screen by pre-measuring an iris center to view the fixed points on the four sides of the screen. The positional relationship is then used to estimate the location of the viewing screen from the location of the other iris center.