TW499306B - Eye pupil tracking method with crisscross line positioning functions - Google Patents

Abstract

An eye pupil tracking method with crisscross line positioning functions is applied on a display mounted on the head or other device mounted on the head for detecting the movements of an eyeball. Through applying a CCD solid-state camera, we obtain 2-D linear signals of an eyeball; using an image processing card and a computer program to analyze and process the image signal; using a crisscross line in the image of the eyeball to determine the position of the eyeball; using the amount and the position of the pixels with a low gray scale on the crisscross line to calculate the perpendicular and horizontal angles in the viewing direction of the eyeball and the conscious blinking movements of the eyeball; processing in real time the movement information of the eyeball; and providing the locus data of the eyeball on the computer screen, thereby forming a high efficiency man-machine interface.

Description

Printed by the Consumer Cooperative of the Central Standards Bureau of the Ministry of Economic Affairs 499306 A7 __ ΒΊ V. Description of the Invention (1) [Technical Field] The present invention relates to a pupil tracking method with a crosshair positioning function, especially a method that can correctly find the center point of the pupil. And it really does detect pupil movements of eyeball movement and blinking for data processing. [Previous technology] Due to the improvement of human living standards, various high-tech home appliances have also become popular. Existing virtual reality systems, head-mounted displays or other viewing and browsing devices have also emerged. Current head-mounted displays System designers often expect to obtain information about the user's eye movement behavior as a reference for the improvement of the overall system architecture, and these materials are also for the study of ophthalmology, psychology, physiology, communication, media art ... The subject that experts and scholars are eager to explore. Regarding the measurement of eye movement behavior, there is a common eye tracking system, which can detect the situation of eye movement in real time. At present, the pupil tracking system can be classified into piezoelectric pupil tracking system, optical pupil tracking system and Three types of magnetic field pupil tracking system. Among them, the piezoelectric pupil tracking system uses the change in intraocular pressure to detect the direction of eye movement. This detection method mainly uses a piezoelectric material to stick around the eye, and then uses a piezoelectric sensor to convert the intraocular pressure into electrical energy. Signal, and the obtained voltage signal is processed and interpreted. However, because the piezoelectric pupil tracking system uses current sensing, it is easy to affect the interpretation of the signal due to sweating and other factors; while the magnetic field pupil tracking system and piezoelectric tracking The system is similar to 'forming a magnetic field around the eyeball to judge the movement of the eye; compared to the piezoelectric pupil tracking system, the optical pupil tracking system has greater flexibility. The imaging of the optical pupil tracking system can be divided into 3 points. Paper size applies Chinese National Standard (CNS) A4 specification (210 father 297 male) (Please read the precautions on the back before filling this page)

Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs 499306 A7 B7 V. Description of the invention (2) Type, line type and matrix. Although the spot scanning speed is fast, it is easy to produce unrecognizable situations when the amplitude of eye movements is too large. As for the matrix, the coordinates of eye gaze are directly determined, and changes in eye size can be recorded at the same time. The optical pupil tracking system is assembled using a commercially available head-mounted display, a pinhole C C D (charge coupled device), a light source, and a personal computer. First, the screen on a personal computer screen is converted to the screen of a head-mounted display, so when we put on the head-mounted display, the image on the screen is exactly the same as that on the screen of a personal computer. The hole CCD is set under the head-mounted display. The focus and light are adjusted to make the captured eyeball image the clearest. When the eyeball image is transmitted to the image capture card through the pinhole CCD, the eyeball image is frozen, and then passes through the existing The image analysis algorithm analyzes and judges the coordinates that the eyeball is currently looking at, and then transmits the analyzed coordinates to the personal computer on the viewing screen via the RS232 communication protocol, and drives the mouse to execute the control instructions. Through the application of the CCD solid-state camera, the two-dimensional linear image signal of the eyeball obtained by photography is analyzed and processed by an image processing card and a computer program, and the position of the eyeball is determined by using the center point of the eyeball image, and the vertical and horizontal angles of the eyeball gaze direction are converted. Then the eye movement information is processed in a real-time manner. Finally, the eye track data can be provided on the computer screen. Although the existing pupil tracking system has been able to analyze the user's eyeball behavior, the optical pupil tracking method is often susceptible to the shadow around the eye or the gray scale of the eyebrows when analyzing and obtaining the center coordinates of the pupil, which can cause misjudgment of the pupil. The problem of the center position, and the human body ’s eyeballs and blinking. The paper size applies the Chinese National Standard (CNS) A4 specification (210X 297 mm) (please read the precautions on the back before filling this page), 1 '丨 ¾ 499306 A7 B7 V. Description of the Invention (3) The movements will cause the misoperation of the optical pupil tracking method. Moreover, the existing optical pupil tracking methods are mostly of low resolution, and it is easy to cause excessive errors for the large-scale movement of the eyeball, and Problems that lead to misjudgments, or even completely untraceable. [Objective and Effect] The main purpose of the present invention is to solve the above-mentioned problems and provide a pupil tracking method with a crosshair positioning function, which mainly uses the crosshairs in the pupil image to calculate and locate the crosshair center point and the pupil. The distance between the center points, and a weighting factor is added in the calculation to correctly calculate the coordinates of the center point of the pupil, and then the number of pixels and the position of the low gray scale crosshairs in each image obtained by photography are changed. To convert the vertical and horizontal angles of eye gaze direction, and conscious blinking movements, to achieve the effect of detecting eye movements and blinking movements. A second object of the present invention is to provide a pupillary chase method that can be used to determine the ability of a user to operate a pupillary chase system by using the four correction coefficients of upper, lower, left, and right obtained in a five-point correction method. . Yet another object of the present invention is to provide an improvement ratio calculated by using the numbers of four correction coefficients of upper, lower, left, and right obtained in the five-point correction method, so that the same user can be compared before and after. Tracing method for improving the operating ability of the pupil tracking system. [Technical content] The present invention provides a pupil chasing method with a crosshair positioning function, which mainly includes the following steps: Capture pupil image: Use CCD camera to capture the eye shadow of the user. The paper size is applicable to the Chinese National Standard (CNS) A4 specification ( 210X 297 mm) (Please read the notes on the back before filling out this page. Printed by the Employees 'Cooperative of the Central Standards Bureau of the Ministry of Economic Affairs. Printed by the Consumers' Cooperative of the Central Standards Bureau of the Ministry of Economy. Printed by 499306 A7 s_____B7__ V. Description of the invention (4) The obtained two-dimensional linear image signal of the eyeball is analyzed and processed by the image processing card and computer program. Determination of the pupil center: When processing the eyeball image, the X and Y coordinates are binarized and classified. When the element of a pixel is gray When the degree 値 is less than the set ，, it is regarded as the pupil position, and a weighting factor is used in the judgment of the poem position to avoid misjudging eyebrows or shadows as pupils. Then select the pupil image and use the cross lines in the pupil image to calculate And locate the distance between the center point of the crosshair and the center point of the pupil, and use the weight method to calculate The coordinates of the central point of the perforated hole gradually increase the number of gray levels in the pixel to the outside to distinguish the position of the eyeball and the pupil. Tracking the position of the pupil · · Subtract a continuous image of the user's eyeball captured by the CCD camera. Changes in the number and position of pixels in the low gray scale of the crosshairs in each image are used to convert the vertical and horizontal angles of the eye gaze direction and the conscious blinking action, and then the information about the eyeball rotation is real-time The information that makes the eyeballs turn on the computer screen is used as the track of the cursor movement, and the conscious blinking motion is the action of the mouse pressing the option. The above and other objects and advantages of the present invention are not difficult to obtain from the following. The detailed descriptions of the selected embodiments and the drawings are used to gain in-depth understanding. Of course, the present invention allows for differences in the arrangement of other parts, or the arrangement of the parts, but the selected embodiment is in this specification. It will be described in detail, and its structure will be shown in the drawings. I Detailed description of the embodiment] Please refer to FIG. 1 and FIG. 2. The present invention is a type having a + word line. Set 6 This paper size applies Chinese National Standard (CNS) A4 specification (210X 297 mm) (Please read the precautions on the back before filling this page)-Clothing

'• IT ^ 306 ^ 306 Printed by A7 ____, _ B7 in the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economics. (5) A pupil tracking method for the function. The method of the present invention uses a matching pupil position in this embodiment. The tracking system consists of the following equipment:

1 · A personal computer (CPU: PentiumH 350, memory: 64MB, graphics card: S3 savage 3d / m with τν-out), used to execute the pupil tracking program, assistive program and eye movement recording program of the system. 2 · One image capture card · · Invi deo / PCi of FOCUS Enhancements, the purpose is to convert the video signal of the CCD camera into a digital signal that can be processed by a computer. 3. A CCD camera is used to capture the user's eyeball image. The considerations for choosing it are: small size, light weight, low illumination and more pixels. The effect of color CC is better than that of black and white. 4. A 5W bulb is used as the light source of the system. 5. A head-mounted display for users to watch the computer screen, including performing the initial calibration of the system, operating eye control programs and viewing other screens for analyzing eye movement data. 6 · Image processing software: Borland C ++ Builder. The purpose of adopting a head-mounted display in the present invention is to overcome a disadvantage common to general video system-type pupil chasing devices, that is, the relative position of the user's head and the camera must be kept fixed, otherwise errors or misjudgments may occur. The advantage of the head-mounted display is that it is fixed on the head, so the relative position between the user's head and the camera will not change. The pupil chaser system used in the present invention is a pinhole CCD camera under the display screen of the head. As shown in Figures 1 and 2, when using 7 paper sizes, the Chinese National Standard (CNS) A4 regulations apply. (210X297 mm) (Please read the notes on the back before filling out this page) Order 499306 5_ Printed by the Consumer Standards Cooperative of the Central Standards Bureau of the Ministry of Economic Affairs A7 B7 Invention Description (s) The person wears a head with a display 10 1 While browsing The CCD camera 1 ο 2 will capture the image of the user's eyeballs and transfer it to the computer 104 for analysis and processing. The computer program will use the method of the present invention to calculate the center position of the sacral hole in the image. The center coordinates correspond to the coordinates of the screen, and eye movements can be used to control the position of the computer mouse. If the pupil tracking system is paired with an application with option screens, such as a voice database system, Chinese and English input programs, and a dialing system, the user can use simple eyes to send simple life conversations without using both hands. Voice, or use your eyes to enter a complete Chinese and English sentence, and even use the pupil tracking system to make phone calls, go online, etc. The pupil tracing method with crosshair positioning function of the present invention mainly includes the following steps: 1. Capture pupil image A: Use a CCD camera 102 on the lower part of the screen with a head 101 to capture a user's eyeball image, and obtain the obtained eyeball image. The two-dimensional linear image signal of the eyeball is transmitted to the computer 104. Please refer to Figs. 3A to 3I, which are images of the pupils of the eyeballs of the user in various fields of view taken by the CCD camera 1 02. From Figs. 3a to 3i, the eyes are above, below, left, and right. , Upper left, upper right, lower left, lower right, and front view of the eye pupil image, because the CCD camera 102 is set at 40 degrees below the head-mounted display 1101, so the eyes Even when looking straight ahead, the pupil image of the eyeball is elliptical, but instead looks circular when looking down the screen. This is a characteristic of our architecture. 8 This paper size is applicable to China National Standard (CNS) A4 (210X297 mm)

(Please read the precautions on the back before filling in this page. J Order 499306 A7 B7 __ _ '_______________ ____ _ _______________ ________. ------------- V. Description of the invention (7) 2. Determination of pupil center B: When judging the eyeball area, because we use the K and Y coordinate binary classification when processing the eyeball image, when the gray level 値 of a certain pixel (pi xel) is less than the set 値, it is regarded as the pupil position. In order to make the system determine the pupil position more accurately to avoid mistakenly identifying eyebrows or shadows as pupils, a weighting factor is used when the system judges pupil positions to avoid misjudging eyebrows or shadows as pupils. Please refer to Section 4 Figure. The method of image processing in this method is on the captured pupil image. Two tangent lines are formed on the pupil circle image with the cross of two-dimensional coordinates. We assume that the pupil is a circle, so we have a very An important principle can be used. The vertical bisector (normal line) of the center point of any tangent line on the circle must pass through the center of the countersink (circle), so the intersection of any two tangent lines It must be the center of the circle. We take the vertical bisector of the vertical and horizontal tangent lines. The two ends of the horizontal and vertical tangent lines (cross lines) are (× 1, Υ ·), (× 2, ΥΪ), (× 1, Υ1). (× *, Y 2), then the pupil center point (X e, Ye) is X e = (χ l + χ 2) / 2 Y e = (Y 1 + Y 2) / 2 Central Standards Bureau of the Ministry of Economic Affairs Printed by the employee consumer cooperative (please read the precautions on the back before filling this page). When looking for the pupil center point on each image captured when the eye moves, you only need to ensure the horizontal line X 1 and vertical line γ, The two lines must intersect with the pupil. Today's image input speed is about 12 to 30 frames per second. In terms of the size of our image, we only need to make each X · and γ * equal to the previous image. The pupil center point (χ'γ ») in the middle can ensure and move 9 paper sizes are applicable to the Chinese National Standard (CNS) A4 specifications (210X297 mm) Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs 499306 A7 B7 -______________ V. Description of the invention The pupils after (8) intersect, and the error will not increase due to the use of time After the distance between the center of the crosshair and the center point of the pupil (X e, Ye) is calculated, the weight method is used next, with the calculated pupil coordinates as the center, and the gray in the pixel (pi xel) is gradually increased to the outside. The number of degrees is used to distinguish the eyeball from the pupil position; • The grayscale weight of the pupil is added to the distance between the center point of the α-cross line and the center point of the pupil (X e, Y e) as D Bei 0 f (X e, Y e) 1 = D α + f (X e, Y e) 3. Track pupil position C: Analyze the continuous images of the user's eyeballs taken by the CCD camera 102, and use the low of the crosshairs in each image Changes in the number and position of gray pixels, to convert the vertical and horizontal angles of eyeball gaze direction, and conscious blinking, and then process the information of eyeball rotation in a real-time manner to make the eyeball rotate The information is used as a cursor movement track on the computer screen, and the conscious blinking action is the action of the mouse pressing the option. The algorithm of the previous step provides an extremely fast search method, but still faces the following two problems: a. The sampling information is small, and the effect of noise is increased in proportion. b. The impact of natural vibration on the human eye is still quite serious. 9 To overcome the cursor drift caused by the natural vibration of the human eye, we propose the following method of f: Let the cursor move direction be 10 (CNS) A4 ^ m (210X 297 ^ f) (Please read the notes on the back before filling this page)

, 1T 499306 Α7 Β7

i. Description of the invention (S v (〇x (,). where X (t) and y < t) are the difference between the coordinates of the pupil center at time t-1 and t, and watt is the direction of output. Provided by the following formula After we judge whether the direction of movement of the pupil is the same twice, v (/) _ v {t-ri) || v (〇 || \\ v (t-r7) \\ It is not difficult to understand if the The images show that the pupil moves in the same direction, but we can confirm that the higher the credibility of the pupil moving in this direction, the lower the confidence. Therefore, the greater the η, the more reliable. I designed the output tile according to this principle ( 〇 (Please read the notes on the back before filling this page) v (t)-v (t -1) m (k-η) m if n < k if n > k Decision, usually our suggestion 値 is m = i and k = 3. Then we go back and look at the formula for judging the direction of pupil movement. We mentioned earlier that the eyeballs will vibrate physiologically, so we proposed The formula is obviously not very reasonable, so we extend it to the following formula: v (/-η) v (〇v (t-η) Printed by the Consumers' Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs. Order Error 値, as long as it meets the above formula, we will call the same moving direction. The 5th figure is a schematic diagram of the algorithm to increase the stability of the cursor. We use 疋 rn = 1 and k = 2 as illustrations. The dotted line is based on the Chinese paper standard (CNS) A4 size (21 × 297 mm) applied to the paper size of the paper. The central government bureau of the Ministry of Economic Affairs's Consumer Cooperatives Co., Ltd. 499306 A7 ___B7 V. Description of the invention (10) Output after processing Cursor movement direction, it is not difficult to find that if the movement direction is the same, the output cursor will quickly catch up with the movement of the pupil, but if there is a change in the movement direction (such as the direction of V 3 is significantly larger than our allowable amount of change) , The output will delay the pace of following, so we can easily find our ability to suppress interference during the movement from Figure 5, in Figure 5: v7 •: represents the i-th measured movement (Input) •: Represents the i-th cursor movement (output) P: Represents the allowable error offset η: Represents the accumulated number of times in the same movement direction Represents the first input Result 2 where η = 0 during the first movement, so v3 # v2 = >; == 〇 The second movement direction is different from the third movement direction, so η is reset to 0, ν5- ρ < νβ < ν5 + ρ = > η = \ ^ νο6 = νβ The directions of five or six movements are within the tolerance range, and η = 1, so the input and output results are the same. See Figures 6 A and 6 B, Figure 6 Figure A is the image captured by the eye in the static state. The cross that intersects the entire pupil is the result of searching for the pupil. The result obtained is the center point marked by the white cross, which is the center point of the pupil. The intersection of the two search line segments This is the last pupil center, and the difference can be seen in Figure 6B. The square points above represent the results of our stability algorithm for pupil movement. Based on this, it can be confirmed when the eyeball moves. Track its movement track as the track of the computer screen's upstream mark. 0 12 This paper size applies to China National Standard (CNS) A4 specification (210 '/ 297 mm) (Please read the precautions on the back before filling this page )

499306 A7 B7 Consumption cooperation by employees of the Central Bureau of Standards of the Ministry of Economic Affairs Du printed According to experiments, eyes blinked to zero. The blinking time is intentional, and the current image is input. Therefore, the CCD camera 1 0 2 will surely have more than ten pupils. The blinking time is quite short. Therefore, we can use the judging ball to wink. (In 11 eyes, the pupil diameter will become smaller, that is, it will take a longer time to change eyes. It usually takes one or two velocities between about 12 and 3.0 frames per second. The captured pupil's pupil image The diameter of the hole will be zero, and there may be only one or two of them when the unconscious blinking is zero. The number of times the pupil diameter is zero is used to judge the eye blinking action as the function of the mouse press. A to 7C, which are the pupil diameter data of unconscious blink, conscious blink for one second and conscious blink for two seconds, respectively. The number of pupil diameter points can be used as the judgment. The points are accumulated, as long as the pupil diameter If it is less than 20, the statistics will be started, and if it is greater than 20, the statistics will be ended. When the accumulated points are more than 10, it can be judged as an intentional blink. In addition, we will add the weight 値 to calculate so that The result of conscious blinking is more accurate. The statistical score formula for conscious blinking weights is as follows: η wt = 2 = 1 Figure 8 is the comparison table of the weights of conscious blinking and the vertical line of the pupil. The method can be used as the function of mouse pressing when the user blinks consciously. Because the flexibility and rotation stability of each eyeball are not the same, the user's position correction action must be performed before use. For the adjustment of the distance of pupil position correction, we take a look at the screen (Sex, 5c> J, 13 This paper size is applicable to China National Standard (CNS) A4 specification (210X297 mm) (Please read the precautions on the back before filling This page) Order 499306 A7 B7 V. Description of the invention (12), C ^ x2 ^ y2), dhU, five points, as shown in Figures 9 A to 9 E, to obtain the pupil center point (Mcx, Mcy), (Μχΐ, ΜγΙ), (Mx2, My2), (Mx3, My3), (Mx4, My4) calibration points are compared with the initial setting / 7 = J, 4 and the sampling point is / 3 Fine-tuning can get a correct > 5 coefficient, and the relative is defined by the distance between them Position, then we will assign four quadrants to correct > 5 in each quadrant. When the system calculates the standard calibration point center (TSc; r, 5cyJ), it will automatically cut the four quadrants to calculate the relative position, and so on. We can use the four calibration points and the positioning points of the origin to do the positioning, and find out (β χη / (Please read the precautions on the back before filling this page) (H) 2 (Staff Consumer Cooperatives, Central Standards Bureau, Ministry of Economic Affairs Print / 7 = 1,2,3,4 and then implement (β χβ y J coefficient distance adjustment and correction action, so as to correspond to the coordinates of the pupil position and the actual plane, and find the center point Wcx of the eyeball, AfC 乂? Calculate > 5, analyze each correction point corresponding to the point on the screen, and the initial setting of the correction point corresponding to the screen is C ^ xnf% yn) n = lf2,3,4 Substitute into the aforementioned formula 14 This paper size applies the Chinese National Standard (CNS) A4 specification (210X297 mm) 499306 A7 B7 V. Description of the invention (13) Heart 1, 2, 3, 4 can get a set of correct / 3 Coefficient, where the bar is the proportional control experience index, and you, do) is the offset experience index It can be used to define the relative position by the distance between (5CX, 5cy) (Γ9 可 Λ77, and the screen coordinate system can be assigned to the four phenomena. When the system calculates the standard calibration point center (A / C; 〇Α / cy;), it will automatically The four images are cut to calculate the relative position, and when the position of the center of the eyeball is obtained, it is assumed that the image is replaced by that image (β χη, β yn). We can map to the screen coordinates < f5x, 5y for different positions of the pupil position of the eye calculated after correction; »From this segmented multi-level judgment method, we can accurately calculate the positive coordinates of pupil fixation, However, if the system calculates that the out-of-bounds condition has a probability of 10 frames per second, the user's wearing position may have a slight deviation at this time and continues to exceed the correction range, you can add a maximum out-of-range position offset of 64X, 8K. Calculate the new pupil position. AX = MX-MXS Ay- My ~ Mys is printed by the Consumers Cooperative of the Central Standards Bureau of the Ministry of Economic Affairs (please read the precautions on the back before filling this page). The points Scx and Scy will be automatically subtracted and corrected, that is, the coordinates calculated by the system relative to the screen are as follows. The formulas in each quadrant are as follows, and they are shown on Figure 11: The first quadrant 15 This paper scale applies to China National Standard (CNS) A4 specification (210X297 mm) 499306 A7 B7 Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs

Mv + Ay-Mt CS? &Amp; Erxin + '(乂 + 办-么 土 夺 — 圪 a2 Λ Μ. -Μ ^ αλ-β 2 (— -—) hd' — Κία ^ β ^ ~ -S] Λν2 ^ cv The second quadrant: βχ ^ Ο ^ χ ^ τΛχ- α3 The third quadrant WPJM, Ax-^ y ~ ^ + βγ ^ Μ μ "λ —So a5 two βχΑ Λ Mx3 — Μπα, Αν3 (ϋ: ^ y4 ^ cy Quadrant 4 (Please read the notes on the back before filling out this page) α4

M ^ Ay-M ^ α5 MxJrAx-Mc A 16 This paper size is applicable to Chinese National Standard (CNS) A4 specification (210X 297 mm) 499306 A7 B7 V. Description of the invention (is)

Sx = S ^ Pxl {^^ Ax-

My + Ay ~ Mc α7 M ^ + Ax-Mm ^ 1 = βχΛ H h—M〇y Printed (, &) by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs for the coordinates of the center of the screen. Please refer to Figures 12 to 14 Figure 12 shows the ideal calibration pattern, while Figures 13 and 14 show that when the first and third calibration points are not corrected, it means that they occur when gazing at these points during calibration. The error may be caused by not paying attention to the upper-left situation when calibrating this point, or it may be an individual error, but no matter how it is caused, operating the system with such a correction result will not obtain good results. From the changes of the four correction coefficients X1 ^: β χ3 χ: β y2 Γ β y4, we can see the ability of the user's operating system. If the 値 of the four correction coefficients is too large, the user will not obtain the operating pupil tracking system. Good results, please refer to Figures 15A to 15E, which are the changes of the four correction coefficients obtained from five different cases, when: Four correction coefficients: within 5: Excellent four correction coefficients. 5-10 : Good four correction coefficients: 10 ^ 15 · • may 17 This paper size is applicable to China National Standard (CNS) A4 specification (210X 297g t) (Read the precautions on the back before filling this page) Order # 499306 五 _ I______ Printed by the Consumer Standards Cooperative of the Central Standards Bureau of the Ministry of Economic Affairs A7 B7 Invention Description (16) Four correction factors: 15 or more

Id = C Pa / Pb-1) x 100%: the last time the correction coefficient was used: the last time the correction coefficient was used, we can get the improvement of four correction coefficients in five cases, as shown in Figure 16 It is shown that through the pupil tracking method with crosshair positioning function of the present invention, it is very convenient and effective whether it is to evaluate the ability of the user's operating system or to evaluate the degree of improvement of the user's operating ability. In summary, the present invention uses the cross hair in the pupil image to calculate and locate the distance between the cross hair center point and the pupil center point, and adds a weighting factor to the calculation to correctly calculate the coordinates of the pupil center point, and then uses The change in the number of pixels and the position of the low-gray cross-line pixels in each image obtained by photography can be used to convert the vertical and horizontal angles of the eye gaze direction, and the conscious blinking action, so that the actual detection can be achieved. The effect of eye movement and blinking, and the four correction coefficients of up, down, left, and right obtained in the five-point correction method can be used to determine the user's ability to operate the pupil tracking system, and can be calculated based on this The improvement ratio can be achieved, and the degree of improvement of the operation ability of the pupil chase system by the same user before and after can be compared. The disclosure of the embodiments described above is used to illustrate the present invention, and is not intended to limit the present invention. Therefore, any number of changes or equivalent components should still belong to the scope of the present invention. From the above detailed description, those skilled in the art can understand that the present invention can indeed achieve the aforementioned purpose, and it has already complied with the provisions of the Patent Law. He has proposed a patent 18 This paper size is applicable to the Chinese National Standard (CNS) A4 specification (210X 297 mm) ) (Please read the notes on the back before filling this page) 499306 A7 __B7 V. Description of the invention (17) Application 0 (Please read the notes on the back before filling out this page Printed by the Staff Consumer Cooperative of the Central Standards Bureau of the Ministry of Economic Affairs 9 1 This paper size applies to Chinese National Standard (CNS) A4 specification (210X297 mm) 499306 A7 ΒΊ · V. Printed invention description Us (used by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs) (Please read the precautions on the back before filling this page) Figure 1 shows the situation of the user-loaded chasing device. Figure 2 shows the hardware architecture of the optical pupil tracking system used in the present invention. Figures 3A ~ 3I are the top, bottom, left, The pupil image of the eyeball captured on the right, upper left, upper right, lower left, lower right, and facing forward. Figure 4 shows the pupil image captured by the present invention with a cross The diagram of finding the center coordinates of the perforation. Fig. 5 is a diagram of the algorithm for increasing the stability of the cursor. Fig. 6 A and B are diagrams of searching the pupil with a crosshair when the eyeball moves. 7 A ~ C are unconscious blinking and conscious. The pupil diameter data chart of blinking for one second and conscious blinking for two seconds. Figure 8 shows the weights of conscious blinking and the vertical line of the pupil. Table 9 A to E shows the data obtained during pupil correction. Top, bottom, left, right, and center pupil images. Figure 10 is the number of maximum outliers in each quadrant. Table 11 is a schematic diagram of pupil position mapping to screen coordinates. Figure 1 2 Schematic diagram of ideal correction results Figures 1 3, 1 and 4 are diagrams with correction deviations. Figures 5 A to E are changes in four correction coefficients in five different cases. Figure 16 is in four of five different cases. Improvement ratio of correction coefficient [Illustration of drawing number] (part of the present invention)

Capture the pupil image A Determine the pupil center B 20 This paper size applies the Chinese National Standard (CNS) A4 specification (210X297 mm) 499306 A7 B7 V. Description of the invention (IS) Tracking pupil position C Head with monitor 1 0 1 CCD camera 1 0 2 Conductor 1 0 3 Computer 1 〇4 (Please read the notes on the back before filling in this page. Ordered by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs. This paper is printed in accordance with the Chinese National Standard (CNS) A4 specification (210X297 mm).

Claims (1)

499306 A8 B8 C8 D8 ^, patent application scope 1. A pupil tracking method with crosshair positioning function, which mainly includes the following steps: Capture pupil image: use CCD camera to capture the eyeball image of the user, and obtain the second eyeball Dimensional linear image signals are analyzed and processed by image processing cards and computer programs. Determination of pupil center: X, γ coordinate binarization is used to classify eyeball images. When the element gray of a pixel is less than the set value, That is to identify the pupil position, and use a weighting factor when determining the pupil position to avoid misjudging eyebrows or shadows as pupils. Then select the pupil image and use the crosshairs in the pupil image to calculate and locate the crosshair center point and The distance between the pupil center points, and the weight method is used to gradually increase the number of gray levels in the pixel to the center of the calculated pupil center point coordinates to distinguish the position of the eyeballs and the pupils; the Consumer Cooperative of the Central Standards Bureau of the Ministry of Economic Affairs Printed (please read the precautions on the back before filling this page) Tracking pupil position: Analysis by CCD The continuous image of the user's eyeball taken by the camera uses the change in the number and position of pixels in the low gray scale of the crosshairs in each image to convert the vertical and horizontal angles of the eyeball gaze direction, as well as the conscious Blinking action, and then processing the information of eye movement in a real-time manner, making the information of eye movement on the computer screen as the track of cursor movement, and making the conscious blinking action be the action of the mouse pressing the option. 2. According to the pupil tracking method with cross-line positioning function described in item 1 of the scope of the patent application, in the step of measuring the center of the pupil, two tangent lines are formed on the image of the pupil circle with the cross-line of the two-dimensional coordinates, and The two-dimensional coordinates of the intersection of the crosshair with the pupil and the center point of the crosshair 22 This paper size applies to the Chinese National Standard (CNS) A4 specification (210X297 male " FT 499306 Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs A8 B8 C8 D8. The scope of the patent application is used to calculate the two-dimensional coordinate 垂直 of the vertical bisector of the two tangents, and the point where the two vertical bisectors meet is the center point of the pupil. 3. According to item 1 of the scope of patent application Tracing method with cross-line positioning function, in the step of tracking the position of the perforation, the direction of pupil movement is determined by the difference of the center coordinates of the pupil in each image, and the direction of pupil movement is determined by a formula Whether it is the same, to confirm the direction of conscious eyeball movement, and to avoid reacting to the physiologically unconscious eyeball movement. 4. According to the first scope of the patent application The chase tracking method with cross-line positioning function described in the above item, in the step of tracking the pupil position, a pupil correction action can be added, which focuses on the five points on the screen: up, down, left, right and center The image of the time is used to sample the pupil coordinates of the five pupil center points as the correction points. Then, the correction coefficients of the four correction points except the central correction point are obtained by formulas to perform correction analysis operations on each correction point. When the pupil center position is calculated, the screen coordinates are automatically cut into four quadrants. When the eyeball movement causes the pupil position coordinates to fall into different quadrants, the correction coefficient of the quadrant is substituted to correct the different positions of the pupil position after correction calculation. Map to the screen coordinates to accurately calculate the 眞 positive coordinates of 曈 hole fixation. 5. According to the pupil tracking method with cross line positioning function described in item 1 of the scope of patent application, in the step of tracking the position of the 曈 hole, The pupil diameter decreases to zero when the eye blinks, and the conscious blink time often reaches one or two seconds, which will cause more than ten strokes on continuous images. Pupil diameter data is zero, and when you blink unconsciously, the appropriate hole diameter (please read the precautions on the back before filling this page) C ·, 1T L · 23 This paper size applies to China National Standard (CNS) A4 specifications (210X297) ) 499306 A8 B8 C8 D8 6. The number of patent applications with zero data will be far less than ten, and the formula can be used to determine whether it is a conscious blink, and the blinking action is used as the action of the mole pressing the option. 6 · According to the pupil tracking method with cross-line positioning function described in item 4 of the scope of the patent application, the ability of the user to operate the tracking system can be judged by the number of four correction coefficients during the correction action, and four correction points The larger the number is, the worse the operation ability of the user is. 7 · According to the pupil tracking method with cross-line positioning function described in item 4 of the scope of the patent application, in the corrective action, an improvement ratio can be obtained by the formula using the number of four correction coefficients to compare the same use The degree of improvement of the operation ability of the pupil tracking system. (Please read the precautions on the back before filling this page) Order .1 # Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs This paper uses the Chinese National Standard (CNS) A4 size (210X297 mm)