TWI275977B - Eye-controlled system of virtual keyboard - Google Patents

Abstract

The present invention is an eye-controlled system of virtual keyboard, which comprises: a projection section to project a virtual keyboard onto a plane, and distributing the first to fifth correction points along the two adjacent sides of the virtual keyboard; dividing the virtual keyboard into the first to the third keyboard regions from the perpendicular direction of the virtual keyboard in cooperation with the five correction points arranged in an L-shape; a camera section to input the operator's iris data into a control section when the operator is watching the virtual keyboard. Using the screen display of the control section in cooperation with the determination formula built in its internal program unit, computation is proceeded to control the operation of the virtual keyboard. Thus, it has plural advantages such as more accurate coordinate transformation, simple and accurate correction, simply linear computation, and suitably to be applied in a virtual keyboard which has a long-strip and slightly curved shape.

Description

1275977 IX. Description of the Invention: [Technical Field] The present invention relates to a virtual keyboard, and more particularly to a keyboard system, which has more accurate coordinate conversion, simple and simple linear operation, and is suitable for presenting strips. And a little rounded keyboard and other functions. [Prior Art] One of the control methods of the conventional virtual keyboard is as shown in the first figure. The projection unit projects a virtual keyboard on a flat surface, and has a photography, and has a photosensitive coupled device (Charge Coupled Device). Obtaining the eye of the virtual keyboard from an oblique angle of nearly 45 degrees; when the line of sight moves from left to right and looks at the three horizontal lines on the virtual keyboard and below, respectively, the calculated pupil center is as shown in the second figure. . Because the movement of the pupil center point in the image is an error in the arc-shaped coordinate conversion process, the user must first look at the nine hypothetical hypothetical points and press the Enter key one by one to perform the correction, but for some actions. Difficult people with disabilities (usually poor eyeballs) are too difficult. Further system correction and correction on the virtual keyboard is a cross correction point, as shown in the third figure, its seat on the virtual keyboard (X0, Y〇), (XhYd, (X2, Y2), (X3, Y3), (X4, Y4). In general, if automatic correction is used, the cross will have virtual, accurate and orphan-like eye control in sequence, which is the face of a ball (for example, a special CCD). The coordinates of the middle coordinates are very easy to be on the keyboard. The five functions of the function of the error are the (X0, Y〇), (XhYO, (Χ2, Υ2), which is on the virtual key 6 1275977 disk. (Χ3, Υ3), (Χ4, Υ4), etc. The subject must be constantly watching the cross and moving with it. After the system calculates the center position of the eyeball 20 times at each correction point, it will automatically jump to the next position. Correction point. If manual correction is selected, the user must look at the point when the cross appears at the coordinates of the calibration point, and then press the Enter key to gaze at five points in sequence to obtain five corrected reference point coordinates.

After the calibration step is completed, the five corrected reference point coordinates in the image (as shown in the fourth figure) are obtained, respectively (P〇, Q〇), (PhQi), (P2, Q2), (P3, Q3). ), (P4, Q4). Based on the distribution of the five corrected reference points, the evaluation parameter K can be set to analyze whether the user has the ability to operate the clamp, or whether the correction result is good. Because each person's head and eyes are different in size, the angle with the camera will also be different. These factors will affect the value of K. According to the experience, the value of K is mostly between 0 · 7 and 0.9. We set the threshold of K to 0.5. If the value of the evaluation parameter K is greater than 0.5, the result of the initial calibration is good, and we can continue to analyze the virtual keyboard gaze point of the user based on the five calibration reference points. If the value of the evaluation parameter K is less than 0.5, it means that the initial calibration failed or the user may not be able to operate the clamp. At this point, the system will return to the initial step. The user must re-adjust the search boundary, image binarization threshold, or adjust the angle with the camera, and then re-calibrate to continue using the system, which is quite troublesome. The following is the calculation method for the evaluation parameters: Assume the distance between the reference points (P〇, Q〇), (P^Qi), (P3, Q3), (P4, Q4) to (P 2, Q 2) The calculation method of 1275977 Κ= Κι κ2 κ i by the line segments ^GG, & respectively is: where pρ is the ideal value obtained by the experimental average, respectively 33, 55, 55, 37. The calculation method of Κ2 is:

〇» Ο » Ο » Ο I Ο I 〇Q ^>1 ^4 IV 丨_ - · ·丨丨· ......... · S0 Sx S2 S3 S4 where watts, X, watts, The tile and tile are the mean squared values obtained by analyzing the eyeball coordinates (the data obtained by 10 in one second) when the user looks at the five correction reference points of the virtual keyboard. The iτppP is the ideal value obtained by the experimental average, respectively: 3, 4, 2, 4, 3. Although the above method can take into account the comprehensive coordinate conversion, the calculation amount is large and complicated, and the error amount caused by the actual converted cursor coordinates in each quadrant is not equal (as shown in the fifth figure, the circle is assumed as the target). The coordinates, the fork is the actual coordinate, the error is very large, and it is not linear, and it cannot be compensated by adding a fixed offset. The integration of various systems produces the following deficiencies: [1] The amount of error in coordinate conversion is large. [2] Correction is troublesome, inconvenient, and poor in accuracy. [3] Complex and numerous linear operations. [4] The general coordinate conversion cannot be applied to a virtual keyboard that presents a long strip and is slightly rounded. The 12 1275977 arc-shaped virtual keyboard is prone to a larger amount of error. Therefore, it is necessary to develop new products to solve the above drawbacks. SUMMARY OF THE INVENTION A primary object of the present invention is to provide a system for an eye-controlled virtual keyboard that has the advantage of more accurate coordinate conversion. A secondary object of the present invention is to provide a system for an eye-operated virtual keyboard that has the advantage of simple and accurate correction. It is still another object of the present invention to provide an eye control virtual keyboard system that achieves a simple linear operation. It is still another object of the present invention to provide a system for an eye-operated virtual keyboard that is suitable for presenting a long and slightly arc-shaped virtual keyboard. The present invention provides a system for an eye-controlled virtual keyboard, comprising: a projection portion that illuminates a low-energy light beam toward a plane to form a virtual keyboard, and distributes a first correction point along two adjacent sides of the virtual keyboard; a second calibration point, a third correction point, a fourth correction point and a fifth correction point; the virtual keyboard is divided into a first keyboard area and a first keyboard area by matching the five correction points from a vertical direction of the virtual keyboard And a second keyboard area, a photographic unit that captures an image of the virtual keyboard by a predetermined oblique angle of the eye of the operator, and a control unit is provided with at least one screen and an internal program unit, and After accepting the information input by the photography department, the calculation is performed in accordance with the judgment formula built in the internal program unit, and then the virtual keyboard operation is controlled. The above objects and advantages of the present invention will be readily understood from the following detailed description of the embodiments and the accompanying drawings. 9 1275977 The following is a detailed description of the present invention with reference to the following embodiments: [Embodiment] The present invention is a system for an eye-controlled virtual keyboard, please refer to the sixth, which includes: ^ Projection section 10, system A projection element 1 1 radiates a low-energy beam 1 2 (for example, a laser beam) to form a pseudo-keyboard 1 3; a photographic unit 20, in a preferred embodiment of the present invention Is a photography with a Light Coupled Device (CCD) and captures an operator's eyeball 9 1 at a predetermined oblique angle 0 (in the preferred embodiment of the present invention, a degree angle). The image data of the pupil 9 2 (for example, the face of the hole 9 2 ); a control unit 30 is provided with at least one screen 3 1 and an internal program element 3 2 , and the control unit 30 receives the camera unit 20 The transmitted data is processed by the internal program unit 32 and can be displayed on the screen 31. Thus, the system of the eye-controlled virtual keyboard of the present invention. Referring to the sixth and seventh figures, since the eyeball 9 1 itself is a ball, the eyeball 9 1 (and the pupil 9 2 ) moves left and right to look at the hypotheses on the horizontal line of the virtual keyboard 13 in a state where the head does not move. At point 1 3 1 , the moving coordinate of the moving point 9 2 1 of the hole 9 2 of the eyeball 9 1 is sequentially recorded by the photographing unit 20 from a predetermined oblique angle 0 (for example, a 45-degree angle) (eg, Eight icons). Please refer to the eighth figure, which can be clearly displayed from the screen 3 1 , the plurality of pictures are moved to the virtual machine 4 5 瞳 single fluorescing body 10 1075977 moving point 9 2 1 formed by the movement, in the horizontal direction It is a line. It can be regarded as a straight line in the vertical direction (since it can be assumed to be a linear state, it will not be shown here, and it will be shown in the first place).

In terms of correction, the present invention adopts a five-point correction method in an L shape. Referring to the ninth diagram, it is assumed that a first correction point 1 2 2 and a second correction point 1 3 3 are distributed on the virtual keyboard 13. a third correction point 1 3 4, a fourth correction point 1 3 5 and a fifth correction point 1 3 6, the distance between the first correction point 1 3 2 and the second correction point 1 3 3 is set to Lys The distance between the second correction point 133 to the third correction point 134 is set to Lx i, and the distance between the third correction point 134 to the fourth correction point 135 is set to Lx2. The distance between the fourth correction point 1 3 5 and the fifth correction point 1 3 6 is set to Lx 3. As shown in the tenth figure, during the calibration process, the pupil 9 2 of the eyeball 9 1 looks at the first, second, third, fourth, and fifth correction points 1 3 2 to 1 3 6 respectively. The coordinates are (X2, Y2), (X〇, Y〇), (Xa, Ya), (Xb, Yb) and (Χι, Υι) 〇 the L-shaped five-point correction method of the present invention, just the rectangle The keyboard 13 is divided into three areas of left, middle and right (ie, the first keyboard area 137, the second keyboard area 138, and the third keyboard area 139), that is, corresponding to the first area 3 1 1 on the screen 3 1 The second area 3 12 and the first area 313. After the correction is completed, it is assumed that the pupil 9 2 of an operator's eyeball 9 1 looks at the virtual keyboard 13 (as shown in FIG. 11), and the resulting image falls on the coordinates of (x, y) ( As shown in the screen 31 of the twelfth figure, the control unit 30 automatically calculates the position coordinates (xs) corresponding to the virtual keyboard 13 by the following judgment formula built in the internal program unit 32. , ys): 11 1275977 1. When 戋 <1<弋, in the first area 311 (corresponding to the first keyboard area 137): less one less 0) + -^ yi -^0 χα -χ0

Xs = LXl (x-x〇) ^a~X0 2. When in the second region 312 (corresponding to the second keyboard region 138):

=-^(广^±ZL) + _-a)^^ y2~y〇 2 xb~xa

Xs= L%2 (x-Xa) + Lxx

Xb~Xa 3. Authentic<;^a, in the third area 313 (corresponding to the third keyboard area 139): Less 5 = Lys {yy\)^b Χχ 少 Less 2 Less 0 ~xb xs = Ζχ3—{χ - xb) + Lxx -\-Lx2 \~xb As shown in the thirteenth embodiment, in the preferred embodiment of the present invention, the image of the pupil 92 of the eyeball 91 may indicate that the virtual keyboard 13 is tapped. One of the first keyboard areas 1 3 7 is a predetermined key 1 3 0 (for example, "c ap s 1 〇 ck " key). More detailed description: the first correction point 1 3 2 of the creation, the second correction point 1 3 3, the third correction point 1 3 4, the fourth correction point 1 3 5 and the fifth correction point 1 3 6 substantially along the The two adjacent sides of the virtual keyboard 13 are set, and the five 12 1275977 correction points are taken as the standard, and the vertical direction of the virtual keyboard 13 is divided into three equal parts, regardless of the operator's line of sight. In one area, only the data of the area is calculated. The formula is simple, the operation is fast, and the key point is that the error is small. Therefore, the eye control key is quite accurate.

Of course, the photographing unit 20 can also be photographed upward from the pupil 9 2 of the operator's eyeball 9 1 , so that only the obstruction formed by the movement point 921 of the pupil 92 in the horizontal direction becomes the upper arc, but the pupil 9 2 The moving point 9 2 1 is in the vertical direction, and the first, second and third regions of the keying position are determined to be completely unchanged, and all the functional benefits achieved are completely intact. In addition, the camera unit 20 can be provided with two cameras, one of which captures the image data of the virtual keyboard 13 by the pupil 9 2 of the operator's eyeball 9 at a predetermined oblique angle, and the other captures the image with a predetermined oblique angle. The image data of the operator's finger button can be used by the operator to control the virtual keyboard 13 by using the eye movement and the finger button at the same time. The advantages and effects of the present invention are as follows: [1]. Coordinate conversion is more accurate. The five correction points of the present invention are generally arranged along two adjacent sides of the virtual keyboard, and the parallel vertical direction (for linear tracking, which does not affect the accuracy of calculation at all) divides the virtual keyboard into three equal parts, regardless of the operation. When you hit the button in that area with a line of sight, you only need to calculate the linear data of the area. The calculated point is the button hit by the line of sight, which is quite accurate. [2]· Correction ^ is accurate and accurate. Since the five correction points of the L-shaped arrangement of the present invention are arranged along two adjacent sides of the virtual keyboard, three regions of the arcuate arrangement are formed, which are very close to the actual distribution state. Therefore, the five most simplified correction points can be used to accurately correct the position of the virtual keyboard with the line of sight, and 13 1275977 completes the corrective action with a small error. [3]. Simple linear operation. The present invention linearizes the moving trajectory of the eyeball in the horizontal direction and divides it into three regions from the vertical direction, and only calculates the region where the line of sight is keyed at a time, and the amount of calculation is greatly reduced, and the calculation process is simplified. [4]. Applicable to a virtual keyboard that presents a long strip and is slightly arcuate. The five correction points of the present invention are generally arranged along two adjacent sides of the virtual keyboard, and four of the correction points are substantially evenly distributed on the longer sides, so that the strips are slightly elongated and slightly arcuate. When the virtual keyboard performs coordinate conversion, it is quite correct. The above is a detailed description of the present invention by way of a preferred embodiment, and the invention is not limited to the scope of the invention. From the above detailed description, those skilled in the art can understand that the present invention can achieve the foregoing objects, and the invention has been in compliance with the provisions of the patent law. 14 1275977 [Simple description of the diagram] The first diagram is a schematic diagram of a conventional device. In the second figure, when the operator uses the conventional device, the line of sight moves from left to right, and the three horizontal lines above, in the middle and below the virtual keyboard are respectively looked at to calculate the distribution of the central coordinate of the pupil. The third figure is a schematic diagram of the coordinates of five correction points set on the virtual keyboard by the conventional device. The fourth figure is a schematic diagram of five corrected reference point coordinates of the conventional device.

Fig. 5 is a schematic diagram showing the unequal amounts of errors caused by the coordinate conversion of the conventional device. Fig. 6 is a schematic view showing an embodiment of the present invention. The seventh figure is a schematic diagram of a plurality of hypothetical points on the virtual keyboard of the present invention. The eighth figure is a schematic view showing the movement of the movement point of the eyeball on the screen. The ninth figure is a schematic diagram of the first to fifth correction points set on two adjacent sides of the virtual keyboard of the present invention. The tenth figure is a schematic diagram of the division of the eyeballs of the present invention. The eleventh figure is the intention of the operator of the eyeball of the present invention to look at the virtual keyboard somewhere. The twelfth figure is a schematic view showing the image position of an eyeball gazing at a virtual keyboard on the firefly i of the present invention. The tenth figure is a schematic diagram of the actual application of the present invention in a key of an eye-operated tap virtual keyboard. , 15 1275977 [main component symbol description] 11 projection element 13 virtual keyboard 1 3 1 assumed point 1 3 3 second correction point 1 3 5 fourth correction point 137 first keyboard area 139 third keyboard area 30 control part 312 second The area 32 internal program unit 92 pupil 0 predetermined oblique angle Lys the distance between the first correction point and the second correction point Lxi the distance between the second correction point to the third correction point Lx 2 between the third correction point and the fourth correction point Distance 10 projection part 1 2 low power beam 1 3 0 predetermined key 1 3 2 first correction point 1 3 4 third correction point 1 3 6 fifth correction point 1 3 8 second keyboard area 20 photography unit 3 1 1 One area 3 1 3 third area 9 1 eyeball 9 2 1 moving point

Lx3 distance from the fourth correction point to the fifth correction point (X2, Y2) the coordinates of the first correction point (X 〇, Υ 〇) the coordinates of the second correction point (Xa, Ya), the third correction point Track coordinates (Xb, Yb) track coordinates of the fourth correction point (XbY) track coordinates of the fifth correction point (P〇, Q〇), (PhQO, (P2, Q2), (P3, Q3), ( P4, Q4) Five corrected reference point coordinates 16

Claims (1)

1275977 X. Patent application scope: 1 · A system for eye-controlled virtual keyboard, comprising: a projection portion, which radiates a low-energy light beam toward a plane to form a virtual keyboard, and distributes along two adjacent sides of the virtual keyboard a first calibration point, a second correction point, a third correction point, a fourth correction point, and a fifth correction point; and the virtual keyboard is divided into a first keyboard area by matching the five correction points from a vertical direction of the virtual keyboard a second keyboard area and a third keyboard area; A photographing department is configured to capture an image of a virtual keyboard by a pupil of an operator's eyeball, and a control unit is provided with at least one screen and an internal program unit, and receives information input by the photographing unit. Then, in conjunction with the internal program unit, the operation is performed in a broken manner, and then the virtual keyboard action is controlled. 2. The system of the eye-controlled virtual keyboard according to claim 1, wherein: the first area, the second area and the third area are divided by the screen, respectively corresponding to the first keyboard area, The second keyboard area and the third keyboard area, the pupils look at the first, second, third, fourth and fifth correction points respectively to generate the coordinates of the coordinates (X2, Y2), (X〇, Y〇) (Xa, Ya), (Xb, Yb) and (XhYj; the distance between the first correction point and the second correction point is set to Lys, and the distance between the second correction point and the third correction point is set to Lx, the distance from the third correction point to the fourth correction point is set to Lx2, the 17th 1275977 The distance between the four calibration points and the fifth correction point is set to Lx 3; thereby, when a pupil looks at the virtual keyboard and an image is generated at the coordinates, the control unit is automatically built in the internal program unit, Calculate the coordinate corresponding to the virtual keyboard (xs, ys): 1. When X.幺x<xfl, in the first area: yi~y〇^=-~-(χ-χο) χα-χο 2·When, in the i-th region: ys=^^(y-^,a + ( Ba)^^yi~y〇2 xb—xa xs = L—(x -χα) + Σχλ Xb~Xa 3. When %<yiyiyi, in the third area: less 5 = Lys {yy\) + bx^—x yi~y〇χι ~xb (x,y) at the following position Xs = Lx3 3 · As in the eye-operated virtual keyboard described in the scope of claim 1 * where 投射 the projection system a projection element emits a low-energy beam, the low-energy beam is a laser beam; system, 18 1275977, the camera unit is a camera with a photosensitive rotating and combining component; the camera unit can also be adjusted to another predetermined bevel angle, To capture the image data of an operator's finger button. 4. The system of the eye-operated virtual keyboard according to claim 1, wherein: the operator can control the virtual 'keyboard action' by using an eye movement/finger button. 5. The system of the eye-operated virtual keyboard according to claim 1, wherein the camera unit is two cameras, one of which takes the pupil of the operator's eyeball at a predetermined oblique angle and looks at the virtual keyboard. The image data is captured by the operator at the predetermined oblique angle, and the operator can control the virtual keyboard action by using the eye movement and the finger button at the same time. 19