TW201214197A - A method and system of using human body to control cursor - Google Patents

Abstract

This invention relates to a method and system of using human eyeballs to control cursor, including an acquisition step, in which an acquisition module is used to acquire the eye image of a user; a fixed point acquisition step, in which a fixed point acquisition module is used to acquire the judgment point of a cursor; a function selection step, in which a function selection module is used to judge the function that the user selected according to the eye image and the judgment point so as to trigger the cursor to move in a predetermined adjustable speed; and a cursor positioning step, in which a cursor positioning module stops the movement of the cursor after the cursor reached the position that the user designated. According to the method of using human eyeballs to control cursor disclosed in this invention, the need of repeated execution of complicated calibration procedures by the user, which will cause great inconvenience and produce discomfort, can be avoided so as to increase the user's willingness to use. This invention also provides a system of using human eyeballs to control cursor.

Description

201214197 VI. Description of the Invention: [Technical Field] The present invention relates to a method and system for manipulating a recreation, and more particularly to a method and system for eye manipulation of a cursor applied to human eye tracking.

[Prior Art] A small number of patients with major accidents, due to central nervous system damage or other factors, can not control the limbs or open mouth as smoothly as the average person. 'Even if the patient or the injured person has the ability to think and think, but it is difficult Full expression. Under such circumstances, the patient or the injured person and the caregiver will be unable to establish communication, which will cause many inconveniences, and it is impossible to respond to the medical staff by the patient itself. At present, the method of human eye tracking has been used in the prior art to detect the direction of human eye gaze through an eye tracker. By detecting the human eye's line of sight, the user's interest can be clearly displayed. Therefore, the eye tracker is also widely used in the human-machine interface to judge the disease of the eye. At the same time, it can also use 201214197 in human cognition. Research. If the eye tracker is used to provide atrophic sclerosing sclerosing disease (material) or severe physical dysfunction such as spinal injury, the user can use the eye to manipulate the mouse to enhance the life expectancy of these patients. As a bridge to communicate with the outside world.

The eye tracker records the range of limbs that rotate with the eye. To get the eye to see _ information, you must get the eyeball to rotate the eyepiece for the external environmental space position (four) should be. The more important deficiencies in the Wei field can be generated by the gaze position correction program (4) brace. This eye rotation angle and the gaze space position are used to calculate the data obtained after the calculation. In the 4th forward rotation, it will appear in the plane of the stimulus, such as multiple positions of the computer glory, - a small _. The subject gaze appears on the small dot appearing on the glory screen. At this time, the eye tracker records the data of the eyeball rotation, and then the small dot will be placed on the other side, and the subject _ 嶋 eyes stare at the appearing The small _ on the new position, the eye tracker once again recorded the 201214197 9 data of eye movement. - The general calibration of the experiment is distributed on the position of the glory to enter the miscellaneous, nine Chen Zhengzheng (9 (10) mystery, her eye tracker according to the angle of rotation of the eyeball corresponding to the nine gaze positions and this on the screen 9 points of the _ turn to generate a transfer formula, used to calculate the coordinates of the position of the eyeball should be corresponding to the position coordinates. Can also be based on different experimental needs to make a 3, 5 or 13 point calibration procedure. Correction gaze point The more accurate the position of the estimated gaze, but the longer it takes. In addition, the complex _ calibration process often requires constant execution, which causes great inconvenience in use. However, because people with disabilities usually do _ difficult, The function of eyeball rotation has been poor. It is also difficult to make a multi-point gaze correction procedure. It is too difficult for patients with diseases and people with disabilities. Even if the physiological condition is normal, if the eye is gazing more, Moreover, the user's head is required to be used in a state where it is completely incapable of being moved, and it is inevitable that there will be an uncomfortable feeling, which affects its willingness to use. Therefore, how to provide a human body manipulation The method and system of the cursor can be more convenient for the user in 201214197, avoiding the feeling of discomfort during the multi-point calibration procedure, thereby increasing the user's willingness to use, and has become an important subject [invention] In view of the above problems, the object of the present invention is to provide a method for man and ship mines and a secret of the "Wei thief transfer time (4) to continuously implement a complicated correction process, so that the great inconvenience and the feeling of discomfort arise. The user's willingness to use. In order to achieve the above purpose, according to the method of the invention of the invention - Miscellaneous #_, including the steps of the town: - image step, the image capture device - the user's body image; 4 points to obtain the steps Obtained by the fixed-point acquisition module - the m-energy step of the cursor, by the function selection, according to the human body image and the paste point, the function selected by the maker can be triggered to trigger the bead decoration; and - the cursor positioning According to the invention, the movement of the cursor is stopped by a cursor positioning module in the _ mark_time-time shape. 201214197 According to the present invention, it is difficult to implement the image, and the image captures the (four) towel. Eye image one person.

According to the present invention, Jiashiguan further includes an image analysis step, and the image of the eye is processed by the image analysis module to obtain a pupil data. According to a preferred embodiment of the present invention, the position of the button center area of the button is set in the beginning of the fixed point obtaining step, and _ is the basis for the user to manipulate the cursor.

According to a preferred embodiment of the invention, the movement of the user's eyes in the direction determining step. When it is judged that the eye of the eye according to the preferred embodiment of the present invention is blinked, the movement of the cursor is stopped during the cursor positioning step. According to a preferred embodiment of the present invention, a function execution step is performed, and when the user's eyes perform two blinking actions at 5 hrs, a 201214197 (four): purpose 'acquisition image according to the present invention' Bu Ding point to take ^ = like operation device 'for the use of a user's body shadow to pull, and 'connect the image she hit, secret to obtain the judgment point, 1 can choose the amount, the county slave point to pick up the group According to the human body image and the point, the user is selected to send the cursor movement; and the cursor positioning module is connected to the function and is used for positioning the cursor to the specified one. After that, stop the movement of the game. According to the present invention, the imaging device captures an eye image of the user. Therefore, according to a preferred embodiment of the present invention, the image analysis module is further included: the scene capture device 'hidden the glare image to obtain a boring hole according to the present invention. At the beginning of the system, the relying person should look at the position of the center of the front center as the basis for the user to control the 201214197 control. According to a preferred embodiment of the invention, the direction judges the movement of the user's eyes. According to the embodiment of the present invention, the miscellaneous calibration group stops the movement of the cursor when the user's eye is doing the eye movement. According to the present invention, the Wei-execution module is further connected to the cursor positioning module for performing a function when the user's eyes perform the action of the two sister eyes. # According to the above description, the method and system for the human mother's miscellaneous control cursor according to the present invention can determine the direction of the upper and lower τ from the image without correction, thereby avoiding the user's continuous execution of multiple points during the correction process. The gaze correction produces an uncomfortable feeling. Compared with the prior art, the application of the present invention to the eye control system not only simplifies the calibration procedure before the operation, but also the user's feelings, thereby increasing the willingness to use, and is a humanized design. 201214197 f embodiment, the following description of the preferred embodiment of the preferred embodiment of the present invention, the elements and components of the preferred embodiment of the present invention will be described with the same reference numerals. Referring to Figure 1 tf, the method of manipulating the cursor in a human body according to the present invention comprises the following steps S1 to (10). In step (10), a human body image of a user is taken. The eye image of the user will be described below by way of non-limiting example. The eye image is analyzed in step S110' to obtain a pupil data. The analysis method can be, for example, a preset database, which stores the pupil center, the eye contour, the eyeball awake, the black and white money or the reflective highlight, etc. After the comparison analysis, the desired pupil data is obtained. A decision point of a cursor is obtained in step S120'. The way to obtain the decision point can be, for example, when the system is started, that is, the user is required to first look at the position of the center of the screen to obtain the cursor determination point determined by the system as the basis for the user to manipulate the cursor. 201214197 In step (10), it is determined that the position where the user's eyes are gazing is at the top, bottom, left and right with respect to the determination point, and the Wei selected by the user is determined. For example, in the function selection page, the user is provided with four choices, please refer to phase 3, the four selection buttons are the selection buttons 31 to 34 shown in Fig. 3, and the selection button is for the user to select the left button and press two. Under the function, select button 32 for the user to select the function of the left button click, select _33 for the user to select the right click function, and select the function of the user to select the cursor crane option. If the user wants the lion to select the control vernier function of the 34, the user only needs to see the selection 34 at the bottom of the screen, and the system determines that the user's secret is below the threshold. The Wei selected by the user is the shifting label, so the trigger is recorded to move up and down and left and right. Select the selection method of New Zealand 31~33, just move your eyes to the corresponding selection button, no longer repeat them. In step S14G, the selection of the cursor 34 is controlled by the user, and the secret can provide another function selection screen page for the controller to select four directions. The function selection screen page can be referred to as shown in FIG. Set 201214197 a selection button for the user to select the direction of movement of the cursor. For example, when the user requests to move to the right, the eye gaze function only needs to select the rightmost button of the face, and the system determines that the position of the user's gaze is on the right side relative to the decision point, and determines the user's choice. Move the cursor to the right, so the cursor will be pure at a fixed speed and right. At this time, the Wei selection screen page shown in Fig. 4 is hidden, and the cursor can be positioned to the time specified by the time. ... In detail, the user can move the eye to move the position in the cursor-dimensional direction or the two-dimensional direction. The movement in the dimensional direction is, for example, only moving in the left-right direction or the vertical direction, and the two-dimensional direction is, for example, simultaneously and/or separately moving in the up-and-down direction, and reducing the time required for movement. When the user's eyes move, the direction of the cursor is determined based on the eye image and the decision point. The example of the movement mode that has been moved to the right is detailed above and will not be described again. In step S15G, after the positioning of the shirt is touched, the movement of the cursor is stopped. The way the cursor stops can be, for example, when the cursor reaches the desired position, the user's eye does the action of the sister's eye, and the system determines that the cursor has reached the fixed point and stops moving. The function selection screen on the 3 hemp can be used to provide the user with four selection buttons. 201214197 In this example, the way the cursor reaches the fixed point and wants to perform the fixed point function can select the eye/month view function at the top of the top of the eye. In addition to this, it can also be combined with other external aids. Wheeling interface, such as single-button switch, micro-switch, etc. It should be noted that the above summary is only for the purpose of illustration and is not intended to limit the invention.

The method of manipulating the cursor according to the eye of the present invention can avoid the need for the user to often perform a complicated calibration process, which causes great inconvenience and generates appropriate money, and the willingness to use it from a high level. v Gymnastics m standard method can be implemented in __ human body control cursor system. First, as shown in Figure 2, the human body manipulation cursor system 10 includes - image capture module 12, image analysis module Η, - fixed point acquisition mode , group 16, - power month b selection module 18 and - cursor positioning module 2 〇. Image manipulation module 12 is the operation - the user's body image. The following will be unrestricted The eye image of the user. In the example of this, the 'view> image capture module 21, for example, but not limited to, the electric county component 201214197 ((10)) financial machine, the reversal of the gold and the half of the film ((5) (6) camera or other achievable The same function is available. The number of the image capturing module 21 is not limited. In the aspectless aspect of the present invention, the human body manipulation cursor system 1G may include two image capturing modules 〖2, which respectively correspond to the use. The left and right pupils of the person, the shirt image _ group M and · Na module ^ miscellaneous, and the image analysis is used to analyze the eye image to take the lion m analysis method can be in the hole ^ ^ knife analysis module 14 Hunan, for example, preset The database, which stores the center of the eye: the eyeball, the eyeball, the black and white difference or the reflection After waiting for the analysis of the lyrics and the analysis of the group 14 and the analysis of the group 14, the required pupil module 16 is blocked and 16 is connected to the image analysis module 14, and the fixed point is obtained as the determination point of the cursor. The method of obtaining the judgment point can be exemplified by the method: the temple requires the user to first look at the position of the center of the screen, and the cursor determination point of the aunt (8), as the user controls the cursor according to the 201214197 function scale module 18 and the fixed point acquisition mode. The group 16 is connected, and the function selection module 18 can trigger the selected function according to the function of the riding image and the decision point, the function selected by the user. Please refer to FIG. 3 at the same time, for example, the user request. When the cursor is to the right axis, just move the eye and watch the cursor movement selection button. The system triggers the screen cursor for the user to move up, down, left, and right: Please also refer to Figure 4, the system can provide another function selection page. The user selects the four chicken directions of the cursor. The user then looks at the rightmost side of the screen to select the right side of the screen. The user's position of the user's gaze is relative to the right side of the _right, and the user selects to move to the right. The tour's domain will move the secret mark to the right at the established speed, and the cursor can be moved to the user's position. The established speed can be based on the speed of the pure setting or the speed selected by the user. In detail, the user can Move the eye and move the cursor-dimensional direction or the position in the two-dimensional direction. - The movement of the dimension is as follows: only the left-right direction or the downward direction is moved, and the two-dimensional direction is simultaneously and/or separately up and down left 201214197 right direction The movement, the reduction of the mobile energy selection module i8 'when the user's eyes move, the power I 卩 image and the decision to move can be moved _ cursor 1 break the financial function... On, no longer _. The case is detailed in

Module: The position (4) ship selects the axis connection, and the _ _ target reaches the position specified by the financial position, and stops the movement of the cursor. When the cursor is positioned, the cursor reaches the demand position, the bu, and the action, and the job determines that the cursor reaches the point and stops moving. In this implementation, it can also be combined with other external _ help input interfaces, such as single-button switches, micro-switches and other existing auxiliary devices. It should be noted that the above description is only for the sake of convenience. A method and system for a human gamma # cursor according to another preferred embodiment of the present invention will now be described with reference to the related drawings. Referring to FIG. $, a method for manipulating a cursor according to a preferred embodiment of the present invention includes the following steps S500 to S54G. In step % (8), the user's body image is captured. 201214197 The following will be a non-limiting example of capturing an eye image of the user. In step S510, the eye image is analyzed to obtain a pupil data. The analysis method may be, for example, a preset database, which stores information such as a pupil center, an eye contour, an eyeball pattern, a black-white difference change or an inverse silk point, and after the comparative analysis, obtains the required pupil data. In step S520, a decision point of the - cursor is obtained. The way to obtain the decision point can be, for example, when the system is started, that is, the user is required to first pay attention to the position of the center of the screen to obtain the cursor point of the marriage, as the basis for the user to manipulate the cursor. I in step S530, determining the movement of the user's eyes to move the cursor at a predetermined speed. For example, if the user asks the cursor to move to the right _, the eye will be measured by the eye _ axis, and the secret mark will continue to move to the right according to the speed of the slave. In detail, the user can move the eye to move in a one-dimensional direction or a one-dimensional direction. The movement in the one-dimensional direction is, for example, only in the direction of the right direction of the Lu or the direction of the downward direction of 201214197, and the two-dimensional direction is left at the same time and/or the axis of the upper and lower directions of the lion line is reduced by the time required to move. When the maker's eyes move,: According to 艮. The k image and the decision point determine the direction in which the cursor is moved. Move the move to the right and move the enemy down, 砰_. In step S540, after the cursor reaches the location specified by the user, the movement of the game is stopped. The way the cursor stops can be, for example, when the cursor reaches the desired position, the user's eye does the action of the sister's eye, and the system determines that the cursor has reached the fixed point and stops moving. In the present embodiment, the cursor reaches the fixed point and the function of the pointing function is to be performed, and the eye movement can be used, for example, blinking twice. In addition, it is also possible to use 辅助° with its auxiliary input interface, such as single-off, micro-close and other existing auxiliary equipment. It is to be understood that the above description is by way of example only, and is not intended to limit the invention. The method of manipulating the cursor according to the eye of the present invention can avoid the need for the user to continually perform complicated correction processes, which causes great inconvenience and production; I; comfortable money, and the desire to use is improved. 201214197 The above method for manipulating the cursor can be implemented in a human body controlled cursor system. Referring to FIG. 6, the human body manipulation cursor system 6 includes an image manipulation module 62, an image analysis module 64, a fixed point acquisition module 66, a direction determination module 68, and a cursor positioning module 70. The image capturing module 62 is configured to capture a human body image of a user. The following is an unrestricted example of the manipulation of the eye image of the user. In this case, it is not limited to an electric hybrid component (CCD) camera, a complementary metal oxide semiconductor (CMOS) camera, or the like that can achieve the same function. / Tian Ran, the number of image capturing modules % is also unlimited. In the eighth embodiment of the present invention, the human body manipulation cursor system 6G may include a two-dimensional image capturing module 62' which each corresponds to the left and right pupils of the user. "Image analysis turtle 64 is connected with Na Huan, and the image analysis kernel, 'and 64 is used to analyze the eye image to obtain pupil data. The analysis method can be, for example, a shirt like a dragon group, such as a preset database. It stores 瞳201214197 w~ eye contour, eyeball pattern, black and white difference change or reflective highlights, etc. After the image analysis model and group 64 comparison analysis, the required pupil data is obtained. & points get lion 66 and The image analysis plane 64 is connected, and the point acquisition module is used to obtain the decision point of the cursor. The manner of obtaining the determination point can be, for example, when the system is started, that is, the user is required to first look at the towel (four) domain position to obtain the system. · The cursor decision point is used as the basis for controlling the cursor.

The universal judging module 68 is connected to the fixed point obtaining module, and the direction determining module 68 is used to determine the movement of the user's eyes to move at a predetermined speed. For example, when the user requests the cursor to move to the right, only the eye decoration = the area on the right side of the point can be used, and the cursor is based on the speed network set by the system *OJ > tl 〇, the system can be cleared, and the cursor-dimensional Direction or two-way (four) axis. - The fiscal direction __ moves only in the left and right direction or the downward direction; for example, the two-dimensional direction moves at the same time and/or separately in the direction of the 201214197, reducing the time required for movement. When the user's eyes move, the square η is broken, and the ι 68 鳞 部 彡 彡 。 _ _ _ cursor. The example of moving the way to the right has been described in detail above, and will not be described again. "The W-bit module 70 is connected to the direction judging module, and the cursor positioning wheel is used to stop the movement of the cup after the cursor reaches the position specified by the user. The way of reading the cursor can be. When the target demand position takes the user's eyes to do the action of the second eye, the system determines that the cursor reaches the fixed point and stops moving. In this embodiment, the cursor reaches the fixed point and wants to perform the fixed point function: The action, for example, blinks twice. In addition, it can also be used with his external auxiliary input interface, such as single-button switch, micro-switch, etc. The existing auxiliary equipment. Cloud, Shi Li ^ and think The above description is only for the convenience of understanding, and is not intended to limit the present invention. The human-controlled cursor system 6 shown in the image of the human body is shown as an image capture module, and a 62-shirt analysis module 64, fixed point. The acquisition module (10), the direction determination module 68, and the cursor positioning module 70 are integrated into a single-electronic structure as an example. 201214197 However, those skilled in the art know that the human body manipulation cursor system 60 can also be used. personal For example, the image capture module, the image analysis module (10), the fixed point acquisition module 66 direction determination module 68, and the cursor positioning module can be implemented on both personal computers. The above example uses the pupil data as an example to determine how to perform the alignment. However, the person skilled in the art can also perform the alignment of the facial features. The method of manipulating the cursor according to this I month includes the following steps. : capturing a facial image; obtaining a decision point of the cursor; determining the cursor according to the facial image and the determination; moving the cursor at a predetermined speed by moving the face of the user; and reaching the side by the mechanical cursor Specifying the fixed money to stop the movement of the cursor. The method and system for controlling the cursor of the face, the step flow, the execution method or the component module are the same as those shown in Fig. 5 / Fig. 5 and Fig. 2 / Fig. 6, In particular, it should be noted that the present invention can achieve the purpose of manipulating the cursor by using an easy combination of the movement of the eyeball up and down, left and right, and blinking of the eye. The combination of the face up, down, left and right movement and the simple action of the face is 201214197. The invention can control the cursor using the combination of the hand up, down, left and right movement and the hand's simple action. The method and system of the cursor can avoid the need for the user to perform the complicated correction process in the past, causing great inconvenience and uncomfortable feeling in use, and improving the user's willingness to use. The above is only an example' Rather than being included in the scope of the patent application, the following is a description of the steps of the method for manipulating the cursor in accordance with a preferred embodiment of the present invention. 2 is a system block diagram of a system for manipulating a cursor in accordance with a preferred embodiment of the present invention. 3 is a functional selection screen of a method and system for manipulating a cursor in accordance with a preferred embodiment of the present invention. 4 is another functional selection of a method and system for manipulating a cursor in accordance with a preferred embodiment of the present invention. 5 is a flow chart showing the steps of a method for manipulating a cursor according to another preferred embodiment of the present invention; and FIG. 6 is a system block diagram of a system for manipulating a cursor according to another preferred embodiment of the present invention. 201214197 [Main component symbol description] ίο: Human body manipulation cursor system 12: Image capture module 14: Image analysis module 16: Fixed point acquisition module 18: Function selection module 20: Cursor positioning module 60: Human body manipulation cursor system 62: image capturing module 64: image analyzing module 66: fixed point obtaining module 68: direction determining module 70: cursor positioning module S100~S150: steps S500~S540: steps

Claims (1)

201214197 VII. Patent application scope: 1. - A method for manipulating the movement of the cursor, including: - image capture step 'by - image sensor device - user's body image; 疋 point acquisition step, by - fixed point acquisition module Acquiring - the decision point of the cursor; - a function selection step 'by - the function selection module determines the function selected by the user according to the body image and the judgment point", and triggers the movement of the cursor; and ... the pure step The cursor positioning game stops the movement of the touch mark after the positioning of the user to the positioning of the user. 2' The method for moving the human body to manipulate the cursor as described in claim 1 of the patent application scope, wherein the film takes the step money and takes the eye-cat image of the maker. ', i affirmed the method of moving the human body to manipulate the cursor as described in item 2, and it further includes _ 3~ image analysis and image analysis to process the image of the township to obtain a pupil data. f. The method of moving a human body to manipulate a cursor according to item 2 of the patent scope of the invention, wherein in the fixed point obtaining step, when the system is started, the user looks at the position of the center of the screen to act as a user to manipulate the cursor. in accordance with. 5. The method of claim 1, wherein the function selecting step determines the movement of the user's eyes. The method for moving a human body manipulated cursor according to the second aspect of the patent application, wherein in the cursor positioning step, the movement of the cursor is performed when the user's eyes perform a blinking motion. TJt /. A system for the movement of the human body to control the cursor - an image capturing device for capturing the image of the human body of (4); -g take _ group 'connected to the film to take the device, the rest is obtained - the cursor is a function to select the county group, the connection is connected to the spine, the test is determined by the person (4) 1 touch the user option, and the cursor movement can be triggered; The pin group 'connects the function selection module, and stops the movement of the cursor after the cursor reaches the position of the cursor. 8. For example, the system for manipulating the movement of the human body as described in claim 7 of the patent scope of the patent _ 撷 撷 撷 该 该 该 该 该 眼 眼 眼 眼 眼 眼 眼 眼 眼. 9, wherein, for example, the scope of the patent scope 8 includes an image analysis module to obtain the pupil data. The system for manipulating the movement of the cursor according to the item further comprises connecting the image capturing device for processing the image of the eye, and the movement of the human body controlled cursor according to the item 8 of the scope is repeated. Xuanzang point acquisition module in the system start, in the silk contribution / i money curtain center area 乍 to use the face control of the cursor according to ^ 10.