TWI521384B - Electronic device with eyeball tracking function and control method thereof - Google Patents

Description

Electronic device with eye tracking function and control method thereof

The present invention relates to an electronic device having an eyeball tracking function and a control method thereof, and more particularly to an electronic device capable of determining whether to maintain a predetermined function by using an eyeball tracking function when a predetermined function is turned on, and a control method thereof.

In the past, there have been examples of using eye tracking technology to control computer operations, but most of them are special devices designed for paralyzed patients and do not have a large number of commercial applications. Driven by the dual factors of technology maturity and price decline, eye tracking technology has gradually expanded to consumer applications such as computers, mobile phones, automobiles and game consoles. The basic principle of the eye tracking technology is to use a low-power infrared beam to illuminate the user's eyeball, and then the sensor can extract the light reflected from different parts such as the pupil, the iris, and the cornea, and then analyze it through a complicated algorithm. The goal of eyeball gaze.

At present, the main application of eye tracking technology is to replace the hand to operate the input device, use the detection of eye movement to select, page, play music, etc. For example, the eye tracking device detects that the user's line of sight moves to the lower right corner of the screen. In the page area, it can be judged that the user has finished reading the information of the page, and then automatically completes the page turning operation, or the eyeball tracking technology allows the computer to judge the user's line of sight position, thereby adjusting the screen brightness and screen update. For the frequency, the detailed principle can be found in U.S. Patent Publication No. 2013/0135196.

As the development of eye tracking technology matures, manufacturers are actively using eye tracking technology to achieve a wide range of application functions to make their products more selling. Therefore, how to use eye tracking technology to increase the added value of products has become one of the important research topics of manufacturers.

An object of the present invention is to provide an electronic device capable of determining whether to maintain a predetermined function by using an eye tracking function when a predetermined function is turned on, and a control method thereof.

According to an embodiment, the electronic device with the eye tracking function of the present invention comprises a display unit, an eye tracking unit and a processing unit, wherein the processing unit is electrically connected to the display unit and the eye tracking unit. The eye tracking unit is used to detect the movement of an eyeball. When a predetermined function is turned on, the processing unit determines whether the eyeball continues to gaze at the display unit for a predetermined time according to the detection result of the eyeball tracking unit. If the eyeball continues to look at the display unit for a predetermined time, the processing unit maintains the predetermined function on.

In this embodiment, if the eyeball continues to look at the display unit for less than the predetermined time, the processing unit may turn off the predetermined function.

In this embodiment, the electronic device with the eye tracking function may further include an output unit electrically connected to the processing unit. If the eyeball continues to look at the display unit for less than the predetermined time, the processing unit can control the output unit to output a warning message.

In this embodiment, after the output unit outputs the warning message, the processing unit may determine, according to the detection result of the eyeball tracking unit, whether the eyeball continues to gaze at the display unit for a predetermined time. If the eyeball continues to look at the display unit for a predetermined time, the processing unit maintains the predetermined function on. If the eyeball continues to look at the display unit for less than the predetermined time, the processing unit turns off the predetermined function.

In this embodiment, when the predetermined function is turned on, the processing unit may display an advertisement window in the display unit. The processing unit determines whether the eyeball continues to gaze at the advertisement window for a predetermined time according to the detection result of the eyeball tracking unit.

According to another embodiment, the control method of the present invention is applicable to an electronic device having an eye tracking function, wherein the electronic device includes a display unit and an eye tracking unit. The control method comprises the following steps: opening a predetermined function; detecting the movement of the eyeball by the eyeball tracking unit; determining whether the eyeball continues to gaze at the display unit for a predetermined time according to the detection result of the eyeball tracking unit; and if the eyeball continues to gaze at the display unit The scheduled function is maintained for a predetermined time.

In this embodiment, the control method may further include the following steps: if the eyeball continues to gaze The display unit has not been scheduled for a predetermined time, and the predetermined function is turned off.

In this embodiment, the electronic device may further include an output unit, and the control method may further include the following steps: if the eyeball continues to look at the display unit for less than a predetermined time, the control output unit outputs a warning message.

In this embodiment, the control method may further include the following steps: after the output unit outputs the warning message, determine, according to the detection result of the eyeball tracking unit, whether the eyeball continues to gaze at the display unit for a predetermined time; if the eyeball continues to gaze at the display unit for a predetermined time The time, maintaining the predetermined function is maintained; and if the eyeball continues to look at the display unit for less than the predetermined time, the predetermined function is turned off.

In this embodiment, the opening step may display an advertisement window in the display unit, and the determining step may determine whether the eyeball continuously looks at the advertisement window for a predetermined time.

In summary, the present invention can determine whether to maintain the predetermined function by using the eye tracking function when the predetermined function is turned on. If the eyeball continues to look at the display unit for a predetermined time, the present invention maintains the predetermined function. If the eyeball continues to look at the display unit for less than the predetermined time, the present invention can directly turn off the predetermined function or control the output unit to output an alert message to remind the user to turn his gaze back to the display unit. After the output unit outputs the warning message, if the eyeball continues to look at the display unit for a predetermined time, the present invention maintains the predetermined function to be turned on. Conversely, if the eyeball continues to look at the display unit for less than the predetermined time, the present invention turns off the predetermined function. Thereby, the present invention can use the eyeball tracking function to determine whether to maintain the predetermined function or to turn off the predetermined function when the predetermined function is turned on, thereby expanding the application of the eye tracking technology. In addition, the present invention can display an advertisement window in the display unit when the predetermined function described above is turned on. Therefore, the present invention can determine whether the eyeball continues to look at the advertisement window displayed by the display unit for a predetermined time to decide whether to maintain the predetermined function or to turn off the predetermined function. In this way, the advertiser can cooperate with the service provider of the predetermined function, and use the present invention to determine whether the user has completely read the advertisement information provided by the advertiser when the user opens the predetermined function, thereby determining whether to allow the user to continue. Use the scheduled feature. Thereby, the present invention can effectively utilize the eyeball tracking technology to expand the advertising business model.

The advantages and spirit of the present invention can be obtained by the following detailed description of the invention and the accompanying drawings. Go to further understanding.

1‧‧‧Electronic device

3‧‧‧ eyeballs

10‧‧‧Display unit

12‧‧‧ Eye Tracking Unit

14‧‧‧Processing unit

16‧‧‧Output unit

18‧‧‧Advertising window

S10-S18, S30-S42‧‧‧ steps

1 is a schematic diagram of an electronic device having an eye tracking function according to an embodiment of the present invention.

Fig. 2 is a functional block diagram of the electronic device in Fig. 1.

Fig. 3 is a schematic diagram showing the display window of the display unit in Fig. 1.

4 is a flow chart of a control method suitable for an electronic device having an eye tracking function according to an embodiment of the present invention.

Fig. 5 is a flow chart showing a control method suitable for an electronic device having an eye tracking function according to another embodiment of the present invention.

Referring to FIG. 1 and FIG. 2, FIG. 1 is a schematic diagram of an electronic device 1 having an eyeball tracking function according to an embodiment of the present invention, and FIG. 2 is a functional block diagram of the electronic device 1 in FIG. As shown in FIG. 1 and FIG. 2, the electronic device 1 having an eye tracking function includes a display unit 10, an eye tracking unit 12, a processing unit 14, and an output unit 16, wherein the processing unit 14 is electrically connected to the display. The unit 10, the eyeball tracking unit 12 and the output unit 16.

In practical applications, the electronic device 1 can be a tablet computer, a desktop computer, a notebook computer, a mobile phone, a personal digital assistant, a smart television or other electronic device; the display unit 10 can be a liquid crystal display or other display; eye tracking Unit 12 can be an infrared device or a camera; processing unit 14 can be a processor or controller having a data processing function; output unit 16 can be a speaker, a lighting element, a vibration motor, or a combination thereof. Generally, the electronic device 1 is also provided with soft and hard components necessary for operation, such as a storage device, a memory, a power supply, an operating system, an application program, a communication module, etc., depending on the actual application.

As shown in FIG. 1, the eyeball tracking unit 12 is for detecting the movement of an eyeball 3. Eye tracking is mainly to collect the information of eye movements to determine the direction of movement of the user's line of sight, stop The technology to stay where. The hardware for eye tracking can be an infrared device or a general camera. No matter what kind of hardware is used, you need the corresponding software support to achieve this function. At present, the more common eye tracking technology on the market can be roughly divided into three types. The first one is based on the changes in the characteristics of the eyeball and the periphery of the eyeball, the second is based on the change of the iris, and the third is the active projection of infrared rays. Wait for the beam to reach the iris to capture the characteristics of eye movement. It should be noted that the eye tracking technology is well known to those skilled in the art and will not be described herein.

In this embodiment, when a predetermined function is turned on, the processing unit 14 determines whether the eyeball 3 continues to look at the display unit 10 for a predetermined time according to the detection result of the eyeball tracking unit 12 (for example, five seconds, ten seconds, one. Minutes, etc.). If the processing unit 14 determines that the eyeball 3 continues to gaze at the display unit 10 for a predetermined time based on the detection result of the eyeball tracking unit 12, the processing unit 14 maintains the on-demand predetermined function. If the processing unit 14 determines that the eyeball 3 continues to gaze at the display unit 10 for less than the predetermined time based on the detection result of the eyeball tracking unit 12, the processing unit can directly turn off the predetermined function.

In another embodiment, if the processing unit 14 determines that the eyeball 3 continues to gaze at the display unit 10 for less than the predetermined time according to the detection result of the eyeball tracking unit 12, the processing unit 14 may first control the output unit 16 to output a warning message to remind The user moves his gaze back to the display unit 10. For example, if the output unit 16 is a speaker, the warning message is a sound effect; if the output unit 16 is a light-emitting element, the warning message is light; if the output unit 16 is a vibration motor, the warning message is vibration; if the output unit 16 is The combination of a speaker, a light-emitting element and a vibration motor provides a combination of sound effects, light and vibration. In other words, the form of the warning message can be designed according to the actual application.

After the output unit 16 outputs the warning message, the processing unit 14 determines whether the eyeball 3 continues to look at the display unit 10 for a predetermined time according to the detection result of the eyeball tracking unit 12. If the processing unit 14 determines that the eyeball 3 continues to gaze at the display unit 10 for a predetermined time based on the detection result of the eyeball tracking unit 12, the processing unit 14 maintains the on-demand predetermined function. If the processing unit 14 determines that the eyeball 3 continues to gaze at the display unit 10 for less than the predetermined time based on the detection result of the eyeball tracking unit 12, the processing unit can directly turn off the predetermined function.

In other words, when a predetermined function is turned on, the user only keeps on looking at the display unit. If the predetermined time is up to 10, the predetermined function can be continued. Otherwise, if the user looks at the display unit 10 for less than the predetermined time, the predetermined function cannot be used. Thereby, the present invention can use the eyeball tracking function to determine whether to maintain the predetermined function or to turn off the predetermined function when the predetermined function is turned on, thereby expanding the application of the eye tracking technology.

In this embodiment, the opening predetermined function may be to turn on the electronic device 1, and the closing the predetermined function may be to shut down the electronic device 1. In other words, when the user turns on the electronic device 1, the user can continue to use the electronic device 1 only when the user continuously watches the display unit 10 for a predetermined time, and the user can continue to use the electronic device 1. When the display unit 10 is not looking at the predetermined time, the electronic device 1 is turned off, and the user cannot continue to use the electronic device 1.

In another embodiment, the opening the predetermined function may be to open an application, and the closing the predetermined function may be to close the application. In other words, when the user opens the application, the user can continue to use the application only after continuously watching the display unit 10 for a predetermined time. Otherwise, if the user looks at the display unit 10 for less than the predetermined time, the application will Will be closed and users will no longer be able to use this app.

Please refer to FIG. 3, which is a schematic diagram showing the display window 18 of the display unit 10 in FIG. When the predetermined function described above is turned on, the processing unit 14 can display an advertisement window 18 in the display unit 10, as shown in FIG. At this time, the processing unit 14 determines whether the eyeball 3 continues to look at the advertisement window 18 displayed by the display unit 10 for a predetermined time according to the detection result of the eyeball tracking unit 12, to determine whether to maintain the predetermined function or to turn off the predetermined function according to the above manner. In this way, the advertiser can cooperate with the service provider of the predetermined function, and use the present invention to determine whether the user has completely read the advertisement information provided by the advertiser when the user opens the predetermined function, thereby determining whether to allow the user to continue. Use the scheduled feature. Thereby, the present invention can effectively utilize the eyeball tracking technology to expand the advertising business model.

Referring to FIG. 4, FIG. 4 is a flow chart of a control method suitable for an electronic device having an eye tracking function according to an embodiment of the present invention. The control method in Fig. 4 can utilize the above The electronic device 1 is implemented. First, step S10 is executed to start the predetermined function. Next, step S12 is performed to detect the movement of the eyeball 3 with the eyeball tracking unit 12. Next, step S14 is executed to determine whether the eyeball 3 continues to look at the display unit 10 for a predetermined time based on the detection result of the eyeball tracking unit 12. If the eyeball 3 continues to look at the display unit 10 for a predetermined time, step S16 is performed to maintain the predetermined function on. If the eyeball 3 continues to look at the display unit 10 for less than the predetermined time, step S18 is performed to turn off the predetermined function. It should be noted that the detailed embodiments of the control method of the present invention are as described above, and are not described herein again.

Please refer to FIG. 5. FIG. 5 is a flow chart of a control method suitable for an electronic device having an eye tracking function according to another embodiment of the present invention. The control method in FIG. 5 can be realized by the above-described electronic device 1. First, step S30 is executed to start the predetermined function. Next, step S32 is performed to detect the movement of the eyeball 3 with the eyeball tracking unit 12. Then, step S34 is executed to determine whether the eyeball 3 continues to look at the display unit 10 for a predetermined time according to the detection result of the eyeball tracking unit 12. If the eyeball 3 continues to look at the display unit 10 for a predetermined time, step S36 is performed to maintain the predetermined function. If the eyeball 3 continues to look at the display unit 10 for less than the predetermined time, step S38 is executed, and the control output unit 16 outputs an alert message. After the output unit 16 outputs the warning message, step S40 is performed to determine whether the eyeball 3 continues to look at the display unit 10 for a predetermined time according to the detection result of the eyeball tracking unit 12. If the eyeball 3 continues to look at the display unit 10 for a predetermined time, step S36 is performed to maintain the predetermined function. If the eyeball 3 continues to look at the display unit 10 for less than the predetermined time, step S42 is performed to turn off the predetermined function. It should be noted that the detailed embodiments of the control method of the present invention are as described above, and are not described herein again.

It should be noted that the control logic of the control method shown in FIG. 4 and FIG. 5 can be implemented by software design. This software can be executed in any electronic device with eye tracking function. Of course, various parts or functions in the control logic can be realized by a combination of software, hardware or software and hardware.

In summary, the present invention can determine whether to maintain the predetermined function by using the eye tracking function when the predetermined function is turned on. If the eyeball continues to look at the display unit for a predetermined time, the present invention maintains the predetermined function. If the eyeball continues to look at the display unit for less than the predetermined time, the present invention can directly close The closed reservation function or the control output unit outputs a warning message to remind the user to turn his gaze back to the display unit. After the output unit outputs the warning message, if the eyeball continues to look at the display unit for a predetermined time, the present invention maintains the predetermined function to be turned on. Conversely, if the eyeball continues to look at the display unit for less than the predetermined time, the present invention turns off the predetermined function. Thereby, the present invention can use the eyeball tracking function to determine whether to maintain the predetermined function or to turn off the predetermined function when the predetermined function is turned on, thereby expanding the application of the eye tracking technology. In addition, the present invention can display an advertisement window in the display unit when the predetermined function described above is turned on. Therefore, the present invention can determine whether the eyeball continues to look at the advertisement window displayed by the display unit for a predetermined time to decide whether to maintain the predetermined function or to turn off the predetermined function. In this way, the advertiser can cooperate with the service provider of the predetermined function, and use the present invention to determine whether the user has completely read the advertisement information provided by the advertiser when the user opens the predetermined function, thereby determining whether to allow the user to continue. Use the scheduled feature. Thereby, the present invention can effectively utilize the eyeball tracking technology to expand the advertising business model.

The above are only the preferred embodiments of the present invention, and all changes and modifications made to the scope of the present invention should be within the scope of the present invention.

S30-S42‧‧‧Steps

Claims (8)

An electronic device with an eye tracking function includes: a display unit; an eye tracking unit for detecting movement of an eye; a processing unit electrically connected to the display unit and the eye tracking unit when a predetermined function When the device is turned on, the processing unit determines whether the eyeball continues to gaze at the display unit for a predetermined time according to the detection result of the eyeball tracking unit. If the eyeball continues to gaze at the display unit for the predetermined time, the processing unit maintains the predetermined function. And an output unit electrically connected to the processing unit; wherein if the eyeball continues to gaze at the display unit for less than the predetermined time, the processing unit controls the output unit to output a warning message, after the output unit outputs the warning message The processing unit determines, according to the detection result of the eyeball tracking unit, whether the eyeball continues to gaze at the display unit for the predetermined time. If the eyeball continues to gaze at the display unit for the predetermined time, the processing unit maintains the predetermined function. If the eyeball continues to look at the display unit for less than the predetermined time, The processing unit closes the predetermined function. An electronic device having an eye tracking function according to claim 1, wherein the opening of the predetermined function is to turn on the electronic device, and turning off the predetermined function is to shut down the electronic device. An electronic device having an eye tracking function as claimed in claim 1, wherein the opening the predetermined function is to open an application, and the closing the predetermined function is to close the application. The electronic device with the eyeball tracking function of claim 1, wherein the processing unit displays an advertisement window in the display unit when the predetermined function is turned on, and the processing unit determines the detection result according to the eyeball tracking unit. Whether the eyeball continues to look at the advertisement window for the predetermined time. A control method for an electronic device having an eye tracking function, the electronic device comprising a display unit, an output unit and an eye tracking unit, the control method comprising the following steps Activating a predetermined function; detecting the movement of an eyeball by the eyeball tracking unit; determining whether the eyeball continues to gaze at the display unit for a predetermined time according to the detection result of the eyeball tracking unit; if the eyeball continues to gaze at the display unit Maintaining the predetermined function for the predetermined time; if the eyeball continues to watch the display unit for less than the predetermined time, controlling the output unit to output a warning message; after the output unit outputs the warning message, according to the eye tracking unit again The detection result determines whether the eyeball continues to gaze at the display unit for the predetermined time; if the eyeball continues to gaze at the display unit for the predetermined time, the predetermined function is maintained; and if the eyeball continues to gaze at the display unit, the predetermined time is not reached Time, turn off the scheduled function. The control method of claim 5, wherein the opening the predetermined function is to turn on the electronic device, and turning off the predetermined function is to shut down the electronic device. The control method of claim 5, wherein the opening the predetermined function is to open an application, and the closing the predetermined function is to close the application. The control method of claim 5, wherein the opening step is to display an advertisement window in the display unit, and the determining step determines whether the eyeball continues to gaze at the advertisement window for the predetermined time.