JP6092173B2 - Electronic device and operation method thereof - Google Patents

Description

  The present invention relates to an electronic device and an operation method thereof, and more particularly to an electronic device incorporating an eye tracking technique and an operation method thereof.

  At present, eye tracking technology is mainly divided into two types, invasive and non-invasive. Invasive eye tracking techniques are mainly the placement of a search coil on the eye or the use of an electrooculogram. Non-invasive eye tracking technology can be divided into free head type or head mounted type. With the development of science and technology, eye tracking technology has been widely applied in various fields such as neuroscience, psychology, management engineering, ergonomics, marketing advertising, and computer science.

  Eye tracking technology provides communication for disabled people and disabled people, assists them with what they need, and provides greater convenience for people with motor disabilities. For example, for ambulatory patients with amyotrophic lateral sclerosis, muscular dystrophy, cerebral palsy, multiple sclerosis, cerebrovascular attacks, spinal cord injury, etc. Enables communication, internet and media entertainment activities using the eyes.

  However, the above-described two types of invasive and non-invasive eye tracking technologies require expensive hardware and software facilities, and further require a complicated calibration program, so the stability of the execution state is not good. Furthermore, the cost is considerably high, and the eye tracking technology cannot be spread.

In order to solve these problems, an operation method of an electronic device according to the present invention includes a step of displaying a graphical interface having a plurality of operation areas on a display unit; When one of the plurality of operation areas is highlighted, the step of canceling highlight display of the previous operation area and when receiving a trigger signal in the scan mode, the scanning process is performed. The group interface corresponding to the selected area is temporarily displayed on the display unit, and when the trigger signal is received, the selected area is set as the operation area highlighted display area , and the group interface is On top of the graphical interface Shown, both the size of the group interface and the graphical interface are the same, or the size of the group interface is smaller than the size of the graphical interface, the steps, the eye movement in the group interface of the user by the eye movement detecting module Detecting and selecting one function option from a plurality of function options of the group interface based on the eye movement and executing the operation , and the number of the plurality of operation areas in which the graphical interface is displayed is the group interface There greater than the number of said plurality of functional options that are displayed, comprising the steps, a.

  In one embodiment of the present invention, when the operation method receives a trigger signal in the scan mode, the operation method is performed by driving the eye movement detection module.

In one embodiment of the present invention, the step of detecting the eye movement at the user group interface by the eye movement detection module acquires an image sequence of the user by an image acquisition unit, and detects an eye movement by analyzing the image sequence. The method further includes causing the module to track the user's eye movement.

In one embodiment of the present invention, the step of tracking the user's eye movement by analyzing the image sequence illuminates the user's eyes with the position of the user's pupil and the light emitted by the light emitting unit in the image sequence. The step of detecting the position of the bright spot, which is the reflection point formed in this way, and when the horizontal distance between the pupil position and the bright spot position is within the error range, it is determined that the eye movement is directly looking forward If the horizontal distance between the pupil position and the bright spot position exceeds the error range, and the pupil position is to the left of the bright spot position, the eye movement is determined to move to the left. When the horizontal distance between the pupil position and the bright spot position exceeds the error range, and the pupil position is located to the right of the bright spot position, the eye movement is determined to move to the right. And based on the vertical distance between the pupil position and the bright spot position A step of eye movement to determine whether a downward movement or a movement upward, the position relationship of both the position of the bright spot in the pupil, and determining the gaze point of the current eye movement Further included.

  In one embodiment of the present invention, before the step of detecting the eye movement of the user by the eye movement detection module, the reference information of the eye movement detection module is obtained by guiding the eye movement to the designated position of the display unit to be viewed by the user. The method further includes the step of calibrating.

  In one embodiment of the present invention, the scan mode further includes detecting a user's eye movement by the eye movement detection module and determining whether to receive a trigger signal based on the eye movement.

  In one embodiment of the invention, the trigger signal is generated from an input unit.

  In one embodiment of the present invention, the number of operation areas in which the graphical interface is displayed is greater than the number of function options in which the group interface is displayed, and the group interface is displayed in an upper layer of the graphical interface. The size of both the group interface and the graphical interface is the same, or the size of the group interface is smaller than the size of the graphical interface.

The present invention further provides an electronic device. The electronic device is connected to the image acquisition unit disposed toward the user of the electronic device, a display unit that displays a graphical interface having a plurality of operation areas, and the display unit and the image acquisition unit. Processing unit to be executed. The plurality of modules include a scan module, an interface switching module, and an eye movement detection module. In the scan mode, the scan module highlights the plurality of operation areas one by one based on the scan order, and highlights the previous operation area when one of the plurality of operation areas is highlighted. Cancel the display. The interface switching module temporarily stops the scanning process when receiving a trigger signal in the scan mode, and displays a group interface corresponding to the selected area on the display unit, and when the selected area receives the trigger signal, This area is a highlight display area of multiple operation areas. The eye movement detection module drives the image acquisition unit to acquire the user's image sequence, detects the eye movement in the user's group interface by analyzing the image sequence, and based on the eye movement, Select a function option from the function options and execute the operation. In this electronic device, the number of the plurality of operation areas in which the graphical interface is displayed is larger than the number of the function options in which the group interface is displayed, and the group interface is displayed on an upper layer of the graphical interface. The size of both the group interface and the graphical interface is the same, or the size of the group interface is smaller than the size of the graphical interface .

  As described above, the present invention provides an electronic device and an operation method thereof, and can perform a simple and quick operation on a graphical interface based on an eye tracking technique.

  In order to further clarify the above-described features and advantages of the present invention, several embodiments will be described below in detail with reference to the drawings.

1 is a block diagram of an electronic device according to an embodiment of the present invention. FIG. 3 is a flowchart of a method for operating an electronic device according to an embodiment of the present invention. FIG. 3 is a flowchart of a method for operating an electronic device according to an embodiment of the present invention. FIG. 3 is a schematic diagram of a partial process for highlighting operation areas of a graphical interface step by step based on a scan order according to an embodiment of the present invention. FIG. 3 is a schematic diagram of a partial process for highlighting operation areas of a graphical interface step by step based on a scan order according to an embodiment of the present invention. FIG. 3 is a schematic diagram of a partial process for highlighting operation areas of a graphical interface step by step based on a scan order according to an embodiment of the present invention. FIG. 3 is a schematic diagram of a partial process for highlighting operation areas of a graphical interface step by step based on a scan order according to an embodiment of the present invention. FIG. 3 is a schematic diagram of a group interface according to an embodiment of the present invention. FIG. 3 is a schematic diagram of a group interface according to an embodiment of the present invention. FIG. 3 is a schematic diagram of a group interface according to an embodiment of the present invention. FIG. 6 is a block diagram of another electronic device according to an embodiment of the present invention. It is the schematic of the positional correspondence of both the position of a pupil and the position of a bright spot by one Example of this invention. It is the schematic of the positional correspondence of both the position of a pupil and the position of a bright spot by one Example of this invention. It is the schematic of the distance information and angle information of both the center of a pupil, and a reflective point by one Example of this invention. It is the schematic of the distance information and angle information of both the center of a pupil, and a reflective point by one Example of this invention.

  FIG. 1 is a block diagram of an electronic device according to an embodiment of the present invention. Referring to FIG. 1, the electronic device 100 includes an image acquisition unit 110, a display unit 120, a processing unit 130, and a storage unit 140. The image acquisition unit 110 is arranged toward the user of the electronic device 100 to acquire the user's image sequence. For example, the image acquisition unit 110 may be installed above the display unit 120, and the image acquisition unit 110 may be built in the display unit 120 or externally attached to the display unit 120. The image acquisition unit 110 is, for example, a camera / camera having a CCD lens, a CMOS lens, or an infrared lens.

  The display unit 120 displays a graphical interface, and the user views the screen of the display unit 120 and executes an operation. The display unit 120 may be any type of display device, such as a liquid crystal monitor, an LED monitor, or a flexible monitor. In the present invention, the graphical interface has a plurality of operation areas, and each operation area is a set of a plurality of function options, and includes a group interface corresponding to each operation area.

  The processing unit 130 is a device such as a CPU, an image processing unit (GPU), or other programmable microprocessor, and is connected to the image acquisition unit 110, the display unit 120, and the storage unit 140.

  The storage unit 140 is, for example, any fixed or removable RAM, ROM, flash memory, hard disk, other similar device, or a combination of these devices. In the present embodiment, the storage unit 140 stores a plurality of modules executed by the processing unit 130. These modules include a scan module 141, an interface switching module 143, and an eye movement detection module 145. In still another embodiment, the scan module 141, the interface switching module 143, and the eye movement detection module 145 may be a chip in which one or a plurality of digital logic gates are combined.

  The storage unit 140 also stores graphical interface data and a plurality of group interface data. The data of the graphical interface defines the size of the graphical interface, the sizes of the plurality of operation areas to be displayed, and the group interface data to which each operation area corresponds. That is, one operation area corresponds to data of one group interface. If N operation areas are set, the setting is made corresponding to the data of N group interfaces. The data of each group interface defines the size of the corresponding group interface, the size of the function option to be displayed, and the function corresponding to each function option.

  In the scan mode, the scan module 141 highlights a plurality of operation areas of the graphical interface one by one at every preset time based on the scan order, and highlights one of the operation areas before that. Cancel highlighting in the operation area. The scanning order is, for example, from left to right and then from top to bottom. However, the present invention is not limited to this. When the electronic apparatus 100 receives the trigger signal in the scan mode, the interface switching module 143 displays the group interface corresponding to the selected area on the display unit 120 as the selected area of the operation area. The eye movement detection module 145 drives the image acquisition unit 110 to acquire the image sequence of the user, and selects one of a plurality of function options of the group interface based on the eye movement and executes an operation.

  Hereinafter, each step of the operation method will be described together with the electronic device 100. 2A and 2B are flowcharts illustrating a method of operating an electronic device according to an embodiment of the present invention. 1 and 2A, first, in step S205, the display unit 130 displays a graphical interface on the display unit 120, and the graphical interface has a plurality of operation areas. For example, the processing unit 130 reads the graphical interface data from the storage unit 140 and displays the graphical interface based on the definition of the graphical interface data. In the present invention, the graphical interface occupies all the screens of the display unit 120, for example, and the screen is operated by 12, 24, 25, etc. divided by 4 × 3, 6 × 4, 5 × 5, etc. It becomes an area. However, the plurality of operation areas are merely examples, and the present invention is not limited to this.

  After displaying the graphical interface, the processing unit 130 activates the scan module 141, and as shown in step S210, the scan module 141 highlights the operation areas of the graphical interface one by one in the scan mode based on the scan order. To do. For example, the target operation area is displayed in a different color from the other operation areas, or the target operation area is displayed with a three-dimensional effect, a shadow effect, or a floating effect. Further, when the scan module 141 highlights one operation area, the scan module 141 cancels the highlight display of the previous operation area.

  For example, FIGS. 3A-3D are schematic views of a partial process for highlighting a plurality of operating areas of a graphical interface step by step based on a scan order according to one embodiment of the present invention. In this embodiment, the graphical interface 300 has 24 operation areas A1 to A6, B1 to B6, C1 to C6, D1 to D6, and the scanning order is from left to right and then from top to bottom. First, as shown in FIG. 3A, the scan module 141 highlights the operation area A1. In the present invention, the highlight display displays the outer frame of the target operation area with a thick black frame. However, in other embodiments, the target operation area may be displayed in 3D, and the present invention is not limited to this.

  After the preset time has elapsed, as shown in FIG. 3B, the scan module 141 highlights the operation area A2 on the right side of the operation area A1, and cancels the highlight display of the operation area A1. After the next preset time has elapsed, as shown in FIG. 3C, the scan module 141 highlights the operation area A3 and cancels the highlight display of the operation area A2. After the next preset time has elapsed, as shown in FIG. 3D, the scan module 141 highlights the operation area A4 and cancels the highlight display of the operation area A3. By analogy with this, after the operation areas A1 to A6 are highlighted one by one, the scan module 141 highlights the first row of operation areas B1 to B6 one by one, and then the further lower row of operation areas C1. ... To C6 are highlighted one by one, and finally, the operation areas D1 to D6 in the bottom row are highlighted one by one. The process is a display method of the graphical interface 300 in the scan mode. The preset time is 0.5 seconds or 1 second, for example, and the user can set the length of the preset time by himself.

  Referring back to FIG. 2A, in step S215, when the processing unit 130 receives a trigger signal in the scan mode, the interface switching module 143 displays the group interface corresponding to the selected area on the display unit 120. The selected area indicates a highlighted display area of the plurality of operation areas when a trigger signal is received. That is, each operation area has a corresponding group interface, and each group interface has a plurality of function options.

  Specifically, referring to FIG. 2B, step S215 is subdivided into steps S231 to S233, and in step S231, the processing unit 130 determines whether to receive a trigger signal in the scan mode. In the present invention, the trigger signal is generated by an input unit such as a mouse, a keyboard or a button. Alternatively, the eye movement detection module 145 detects the eye movement, and further determines whether a trigger signal is received based on the eye movement. When the trigger signal is not received, the scan mode is maintained, each operation area is highlighted one by one, and it is continuously detected whether the trigger signal is received.

  When receiving the trigger signal, as in step S232, when the interface switching module 143 receives the trigger signal, the highlight display operation area is set as the selection area. Subsequently, in step S233, the interface switching module 143 displays a group interface corresponding to the selected area.

  The operation area A4 in FIG. 3D is taken as an example of selection. That is, when a trigger signal is received when the operation area A4 is highlighted in the scan mode, the operation area A4 is selected as the selection area. The group interface corresponding to the area A4 is displayed on the display unit 120. In the present invention, the number of operation areas displayed on the graphical interface is greater than the number of function options displayed on the group interface. In addition, the group interface is displayed above the graphical interface. In addition, the size of both the group interface and the graphical interface may be the same, or the size of the group interface is smaller than the size of the graphical interface.

  For example, the interface switching module 143 reads the data of the group interface corresponding to the operation area A4 from the storage unit 140, acquires the size of the group interface, the size of the function option to be displayed, and the function corresponding to each function option. Display in the group interface. The display of the group interface will be described below using three examples. However, the present invention is not limited to this.

  4A to 4C are schematic views of a group interface according to an embodiment of the present invention. The group interface 410 of FIG. 4A is expanded vertically and is located in the upper layer of the graphical interface 300 and its size is smaller than the graphical interface 300. The group interface 410 has function options F1 to F3. The group interface 420 of FIG. 4B is expanded horizontally and is located in the upper layer of the graphical interface 300. The group interface 420 has function options F1 to F3, and the size thereof is smaller than the graphical interface 300. When displayed as group interfaces 410 and 420 in FIGS. 4A and 4B, the user can see an unobstructed operating area of the bottom graphical interface 300. FIG. In the present invention, the function option F1 to the function option F3 are integrated into one group and correspond to the operation area A4. When the operation area A4 is selected in the scan mode, the function options F1 to F3 of the corresponding group are displayed.

  The group interface 430 of FIG. 4C is expanded in a full-screen manner, and its size and the size of the graphical interface 300 are substantially the same (in other embodiments, it may be completely the same). Located in the upper layer. Since the group interface 430 is expanded in a full-screen manner, the number of function options that can be displayed is larger than the number indicated by the group interfaces 410 and 420 in FIGS. 4A and 4B. Here, the group interface 430 has function options F1 to F9.

  Referring back to FIG. 2A, after opening the group interface, the eye movement of the user is detected by the eye movement detection module 145 in step S220. Specifically, when the trigger signal is received, the eye movement detection module 145 is driven to execute the operation. That is, the user's image sequence is acquired by the image acquisition unit 110, and the eye movement detection module 145 is made to track the user's eye movement by analyzing the image sequence. For example, referring to steps S241 and S242 of FIG. 2B, in step S241, the eye movement detection module 145 detects the position of the user's pupil and the position of the bright spot in the image sequence. The position of the bright spot is a reflection point formed by the light emitted from the light emitting unit illuminating the eyes of the user. Subsequently, in step S242, the eye movement detection module 145 determines the gazing point of the current eye movement based on the positional correspondence between the position of the pupil and the position of the bright spot, and the function option that the user wants to select from the gazing point Determine what is. However, the determination of the gazing point is one execution method, and the gazing point may be acquired by determining the displacement of the pupil based on the ratio between the white eye and the eyeball, and the present invention is not limited to this.

  Finally, in step S225, based on the eye movement, the processing unit 130 selects one of a plurality of function options of the group interface and executes an operation. For example, taking the group interface 410 of FIG. 4A as an example, when the eye movement detection module 145 detects that the user's gaze point is located at the function option F1, the processing unit 130 executes the function corresponding to the function option F1.

  The execution time of the eye movement detection module 145 may be the time when the trigger signal is received in the scan mode, and the processing unit 130 drives the eye movement detection module 145 to execute the operation. That is, in the scan mode, the eye movement detection module 145 does not detect eye movement. The trigger signal may be generated by an input unit such as a mouse, a keyboard, or a button, and the processing unit 130 may determine whether the trigger signal is received from the input unit by the embedded control unit.

  For example, FIG. 5 is a block diagram of another electronic device according to one embodiment of the present invention. In the present embodiment, the functions corresponding to the electronic device 100 of FIG. 1 described above are indicated by the corresponding symbols, and the related description is omitted. The electronic device 500 includes an embedded control unit 510 and an input unit 520.

  The embedded control unit 510 is connected to the processing unit 130 and the input unit 520. The processing unit 130 determines whether the embedded control unit 510 receives a trigger signal from the input unit 520 and further transmits a message to the scan module 141 and the interface switching module 143 when the trigger signal is received. The scan process is temporarily stopped, and the interface switching module 143 displays the group interface corresponding to the selected area on the display unit 120.

  The processing unit 130 also drives the eye movement detection module 145 in the scan mode to detect the user's eye movement, and determines whether to receive a trigger signal based on the detection result. Further, when the processing unit 130 determines that the trigger signal has been received based on the eye movement, the processing unit 130 transmits a message to the scan module 141 and the interface switching module 143, causing the scan module 141 to temporarily stop the scanning process, and the interface switching module. In step S143, the group interface corresponding to the selected area is displayed on the display unit 120. For example, the processing unit 130 determines that the trigger signal has been received when the eye movement detected by the eye movement detection module 145 is wink. However, the present invention is not limited to this. The trigger signal in the scan mode may be generated from the input unit 520, may be generated based on the user's eye movement, or may be generated by voice control.

  An implementation method for detecting eye movement will be described below again using an example.

  Here, the user's image sequence is acquired by the image acquisition unit 110, and the eye movement detection module 145 is made to track the user's eye movement by analyzing the image sequence. For example, the eye movement detection module 145 detects the position of the user's pupil and the position of the bright spot in the image sequence, and further, based on the positional correspondence between the position of the pupil and the position of the bright spot, the current eye movement gaze point Judging. The position of the bright spot is a reflection point formed by the light emitted from the light emitting unit illuminating the eyes of the user.

  Specifically, the image acquisition unit 110 captures an image of the eye, and the image acquisition unit 110 has a built-in light-emitting unit that emits light to the user's eyes and places a reflection point in the captured image. Cause it to occur. As a result, the eye movement detection module 145 determines the gazing point by simply using the positional correspondence between the position of the bright spot and the position of the pupil, which are reflection points formed by the light illuminating the user's eyes. .

  For example, when the user's eyeball is looking directly forward, the pupil position and the bright spot position are almost on the vertical line in the eye image, that is, the horizontal distance between the pupil position and the bright spot position is within the error range. Is in. Therefore, when the horizontal distance between the pupil position and the bright spot position exceeds the above error range, and the pupil position is located to the left of the bright spot position, the user's eyeball is moved to the left. Show. In addition, when the horizontal distance between the pupil position and the bright spot position exceeds the above error range, and the pupil position is located to the right of the bright spot position, the user's eyeball moves to the right. Show. Further, it is determined whether the eyeball moves upward or downward based on the vertical distance between the pupil position and the bright spot position.

  For example, FIGS. 6A and 6B are schematic views of the positional correspondence between both the pupil position and the bright spot position according to an embodiment of the present invention. The group interface shown in FIGS. 6A and 6B of this embodiment is an example of full screen display. However, when the size of the group interface is smaller than the full screen, the following implementation method is applied. FIG. 6A shows four function options in which the screen is divided in a 2 × 2 system, and FIG. 6B shows nine function options in which the screen is divided in a 3 × 3 system. The position of the pupil and the position of the bright spot of each function option in this embodiment are only for the sake of simplification of description, and do not represent displaying the above-mentioned positions on the display screen.

  As shown in FIGS. 6A and 6B, when the eyeball moves to the left, the position of the pupil is located to the left of the position of the bright spot. When the eyeball moves to the right, the position of the pupil becomes the position of the bright spot. Located to the right of In addition, as the eyeball moves upward, the vertical distance between the pupil position and the bright spot position increases. Conversely, as the eyeball moves downward, the vertical distance between the pupil position and the bright spot position decreases. According to the above definition, after searching for the position of the pupil and the position of the bright spot from the image of the user's eyes, it is possible to know in which operation area the user's gaze point is.

  In the present embodiment, the plurality of modules executed by the processing unit 130 further includes an induction calibration module 530. Before the eye movement detection module 145 detects the eye movement of the user, the guide calibration module 530 calibrates the reference information of the eye movement detection module 145 by guiding it to the designated position of the display unit 120 that the user sees. For example, the guidance calibration module 530 obtains the position of the bright spot and the position of the pupil corresponding to the center position of the display unit 120 as reference information for subsequent comparison by guiding the guide calibration module 530 to the center position of the display unit 120 viewed by the user. .

  In addition to the above-described method, the eye movement detection module 145 may convert the position of the pupil and the position of the bright spot into coordinates corresponding to the display unit 120, and thereby, the gaze point can be more accurately identified. Specifically, each time the display unit 120 displays one operation area to show to the user, calculates the reference distance and reference angle of both the pupil center and the bright spot, and then displays another operation area. To the user and finish displaying the planned number of operation areas. Thereby, the relationship between each operation area, the reference distance, and the reference angle can be acquired.

  The definition of the reference distance and reference angle of both the pupil position and the reflection point is as follows. 7A and 7B are schematic diagrams of distance information and angle information of both the center of the pupil and the reflection point according to an embodiment of the present invention. FIG. 7A is an eye image, and FIG. 7B is an enlarged view of range 700 of FIG. 7A. In FIG. 7B, the y-axis (vertical axis) and the x-axis (horizontal axis) are defined by the pupil center C1, and the reference distance L on the y-axis between both the pupil center C1 and the reflection point C2 is calculated. A reference angle θ between the vector V of the reflection point C2 relative to the center C1 and the y axis is calculated.

  The user obtains the reference distance L and the reference angle θ of both the center of the plurality of pupils and the reflection point by looking at each operation area in advance, and further, the reference distance L and the reference angle θ and the plurality of display units 120. The operation area forms a normal distribution map. Thereafter, when the eye movement detection module 145 detects the eye movement of the user, after obtaining the reference distance L and the reference angle θ of both the pupil position and the reflection point, the corresponding normal distribution map has one coordinate point. And the coordinate points are converted into position coordinates corresponding to the display unit 120 by interpolation. As a result, it is possible to obtain the gaze point of the gaze person more accurately.

  As described above, in the above-described embodiment, the scan mode is first activated and the user is made to make the first selection (first stage). After the user selects one operation area (for example, one icon), the eye movement detection mode is executed (second stage), and the function options attached to the operation area selected in the first stage are displayed (for example, Icon display). Also, the eye movement detection module is activated to provide the user with a finer selection by eye movement. Since the first stage is the scan mode, a relatively large number of operation areas can be displayed (for example, 16 or more, or 24 or more). In order to improve the accuracy of eye movement detection, the number of function options displayed in the second stage is reduced. For example, only 3 to 9 function options are displayed, and the screen appears every time through the above two stages. Items to be selected by the user can be reduced (operation area and function options). In addition, since the number of items provided in the eye movement detection mode is reduced, the calibration program is simplified and only one point of calibration is required, improving the complicated calibration program required by conventional eye tracking technology welfare equipment. did. Further, it is not necessary to attach an additional device to the body, and the user does not need to pay an expensive cost, and the accurate effect of the position setting function can be obtained.

  Although the present invention has been shown as in the above embodiments, the present invention is not limited to this and can be changed or modified by those skilled in the art without departing from the spirit of the present invention. Therefore, the protection scope of the present invention extends to an equivalent range.

  The present invention is a combination of scanning technology and eye tracking technology. In the scanning mode, a large range of groups is selected, and eyeball movement is used to select functional options. By providing an excellent balance between scanning and eye-tracking technology in terms of technology and cost, eye-tracking technology does not require expensive hardware and allows simple and fast operation of the graphical interface. .

DESCRIPTION OF SYMBOLS 100: Electronic device 110: Image acquisition unit 120: Display unit 130: Processing unit 140: Storage unit 141: Scan module 143: Interface switching module 145: Eye movement detection module 300: Graphical interface 410: Group interface 420: Group interface 430: Group interface 510: Embedded control unit 520: Input unit 530: Inductive calibration module 700: Range

Claims (14)

Displaying a graphical interface having a plurality of operation areas on the display unit;
In the scan mode, a step of highlighting the plurality of operation areas one by one based on a scan order and canceling highlight display of the previous operation area when one of the plurality of operation areas is highlight display When,
When the trigger signal is received in the scan mode, the scan process is temporarily stopped, and a group interface corresponding to the selected area is displayed on the display unit, and the selected area therein receives the trigger signal, The group interface is displayed in an upper layer of the graphical interface, and the size of both the group interface and the graphical interface is the same, or the size of the group interface is the graphical interface. Smaller than the size of the step,
Detecting eye movement at the user's group interface by an eye movement detection module;
Selecting a function option from a plurality of function options of the group interface based on the eye movement and executing an operation , wherein the number of the plurality of operation areas in which the graphical interface is displayed is displayed by the group interface greater than the number of said plurality of functional options, the steps,
A method for operating an electronic device, comprising:   The method of operating an electronic device according to claim 1, wherein when the trigger signal is received in the scan mode, the operation is performed by driving the eye movement detection module. Detecting the eye movement at the group interface of the user by the eye movement detection module;
3. The method according to claim 1, further comprising: acquiring an image sequence of the user by an image acquisition unit, and causing the eye movement detection module to track the eye movement of the user by analyzing the image sequence. A method for operating the electronic device according to claim 1. The step of tracking the eye movement of the user by analyzing the image sequence includes reflecting the position of the user's pupil and the light emitted from the light emitting unit in the image sequence formed by illuminating the user's eyes. Detecting a position of a bright spot that is a point;
When the horizontal distance between the position of the pupil and the position of the bright spot is within an error range, it is determined that the eye movement is looking directly forward.
When the horizontal distance between the position of the pupil and the position of the bright spot exceeds the error range, and the position of the pupil is located to the left of the position of the bright spot, the eye movement moves to the left. It is determined that it is moving,
When the horizontal distance between the position of the pupil and the position of the bright spot exceeds the error range, and the position of the pupil is located to the right of the position of the bright spot, the eye movement moves to the right. It is determined that it is moving,
Determining whether the eye movement is an upward movement or a downward movement based on a vertical distance between the position of the pupil and the position of the bright spot;
The method of operating an electronic device according to claim 3, further comprising: determining a current gazing point of the eye movement based on a positional correspondence relationship between the position of the pupil and the position of the bright spot. . Before detecting the eye movement of the user by the eye movement detection module,
5. The electronic device according to claim 1, further comprising a step of calibrating reference information of the eye movement detection module by being guided to a designated position of the display unit viewed by the user. 6. How to operate. In the scan mode,
Detecting the eye movement of the user by the eye movement detection module;
The method of operating an electronic device according to claim 1, further comprising a step of determining whether to receive the trigger signal based on the eye movement.   The method of operating an electronic device according to claim 1, wherein the trigger signal is generated from an input unit. An image acquisition unit arranged towards the user of the electronic device;
A display unit for displaying a graphical interface having a plurality of operation areas;
A display unit and a processing unit connected to the image acquisition unit and executing a plurality of modules;
In the scan mode, the plurality of modules highlights the plurality of operation areas one by one based on the scan order, and when one of the plurality of operation areas is highlighted, highlights the previous operation area. A scan module to cancel the display;
When the trigger signal is received in the scan mode, the scan process is temporarily stopped, and a group interface corresponding to the selected area is displayed on the display unit, and the selected area therein receives the trigger signal, An interface switching module to be the highlighted display area of the operation area,
Driving an image acquisition unit to acquire an image sequence of the user, detecting eye movements in the group interface of the user by analysis of the image sequence, and based on the eye movements, select one of the functions options from the function option, have a and eye movement detection module to perform the operation,
The number of the plurality of operation areas in which the graphical interface is displayed is greater than the number of the function options in which the group interface is displayed.
And the group interface is displayed in an upper layer of the graphical interface, and the size of both the group interface and the graphical interface is the same, or the size of the group interface is smaller than the size of the graphical interface.
An electronic device characterized by that. The electronic apparatus according to claim 8 , wherein the scan module drives the eye movement detection module to perform an operation when receiving the trigger signal in the scan mode. The eye movement detection module includes:
In the image sequence, the position of the pupil of the user and the position of a bright spot which is a reflection point formed by the light emitted from the light emitting unit illuminating the eyes of the user are detected,
When the horizontal distance between the position of the pupil and the position of the bright spot is within an error range, it is determined that the eye movement is looking directly forward.
When the horizontal distance between the position of the pupil and the position of the bright spot exceeds the error range, and the position of the pupil is located to the left of the position of the bright spot, the eye movement moves to the left. It is determined that it is moving,
When the horizontal distance between the position of the pupil and the position of the bright spot exceeds the error range, and the position of the pupil is located to the right of the position of the bright spot, the eye movement moves to the right. It is determined that it is moving,
Determining whether the eye movement is upward or downward based on a vertical distance between the position of the pupil and the position of the bright spot;
Further electronic apparatus according the position relationship of both the position of said bright spot of the pupil, in claim 8 or 9, characterized in that determining the gaze point of the current of the eye movement. Wherein the plurality of modules, by guiding the designated position of said display unit in which the user sees, the claims 8 to 10, further comprising an inductive calibration module for calibrating a reference information of the eye movement detecting module The electronic device as described in any one. The scan module drives the eye movement detection module in the scan mode, causes the eye movement detection module to detect the eye movement of the user, and the processing unit receives the trigger signal based on the eye movement. A message is transmitted to the scan module and the interface switching module, the scan module temporarily stops the scanning process, and the interface switching module displays the group interface corresponding to the selected area in the display unit. The electronic device according to any one of claims 8 to 11 , wherein the electronic device is displayed. An input unit;
It is connected to the processing unit and the input unit to determine whether to receive the trigger signal from the input unit, and when the processing unit receives the trigger signal, a message is sent to the scan module and the interface switching module. The embedded control unit further includes an embedded control unit that causes the scan module to temporarily stop a scan process and to display the group interface corresponding to the selected area on the display unit by the interface switching module. Item 13. The electronic device according to any one of Items 8 to 12 . A storage unit connected to the processing unit for storing the module, data of the graphical interface and data of a plurality of group interfaces;
The graphical interface data defines one of a size of the graphical interface, each size of the plurality of operation areas to be displayed, and data of the plurality of group interfaces corresponding to each of the plurality of operation areas,
The data of each of the plurality of group interfaces defines a size of the corresponding group interface, a size of the plurality of function options to be displayed, and a function corresponding to each of the plurality of function options. The electronic device according to any one of 8 to 13 .

Images