TW202020625A - The method of identifying fixations real-time from the raw eye- tracking data and a real-time identifying fixations system applying this method - Google Patents

Abstract

The method of identifying fixations real-time from the raw eye- tracking data and a real-time identifying fixations system applying this method, which including steps of: presenting a default screen with at least one defined interest of area for a high-priority knowledge person by a display unit of an information processing device；capturing the first eye movement data which responding to the default screen by the high-priority knowledge person with a sensor unit of an eye movement sensing module, then transmitting the first eye movement data to the information processing device through a transmission unit of the eye movement sensing module；executing an application to real-time computing the first eye movement data for establishing an expert eye movement model with a computing unit of the information processing device, and the expert eye movement model includes at least one suitable attention area, one suitable attention sequential of said at least one suitable attention area and one suitable percentage of attention time of said at least one suitable attention；and storing the expert eye movement model with a first storage unit of the information processing device.

Description

Real-time identification method for eye movement gaze and instant identification system for eye movement gaze adopting the method

This case is about a method of analyzing eye movement behavior to establish an expert eye movement model, especially a method involving real-time identification of eye movement gaze to establish an expert eye movement model.

The eye is the window of the soul. 80% of the information that humans obtain from the surrounding environment comes from vision. In learning, the behavioral indicators of vision can also reflect the cognitive process of the learner. Conventional techniques often use post-event interviews, questionnaire surveys, or loud thinking to understand the subject's cognitive processing history. However, the above methods may interfere with learning; or there may be problems such as memory errors and social expectations errors.

Because attention has the characteristics of unobservable, immeasurable, and uncertain, it is difficult to measure effectively. However, in recent years, the development of eye tracking technology has solved the problem of attention measurement. Rayner defines the development of eye tracking technology through the following three stages:

The first stage: From 1879, Javal found that eye movements were not continuous through simple observation and analysis, but consisted of a series of saccades and fixations.

The second stage: due to the rise of behaviorism, the researchers regarded the process of internal cognitive processing as a black box and focused on the essence of eye movement. They did not try to infer the internal learning and cognitive process of the subjects.

The third stage: Since 1970, advances in information technology and eye tracking technology have made eye movement data accurate and easy to obtain. The use of eye movement data for cognitive system research has gradually received attention, and related research has also mushroomed. Used in neuroscience, psychology, marketing, advertising, industrial engineering, human factors engineering, and computer science.

Eye-mind hypothesis (Eye-mind hypothesis) is proposed in the literature. When the subject looks at a certain text or a certain area, it means that the person is processing the information in the text or the area, and the gaze time represents the processing of the The time of regional information means that there is a close connection between the subject’s eye movement data and its focus. The researcher can analyze the subject’s psychological state through the eye movement data.

The innermost retina (retina) of the eyeball includes cone and rod photoreceptor cells. Rod cells are more sensitive to light and can feel very weak light, but they cannot distinguish various colors; cone cells It has the ability of color vision and is also responsible for seeing the details of the image, but it must be able to operate smoothly in a well-lit environment. Most of the cone cells are located on the omentum directly behind the crystal. Here is the place with the highest acuity, called the fovea.

When we look at an object, we will constantly rotate our eyes. The purpose is to project the image of an external object to the central fossa, and the content of the field of view with a 1 degree angle of view to the left and right of the gaze point will fall into the central fossa. Now when the central fovea is in view, the object can be recognized without moving the eyeball; however, if the image of the object appears around the central fossa or the area around 更, you must move the eyeball or even turn the head to recognize the object. Therefore, the researcher can understand the movement trajectory of the subject's attention by tracking the position of the eye's gaze point or the movement of the eyeball relative to the head.

Some literature pointed out that the test subject’s eye movement trajectory is not absolutely unidirectional from the first word to the last word. Sometimes after reading a certain sentence, the eyeball’s gaze point will be moved back to the previous one. The sentence, this phenomenon is called "regression" (regression). Retrospection often occurs when reading a sentence that is confusing, and the position looked back is considered to provide clues to resolve its confusion. When the subject's reading ability is weaker, the article's consistency is worse, or the reading content is more difficult. When it is high, the number of testees' gaze and the total time of gaze will also increase significantly. Therefore, the difference between those with poor reading ability and those with general reading ability is that the former has a longer gaze time, shorter gaze time, and more gaze. The number of times and more look back.

There are also documents that the longer the viewing time of the appropriate prompt animation area is, the better the learning performance of the longer time is compared with the shorter viewing time of the appropriate prompt animation area. Those with abilities have more gaze points in the initial stage, while those with low-spectrum abilities use more times and time to look back in the later stages.

Therefore, experts and novices are at different places. The difference between visual messages is that experts will focus more on key messages. Therefore, researchers can not only use eye movement data to judge and distinguish experts from novices, but also can be used to Guide and train new students to focus their attention on key messages to enhance their learning effectiveness.

Known technologies such as the patent "Teaching Eye Tracker for Expert Knowledge Acquisition" of the Republic of China TWI485666, which reveals a kind of teaching eye that can record all the thinking and attention processes of expert personnel in teaching to achieve two-way and one-way teaching of expert knowledge acquisition The moving instrument is characterized by "comprising: a two-way photography module installed on the teaching eye tracker. The two-way photography module includes a first photography module capable of photographing environmental topography and a first photography module. A group of information links and a second photography module capable of shooting eyeball attention movements; a display lens is provided on the teaching eye tracker and is connected to the information of the two-way photography module; one is connected to the two-way photography module and the display The central processing module for lens information connection is set on the teaching eye tracker, and the central processing module information is connected to a transmission unit; and a communication module is set on the teaching eye tracker, and the communication The module is linked to the above central processing module information."

However, the applicable scope of the teaching eye tracker is limited to the form of a mobile eye movement sensing module worn on the human body; and when the teaching eye tracker is applied to one-way teaching, it is the reflection of the professor. The process and the direction of attention are recorded. The patent does not describe and capture and record those eye movement data, nor does it disclose how to use the eye movement data to calculate to establish an expert eye movement model and whether the patent can achieve The purpose of creating an expert eye movement model in real time; when used in two-way interactive teaching, the teaching eye tracker needs to rely on the professor’s personal line-of-sight broadcast connection to each student’s eye tracker for synchronous guidance, and cannot provide asynchronous synchronization. The guidance function also cannot provide adaptive real-time feedback and guidance functions for multiple students. Therefore, there is an urgent need for a novel real-time identification method for eye movement gaze.

One of the purposes of this case is to disclose a real-time identification method for eye movement gaze, which can simultaneously execute reading and processing operations of eye movement data of high-preparation knowledge persons by executing an application program, so as to establish an expert eye movement model in real time.

Another purpose of this case is to disclose a real-time recognition method for eye-tracking gaze, which further has a real-time feedback method for eye-tracking gaze, which can simultaneously read the eye-tracking data of those with low prior knowledge by executing the application And process the operation, and compare the operation result with the expert eye movement model in real time and perform real-time eye movement feedback and guidance for those with low prior knowledge based on the real-time comparison.

Another objective of this case is to disclose a real-time identification method for eye movement gaze, in which the application can provide a preset screen with at least one defined area of interest to the function of the person with high prior knowledge or the person with low prior knowledge, and It can simultaneously read and follow up the eye movement data during the test.

Another purpose of this case is to reveal a real-time identification method of eye movement gaze, which focuses on: "seeing the right place" (appropriate attention area), "seeing the right order" (appropriate attention order) and "seeing long enough" ”(Appropriate attention to time percentage) and other three parts, to simplify the calculation and achieve the purpose of creating an expert eye movement model in real time.

Another object of this case is to disclose a real-time recognition method for eye movement gaze, in which the eye movement sensing module has a second storage unit and a warning unit, the second storage unit is used to store a preset threshold of loss rate, It can provide a warning function through the warning unit when the loss rate of one of the eye movement data is greater than the preset threshold of the loss rate.

Another objective of this case is to disclose a real-time recognition method of eye movement gaze, in which the information processing device provides user identification function to provide different users to log in and use the real-time feedback method of eye movement gaze to adjust their attention habits, and thus improve Its learning effectiveness.

To achieve the aforementioned objective, a real-time recognition method for eye movement gaze is proposed, which includes the following steps: presenting a preset screen with at least one defined area of interest to a person with high prior knowledge on a display unit of an information processing device; A sensing unit of an eye movement sensing module acquires first eye movement data in response to the preset picture by the person with advanced knowledge, and transmits the first eye movement data through a transmission unit of the eye movement sensing module To the information processing device; an operation unit of the information processing device executes an application program to perform a real-time operation on the first eye movement data to create an expert eye movement model, and the expert eye movement model includes at least one appropriate attention An area, an appropriate attention sequence of one of the at least one appropriate attention area and a percentage of appropriate attention time of one of the at least one appropriate attention area; and storing the expert eye movement model in a first storage unit of the information processing device.

In one embodiment, it further has an eye movement gaze real-time feedback method, including the steps of: after storing the expert eye movement model, presenting the pre-defined area of interest with at least one defined area on the display unit of the information processing device Set a picture to a person with low prior knowledge; use the sensing unit of the eye movement sensing module to obtain second eye movement data in response to the preset picture by the person with low preliminary knowledge, and pass the eye movement sense The transmission unit of the test module transmits the second eye movement data to the information processing device; the operation unit of the information processing device executes the application program to perform a real-time operation on the second eye movement data and calculate the operation result with the The expert eye movement model performs a real-time comparison; and according to the real-time comparison, the display unit of the information processing device performs a real-time feedback and guidance function of a corresponding eye movement to the person with low prior knowledge.

In one embodiment, the application can provide a preset screen with at least one defined area of interest to the function of the person with high prior knowledge or the person with low prior knowledge, and can simultaneously perform the first glance when performing the test Reading and subsequent processing of moving data or the second eye movement data.

In one embodiment, the information processing device includes a desktop computer, a notebook computer, or a handheld device; the transmission unit includes a wired transmission unit or a wireless transmission unit; and the eye movement sensing module includes a headgear type An eye movement sensing module, a mobile eye movement sensing module or a telemetry eye movement sensing module.

In one embodiment, the preset picture is selected from one of the group consisting of text, static image, dynamic image or video, or a combination thereof.

In an embodiment, the real-time feedback and guidance function is selected from one or a combination of groups consisting of text, images, voices, and freezes or repeated playback of the video to provide feedback and guidance.

In one embodiment, the eye movement sensing module further has a second storage unit and a warning unit, the second storage unit is used to store a preset threshold of the loss rate, which is based on the first eye movement data Or one of the second eye movement data loss rate is greater than the loss rate preset threshold can provide a warning function through the warning unit, and the first storage unit, the second storage unit includes a temporary storage, a fast Retrieve memory or a random access memory.

In one embodiment, the information processing device further provides a user identification function to provide the user to log in and use the eye-tracking gaze real-time feedback method to adjust their attention habits.

In one embodiment, the user identification function includes a password protection method or a biometric protection method.

In one embodiment, a real-time recognition system for eye movement gaze is proposed, which has an information processing device and an eye movement sensing module, wherein the information processing device has an arithmetic unit and the arithmetic unit executes the eye The real-time identification method of dynamic gaze realizes the real-time identification process of one-eye gaze.

In order to enable your review committee to further understand the structure, features and purpose of the present invention, the drawings and the detailed description of the preferred embodiments are attached as follows.

Please refer to FIG. 1, which shows a flowchart of the steps of the real-time recognition method for eye movement gaze according to a preferred embodiment of the present case.

As shown in the figure, the real-time recognition method of eye movement gaze in this case is implemented by an application program and includes the following steps: presenting a preset screen with at least one defined area of interest to a display screen of a display unit of an information processing device to a Those with advanced knowledge; (Step a); Obtain the first eye movement data in response to the preset picture by one of the sensing units of an eye movement sensing module, and through the eye movement sensing module A transmission unit transmits the first eye movement data to the information processing device; (step b); an operation unit of the information processing device executes the application program to perform a real-time operation on the first eye movement data to create an expert eye A dynamic model, and the expert eye movement model includes at least one appropriate attention area, a suitable attention sequence of one of the at least one suitable attention area, and a percentage of appropriate attention time of one of the at least one suitable attention area; (step c); and The expert eye movement model is stored in a first storage unit of the information processing device; (step d).

The expert eye movement model is established based on real-time calculation of the first eye movement data of at least one person with high prior knowledge.

Since the eye movement data is composed of a series of time and coordinates, the eye movement data obtained by the eye movement sensing module of the conventional technology is presented in the EDF file format, and the software provided by the manufacturer (such as Data Viewer software) needs to be reused ) After the test, data reading and subsequent processing are performed, so the purpose of real-time processing cannot be achieved.

This case uses a self-made application, which can not only provide the tester with a preset screen with at least one defined area of interest from one or a combination of text, static images, dynamic images or videos, but also to make the eyes move The eye movement data acquired by the test module simultaneously reads the data and performs subsequent processing during the test.

The processing procedure is: recognizing gaze and saccade gaze data from the acquired gaze data; converting the gaze and saccade gaze data into the coordinate position of the gaze point and the start and end time of the gaze point; Compare the coordinate positions of different gaze points with at least one defined area of interest in order to identify the area of interest that the subject is interested in; calculate the percentage of time the subject is interested in each area of interest, the total number of gazes and the total Gaze time; and according to the percentage of time of interest in each of the interest areas, the total number of gazes and the total gaze time calculated to include at least one appropriate area of interest, one of the at least one appropriate area of interest and the appropriate sequence of interest Describe an eye movement model of at least one appropriate attention area and appropriate attention time percentage.

In addition, the case is simplified by focusing on three parts: "seeing the right place" (appropriate attention area), "seeing the right order" (appropriate attention order) and "seeing long enough" (appropriate attention time percentage) to simplify The calculation can achieve the purpose of establishing an expert eye movement model in real time.

Please refer to FIG. 2, which illustrates a flowchart of a method for real-time recognition of eye movement gaze according to another preferred embodiment of the present case.

As shown in the figure, the real-time recognition method of eye movement gaze in this case, which further has a real-time feedback method of eye movement gaze, is realized by using the application, and includes the following steps: after storing the expert eye movement model, the information processing The display unit of the device presents the preset screen with at least one defined area of interest to a person with low prior knowledge; (step e); the low priority is acquired by the sensing unit of the eye movement sensing module The knowledgeable person responds to the preset image with a second eye movement data, and transmits the second eye movement data to the information processing device through the transmission unit of the eye movement sensing module; (step f); using the information The computing unit of the processing device executes the application program to perform a real-time calculation on the second eye movement data and perform a real-time comparison with the expert eye movement model; (step g); and according to the real-time comparison The display unit of the information processing device performs a real-time feedback and guidance function of a corresponding eye movement to the person with low prior knowledge; (step h).

Wherein, the application can provide a preset screen with at least one defined area of interest to the person with high prior knowledge or the person with low prior knowledge, and can synchronize the first eye movement data or the second eye movement The reading and subsequent processing of the data is for real-time eye feedback and guidance for the adaptive eye movements for those with low prior knowledge.

The preset picture is selected from one of the group consisting of text, static image, dynamic image or video, or a combination thereof; the real-time feedback and guidance function is selected from the stop of text, image, voice, video or Repeatedly play one or a combination of the groups formed for feedback and guidance.

Please refer to FIG. 3a to FIG. 3d together, wherein FIG. 3a illustrates a schematic diagram of a preferred embodiment of the real-time feedback method of the eye-tracking gaze of the present invention. The preset screen has at least one defined area of interest, and FIG. 3b illustrates the present invention. A schematic diagram of the instant feedback and guidance function of a preferred embodiment of the instant feedback method of the eye movement gaze, FIG. 3c illustrates the instant feedback and guidance function of another preferred embodiment of the instant feedback method of the eye movement gaze of the case FIG. 3d is a schematic diagram showing the instant feedback and guidance function of yet another preferred embodiment of the instant feedback method of the eye movement gaze in this case.

This embodiment presents a preset screen of the theme of "refraction of light in different media" for those with low knowledge, and defines seven Area Of Interest (AOI), but not limited to this. Those with low prior knowledge can understand the relationship between refraction and reflection by changing the angle of incidence of a light beam, and can also understand the propagation of light between different media by changing the medium.

As shown in FIG. 3a, the seven interest areas include "upper panel area U", "prompt area H", "lower panel area D", "beam area L", "reflection area R", and "refractive area F" And the "remaining area O" that does not belong to the above area. Then analyze the eye movement data using the seven interest areas as a unit. The black point "gaze point P" located in the "reflection area R" in the figure is the focus position of the current subject at this time, and the gaze point P The location changes with the subject's focus on the area.

As shown in FIG. 3b, when the sensing unit (not shown in the figure) of the eye movement sensing module acquires the eye movement data of the person with low prior knowledge responding to the preset screen, if the operation results in the " When the position of the gaze point P stays in the "remaining area O" (that is, the subject is distracted) or the testee answers the question in the "prompt area H" but answers incorrectly, a prompt screen will appear in the "prompt area H" to test the subject Give instant feedback.

As shown in Figure 3c, "Guide Eye G" appears in the "upper panel area U" to remind the subject that they should focus on the "upper panel area U" (the appropriate focus area of the expert eye movement model) at this time, and when the subject is tested When the person pays attention to the appropriate area of interest (that is, the position of "gaze point P" falls within the range of "upper panel area U"), the surrounding color of "guide eye G" gradually changes from light to dark (that is, the expert eye movement model Pay close attention to the percentage of time), so as to guide the subjects to "look at the right place long enough" and hope to see the clues.

As shown in Figure 3d, "Guide Eye G" appears in "Lower Panel Area D" to remind the subject that they should then focus on "Lower Panel Area D" (that is, the appropriate attention order of the appropriate focus area of the expert eye movement model) , And when the subject pays attention to the appropriate area of interest (that is, the position of "gaze point P" falls within the range of "lower panel area D"), the color around the "guide eye G" will gradually change from light to dark (That is, the appropriate attention time percentage of the expert eye movement model), so as to guide the subjects to "see the right place", "see the right order" and "see long enough" to adjust their attention habits and thereby improve their learning performance.

Please refer to FIG. 4a, which illustrates a schematic diagram of an eye movement gaze real-time recognition system architecture using the instant eye movement gaze real-time recognition method according to a preferred embodiment of the present case.

As shown in the figure, the real-time gaze recognition system includes: an information processing device 100 and an eye movement sensing module 200.

The information processing device 100 has a display unit 110, an arithmetic unit 120, and a first storage unit 130; the eye movement sensing module 200 has a sensing unit 210 and a transmission unit 220.

The display unit 110 is used to present a preset picture with at least one defined interest area to at least one subject 300, and the sensing unit 210 is used to obtain the at least one subject 300 to perform the preset picture The responding eye movement data (including the first eye movement data of those with high prior knowledge or the second eye movement data of those with low prior knowledge), and transmits the first eye movement data or the second eye movement data to the transmission unit 220 In the information processing device 100, the operation unit 120 is used to execute an application program (not shown in the figure) to perform real-time operation on the first eye movement data to create an expert eye movement model and perform real-time operation on the second eye movement data The operation result is compared with the expert eye movement model in real time, and the first storage unit 130 stores the expert eye movement model.

Wherein, the information processing device 100 includes, for example but not limited to, a desktop computer, a notebook computer or a handheld device; the first storage unit 130 includes, for example but not limited to, a register, a cache memory or a Random access memory; the transmission unit 220 includes, for example but not limited to, a wired transmission unit or a wireless transmission unit; the eye movement sensing module 200 includes, for example but not limited to, a head-set eye movement sensing module, a mobile eye A motion sensing module or a telemetry type eye movement sensing module.

The expert eye movement model includes at least one suitable attention area, a suitable attention sequence of one of the at least one suitable attention area, and a suitable attention time percentage of one of the at least one suitable attention area.

The information processing device 100 further provides a user identification function (not shown in the figure) to provide the user to log in and use the eye-tracking gaze real-time feedback method to adjust their attention habits, thereby improving their learning performance; the use of The personal identification function includes a password protection method or a lifetime object identification protection method. This is a conventional technology and will not be repeated here.

Please refer to FIG. 4b, which illustrates a schematic diagram of an eye movement gaze real-time recognition system architecture using the instant eye movement gaze real-time recognition method according to another preferred embodiment of the present case.

As shown in the figure, the real-time gaze recognition system includes: an information processing device 100; and an eye movement sensing module 200.

The information processing device 100 has a display unit 110; an arithmetic unit 120; and a first storage unit 130, and the eye movement sensing module 200 has a sensing unit 210; a transmission unit 220; and a second storage unit 230;与一个警示unit 240.

The display unit 110 is used to present a preset picture with at least one defined interest area to at least one subject 300, and the sensing unit 210 is used to obtain the at least one subject 300 to perform the preset picture The responding eye movement data (including the first eye movement data of those with high prior knowledge or the second eye movement data of those with low prior knowledge), and transmits the first eye movement data or the second eye movement data to the transmission unit 220 In the information processing device 100, the operation unit 120 is used to execute an application program (not shown in the figure) to perform real-time calculation on the first eye movement data to create an expert eye movement model and perform real-time operation on the second eye movement data Calculate and compare the operation result with the expert eye-movement model in real time. The first storage unit 130 stores the expert eye-movement model. The second storage unit 230 is used to store a loss threshold , If one of the first eye movement data or the second eye movement data has a loss rate greater than the loss rate preset threshold, such as but not limited to the high pre-knowledge or the low-pre-knowledge When the time is distracting, dozing or the machine is malfunctioning, the warning unit 240 can provide a warning function.

Wherein, the information processing device 100 includes, for example but not limited to, a desktop computer, a notebook computer or a handheld device; the first storage unit 130 and the second storage unit 230 include but not limited to a temporary memory, A cache memory or a random access memory; the transmission unit 220 includes, for example but not limited to, a wired transmission unit or a wireless transmission unit; the eye movement sensing module 200 includes, for example but not limited to, a head-set type eye movement sensing Module, a mobile eye movement sensing module or a telemetry eye movement sensing module.

The expert eye movement model includes at least one suitable attention area, a suitable attention sequence of one of the at least one suitable attention area, and a suitable attention time percentage of one of the at least one suitable attention area.

The information processing device 100 further provides a user identification function (not shown in the figure) to provide the user to log in and use the eye-tracking gaze real-time feedback method to adjust their attention habits, thereby improving their learning performance; the use of The personal identification function includes a password protection method or a biometrics protection method. This is a conventional technology and will not be repeated here.

The principle of the invention in this case will be explained below:

This case focuses on two eye movements: gaze and saccade:

Duchowski divides the eye movements of primates into the following types: fixation, saccade, smooth pursuit, vestibular, convergence, and nystagmus ).

Among them, the smooth tracking is the phenomenon of slowly staring at the moving target; the vestibular eye movement is to compensate for the movement of the head or body to maintain the eye movement response to watching something; the convergence effect is to obtain a three-dimensional perception; nystagmus It is to avoid the subtle movements generated autonomously depending on the habit of cells. The four eye movements mentioned above are not related to cognition and learning, so we will not discuss them here. The focus of this case is on the two eye movements: fixation and saccade.

The saccade is a rapid eye movement to make the central fossa focus on a stimulus. The average speed of the saccade is usually as high as 500 degrees per second. The literature indicates that the distance of the saccade is related to the nature of the work performed by the subject, such as About 7 to 9 letters when reading English; about 2.5 to 3.3 words when reading Chinese; about 4 to 5 perspectives when viewing pictures.

The relatively static 狀 between the two saccades is the gaze, which means that the subject is now recognizing and recognizing the perception 理. The gaze time is also related to the nature of the operation being performed, for example, about 225 to 250 milliseconds when reading, and about 260 to 330 milliseconds when viewing pictures.

When gazing, the eyeball is not completely still, but also includes subtle eye movements such as nystagmus and micro-jumping; and the time required for the subject to judge and determine the position of the next gaze point and start the next gaze is the jump. As for the saccade latency, the gaze time referred to in this case includes the time required for the current stimulation and the latency to saccade.

The eye tracking methods used in this case are "eyeball image analysis" and "pupillary and corneal image merge analysis" :

Because the threshold time of the eyeball’s line of sight to reach the area of interest is extremely short, how to accurately capture the subtle movements of the subject’s eyes and eliminate unconscious eye movements such as subconscious eye movements or habitual blinking and other noises, The literature proposes the following four eye tracking methods:

1. "Electro-Oculography":

This method uses an electrode sheet to record the electro-oculogram (EOG) obtained by recording the myoelectric potential of the skin around the eye frame to determine the movement of the eyeball. Although this method is suitable for measuring saccade latency, The measurement of the gaze position is less accurate.

2. "Scleral Search Coil":

This method is to wear contact lenses with a built-in metal induction coil on the cornea and sclera of the subject, and then use the electromagnetic field frame to measure the movement of the coil. Although this method is more accurate, it is an invasive measurement method. Used in special situations.

3. "Photo-OculoGraphy" (Photo-OculoGraphy):

This method projects an infrared light source onto the surface of the eyeball of the subject, and then captures the eyeball image with a high-power camera, and quickly analyzes the pupil shape of the eyeball and the boundary position of the iris through the computer to infer the gaze position of the eyeball and Direction of movement. The eye movement data obtained by this method is the displacement of the subject's eyeball relative to the head, so it needs to be worn on the subject's head; or use the head frame or chin stand to fix the subject's head to ensure a stable eyeball position .

4. "Video-Based Combined Pupil" (Video-Based Combined Pupil):

This method simultaneously records and analyzes the difference between the reaction point of the far infrared rays on the cornea and the central position of the pupil in a specific direction to calculate the position of the subject's eyeball and the movement of the head. This method only needs a simple calibration to obtain accurate head and eyeball positions. Therefore, the subject can maintain free movement of the head and effectively record the eyeball trajectory.

In addition, the eye movement sensing module can be divided into the following three categories according to its sensing unit:

1. Headgear type: The head of the subject must remain stationary, the chin is placed on the eye movement sensing module and the eye movement is captured through the sensing unit provided above the eyeball. The advantage of the headgear type is higher accuracy, and the disadvantage is that the subject's posture is more rigid and unnatural, and it cannot properly simulate the real situation.

Second, mobile: It is worn on the head of the subject and can combine the real scene with the eye movement data. The advantage of the mobile type is that the test subject can be carried in a variety of scenes or even sports for testing, so it can be more widely used in real environments.

3. Telemetry: It is placed under the display unit viewed by the test subject. Compared with the mobile type, the test subject does not need to wear any instruments; compared to the head-set type, the test subject can perform the head during the test. Move, so you can relax and test.

The eye movement sensing module records the subject's eye movement position at a certain sampling rate (usually represented by X and Y coordinates), which means that the eye movement data is composed of a series of time points and coordinates.

In this case, a real-time identification system for real-time gaze was completed and the feasibility of the method proposed in this case was verified by experiments.

Experimental equipment description:

The following experiment uses a Dell OptiPlex Gx260 compatible personal computer; the default screen with at least one defined area of interest is presented on a Dell 19-inch SVGA screen and the screen resolution 度 is 1024 × 768 pixels; the eyes of the subject during the experiment 65 cm from the screen; Dispersion-based algorithm is used to determine the gaze point, that is, as long as the cumulative duration within a radius of 40 pixels exceeds 150 milliseconds, the gaze point is determined, and the gaze interval is shorter than 100 milliseconds or longer than 1000 milliseconds are 不列 invested 列 analysis.

Before the start of the formal experiment, in order for the eye movement sensing module to be accurately synchronized with the subject's eyeball gaze position, the subject must be subjected to 行9 point calibration and validation. This nine-point calibration procedure refers to The eye movement sensing module generates an estimation formula based on the rotation angle 度 of 9 position points evenly distributed on the corresponding display unit of the subject’s eyeball and the relative object 理離來 of the 9 position points, and calculates the eye rotation angle 度 on the screen corresponding to 來Position coordinates.

As long as the examiner finds that the eyes of the subject have a slight deviation in a certain test, he will re-enter the 行9-point calibration operation after the test to ensure that 不 will affect the judgment of eye movements due to eye deviation讀.

Experiment 1, verify the completeness of the application mentioned in this case in obtaining eye movement data:

This experiment is equipped with Eye Tribe eye movement sensing module, three subjects were asked to pay attention to the same preset picture, the preset picture is a static picture, the eye movement sensing module was 30 times/second and 60 times/ The sampling frequency of seconds is used to obtain eye movement data of the three subjects in three different types of attention (not blinking attention, attention for 1 minute and attention for 5 minutes), as shown in Table 1.

Table 1

Among them, the non-blinking attention type is because there are no blinks, so the eye movement data acquired by the eye movement sensing module is the most complete among the three attention forms; the 5-minute attention type is also because the number of blinks is more than 1 minute. As a result, the loss rate of eye movement data obtained by the eye movement sensing module is high, and the completeness is low.

According to the experimental results, when the sampling frequency is 30 times/second, the completeness of non-blink attention, attention for 1 minute and attention for 5 minutes are 100%, 99.74% and 98.38% respectively; when the sampling frequency is 60 times/second, no The completeness of blinking attention, attention for 1 minute and attention for 5 minutes are 99.93%, 92.05% and 87.02%, respectively. Therefore, it is verified that the application mentioned in this case can achieve a certain level of completeness in obtaining eye movement data.

Experiment two: verify the accuracy of the gaze point judgment of the application mentioned in this case :

This experiment uses Eye Tribe eye movement sensing module and self-made application program to analyze eye movement data tracking and analysis data, and compares it with another mature eye movement sensing system FaceLAB 4.5 combined with GazeTracker 9.0 software. The sampling frequency is 60 times/second to verify the accuracy of the real-time identification method of eye movement gaze mentioned in this case.

Among them, accuracy refers to the difference between the average value of each independent measurement and the known true value of the data (that is, the degree of agreement with the theoretical value).

The test is divided into four procedures, of which there are two procedures for the dynamic picture test procedure and the static picture test procedure, which are used to obtain eye movement data and judge the gaze point. Comparing with the eye tracking system FaceLAB 4.5 and calculating the number of differences in gaze point judgment, the average accuracy of 99.94% is obtained, as shown in Table 2.

Table 2

From the experimental results, it can be seen that the accuracy rate of the application mentioned in this case at the gaze point is between 99.82% and 99.99%, and the average accuracy rate is also 99.94%.

Experiment three: verify that the application mentioned in this case can distinguish experts (high group) and novices (low group):

The participants in this experiment were 5 graduate students, with an average age of 25.8 years. In this experiment, the ASL Mobile Eye module was used in conjunction with a self-made application program, and the sampling frequency was 60 Hz.

The experiment is divided into two parts: the first part is to present a preset picture to the subject and conduct a 5-minute eye movement sensing experiment. This experiment is to use the self-made application to present the theme of the theme of "refraction of light in different media" Setting the picture, the subject can change the incident angle of the light beam to understand the relationship between refraction and reflection, and can also change the medium to understand the light propagation between different media.

Please refer to FIG. 5, which shows a schematic diagram of the predefined screen of Experiment 3 in this case with six defined regions of interest.

As shown in the figure, this experiment defines six interest areas on the default screen, including "upper panel area U", "beam area L", "lower panel area D", "reflection area R", "refractive area F" And the "remaining area O" that does not belong to the above area, and then the six interest areas are used as units 來 to analyze their eye movement data.

When a visual stimulus appears, the subject will give priority to the area of interest that attracts the most attention or obtains the most important information in the limited attention time. The gaze time of the subject’s gaze from entering an area of interest to the time before leaving is the “total gaze time”. The subject’s gaze may enter and exit the area of interest more than once. The total gaze time includes the sum of at least one gaze time, and The longer the gaze time, the more information the subject wants or needs to spend 理 the area of interest, reflecting the subject’s ongoing time-consuming awareness of the area of interest 理 or the area of interest is more likely to cause the test The attention of the people.

The second part is a written test. The testee must answer 4 multiple-choice questions on the topic of "refraction of light in different media" and explain the reasons for choosing this option. Each multiple-choice question and explanation reason are allocated 1 point , A total of 8 points.

Subjects were divided into high and low groups according to the score of the written test. Among them, the average score of the high group is 8 points (standard deviation is 0); the average score of the low group is 3.4 points (standard deviation is 3.01).

The eye movement data of high group and low group are shown in Table 3.1 and Table 3.2.

chart 3.1

Table 3.2

From the above table, the high group focus on: the beam area L (29.57%), the reflection area R (3.38%) and the refractive area F (19.38%) compared to the low group focus on the same beam area L ( 13.24%), the reflection area R (1.89%) and the refraction area F (11.73%) take nearly twice as much time; in contrast, the low group focuses on the upper panel area U (14.75%) and the lower panel Area D (16.02%) and the rest of the area O (42.38%) are also more time-consuming than the high group focus on the upper panel area U (10.63%), the lower panel area D (8.39%) and the remaining area O (28.65% ) The time spent is almost twice as much.

The low group spends insufficient time and frequency in the appropriate area of interest (beam area L, reflection area R and refraction area F), that is to say, the low group has no "look right" (appropriate area of interest), nor "Looking long enough" (appropriate attention to the percentage of time) makes its learning less effective. As for "seeing the right order" (appropriate order of attention), we will not discuss it in this experiment.

The aforementioned literature states that ``viewing the appropriate reminder animation area for a longer time (higher knowledge) has better learning performance than viewing the appropriate reminder animation area for a shorter time (lower knowledge)'' and another document states that Art-trained learners (higher preliminaries) tend to gaze at creative elements (appropriate areas of concern) to discover themes, while untrained art trainers (lower preliminaries) gaze at the use of image elements and semantics (not appropriate) Focus area)." The results of this experiment are also consistent with the conclusions of the above literature.

In addition, there are also documents that "use the eye movement sensing module to record the subject's attention to the relevant features (appropriate areas of concern) and irrelevant features (non-appropriate areas of interest) when learning the categorization task. After analysis of the force distribution situation, it was found that after the subject is proficient in the complex point assignment or after several attempts, the subject's attention distribution will become more optimized."

To sum up, the reason why experts (high-preparation knowledge) are better than experts (low-preparation knowledge) when they are at the place 理 is because experts focus their attention on key messages (appropriate attention area), and this case can not only be used Eye movement data 來 Judge and distinguish between experts and novices, and focus on: "see the right place" (appropriate attention area), "see the right order" (appropriate attention order) and "see long enough" (appropriate attention time) (Percentage) and other three parts to simplify the calculation and achieve the purpose of creating an expert eye movement model in real time. Therefore, the expert eye movement model can be used to guide and train the students to focus their attention on "seeing the right place", "seeing the right order" and "seeing long enough" and perform the corresponding eyeball real-time feedback and guidance functions and Provide a visual function of the gap between their eye movement data and the expert's eye movement model to remind them to adjust their attention habits and improve their learning performance.

With the design disclosed above, this case has the following advantages:

1. This case discloses a real-time identification method for eye movement gaze, which can simultaneously execute the reading and processing operations of eye movement data of high-preparation knowledge persons by performing an application program to establish an expert eye movement model in real time.

2. This case discloses a real-time recognition method for eye-tracking gaze, which further has a real-time feedback method for eye-tracking gaze, which can perform reading of eye-tracking data of those with low prior knowledge by carrying out the application while testing And process the operation, and compare the operation result with the expert eye-movement model in real time and follow the real-time comparison to perform adaptive eye movement real-time feedback and guidance for those with low prior knowledge.

3. This case discloses a real-time recognition method for eye movement gaze, in which the application can provide a preset screen with at least one defined area of interest to the function of the person with high prior knowledge or the person with low prior knowledge, and can perform During the test, reading and subsequent processing of the first eye movement data or the second eye movement data are performed simultaneously.

4. This case discloses a real-time identification method of eye movement gaze, which focuses on: "seeing the right place" (appropriate attention area), "seeing the right order" (appropriate attention order) and "seeing long enough" (appropriate Pay attention to the percentage of time) and other three parts, in order to simplify the calculation and achieve the purpose of creating an expert eye movement model in real time.

5. This case discloses a real-time recognition method for eye movement gaze, in which the eye movement sensing module has a second storage unit and a warning unit, the second storage unit is used to store a preset threshold of the loss rate When the loss rate of one of the first eye movement data or the second eye movement data is greater than the preset threshold of the loss rate, a warning function can be provided through the warning unit.

6. This case discloses a real-time recognition method for eye movement gaze, in which the information processing device provides user identification function to provide different users to log in and use the real-time feedback method of eye movement gaze to adjust their attention habits, thereby improving their learning performance .

The case disclosed in this case is a preferred embodiment, and any part of the changes or modifications that originated from the technical ideas of this case and can be easily inferred by those skilled in the art will not deviate from the scope of patent rights in this case.

In summary, regardless of the purpose, means and effects of this case, this case is showing its technical characteristics that are very different from the conventional ones, and its first invention is in practical use, and it is also in compliance with the patent requirements of the invention. Granted patents as soon as possible, so as to benefit the society and feel virtuous.

Step a: A display unit of an information processing device is used to present a preset screen with at least one defined area of interest to a person with high prior knowledge.

Step b: Obtain the first eye movement data in response to the preset image by a sensor unit of an eye movement sensing module, and transmit the first eye movement data through a transmission unit of the eye movement sensing module Glance data to the information processing device.

Step c: An application unit of the information processing device executes the application program to perform a real-time operation on the first eye movement data to create an expert eye movement model, and the expert eye movement model includes at least one appropriate area of interest Describe the appropriate attention sequence of one of the at least one suitable attention area and the percentage of suitable attention time of one of the at least one suitable attention area.

Step d: The expert eye movement model is stored in a first storage unit of the information processing device.

Step e: After storing the expert eye-movement model, the display unit of the information processing device is used to present the preset screen with at least one defined area of interest to a person with low prior knowledge.

Step f: Use the sensing unit of the eye movement sensing module to obtain second eye movement data in response to the preset picture by the person with low prior knowledge, and pass the transmission unit of the eye movement sensing module Transmitting the second eye movement data to the information processing device.

Step g: Use the computing unit of the information processing device to execute the application program to perform a real-time calculation on the second eye movement data and perform a real-time comparison with the expert eye movement model.

Step h: According to the real-time comparison, the display unit of the information processing device performs a real-time feedback and guidance function of a corresponding eye movement to the person with low preliminaries.

U: upper panel area

H: prompt area

D: Lower panel area

L: beam area

R: reflective area

F: refraction area

O: the rest of the area

P: Gaze point

G: Guide the eyeball

100: Information processing device

110: display unit

120: arithmetic unit

130: first storage unit

200: Eye movement sensing module

210: sensing unit

220: transmission unit

230: second storage unit

240: warning unit

FIG. 1 is a flowchart showing the steps of a real-time recognition method for eye movement gaze according to a preferred embodiment of this case.

FIG. 2 is a flowchart showing the steps of a real-time recognition method for eye movement gaze according to another preferred embodiment of this case.

FIG. 3a illustrates a schematic diagram of a preset image defining at least one region of interest in a preferred embodiment of the instant feedback method for eye-tracking gaze in this case.

FIG. 3b shows a schematic diagram of the instant feedback and guidance function of a preferred embodiment of the instant feedback method of the eye movement gaze in this case.

FIG. 3c shows a schematic diagram of the instant feedback and guidance function of another preferred embodiment of the instant feedback method of the eye movement gaze in this case.

FIG. 3d shows a schematic diagram of the instant feedback and guidance function of yet another preferred embodiment of the instant feedback method of the eye movement gaze in this case.

FIG. 4a shows a schematic diagram of an eye movement gaze real-time recognition system architecture using the instant eye movement gaze real-time recognition method according to a preferred embodiment of the present case.

FIG. 4b shows a schematic diagram of an eye movement gaze real-time recognition system architecture using the instant eye movement gaze real-time identification method according to another preferred embodiment of the present case.

FIG. 5 shows a schematic diagram of the preset screen of Experiment 3 in this case with six defined regions of interest.

Step a: Present a preset screen with at least one defined area of interest to a person with high prior knowledge on a display unit of an information processing device

Step b: Obtain the first eye movement data in response to the preset image by a sensor unit of an eye movement sensing module, and transmit the first eye movement data through a transmission unit of the eye movement sensing module Glance data to the information processing device

Step c: An operation unit of the information processing device executes an application program to perform a real-time operation on the first eye movement data to create an expert eye movement model, and the expert eye movement model includes at least one appropriate area of interest Describe the order of appropriate attention of one of the at least one appropriate area of interest and the percentage of appropriate attention time of one of the at least one appropriate area of interest

Step d: The expert eye movement model is stored in a first storage unit of the information processing device

Claims (10)

A real-time recognition method for eye movement gaze includes the following steps: presenting a preset screen with at least one defined area of interest to a person with high prior knowledge by a display unit of an information processing device; The measurement unit acquires first eye movement data in response to the preset screen by the person with high prior knowledge, and transmits the first eye movement data to the information processing device through a transmission unit of the eye movement sensing module; An operation unit of the information processing device executes an application program to perform an instant operation on the first eye movement data to create an expert eye movement model, and the expert eye movement model includes at least one appropriate attention area and the at least one appropriate attention The appropriate attention sequence of one of the regions and the appropriate attention time percentage of one of the at least one suitable attention region; and storing the expert eye movement model in a first storage unit of the information processing device; wherein the expert eye movement model is It is created based on real-time calculation of the first eye movement data of at least one person with advanced knowledge. The real-time recognition method for eye movement gaze as described in item 1 of the patent scope further includes a real-time feedback method for eye movement gaze, which includes the following steps: After storing the expert eye movement model, the display of the information processing device is used The unit presents the preset screen with at least one defined area of interest to a person with low prior knowledge; the person with low eye knowledge acquired by the sensing unit of the eye movement sensing module responds to the preset screen A second eye movement data, and transmits the second eye movement data to the information processing device through the transmission unit of the eye movement sensing module; the operation unit of the information processing device executes the application program to the second Perform an instant calculation on the eye movement data and perform an instant comparison with the expert eye movement model; and perform a corresponding eyeball on the low-preliminary knowledge by the display unit of the information processing device according to the instant comparison One of the functions of instant feedback and guidance. The real-time recognition method for eye movement gaze as described in item 1 or item 2 of the patent application scope, wherein the application can provide a preset screen with at least one defined area of interest to the at least one person with high prior knowledge or the at least one It is a function of those with low prior knowledge, and can simultaneously read and follow-up the first eye movement data or the second eye movement data during the test. The real-time recognition method for eye movement gaze as described in item 1 or item 2 of the patent scope, wherein the information processing device includes a desktop computer, a notebook computer or a handheld device; the transmission unit includes a wired transmission A unit or a wireless transmission unit; the eye movement sensing module includes a head-set type eye movement sensing module, a mobile eye movement sensing module or a telemetry type eye movement sensing module. The real-time recognition method for eye movement gaze as described in item 1 or item 2 of the patent application scope, wherein the preset image is selected from one of the group consisting of text, static image, dynamic image or video or combination. The real-time recognition method for eye-tracking gaze as described in item 2 of the patent application scope, wherein the real-time feedback and guidance function is selected from the group consisting of pauses or repeated playback of text, images, voices, and videos One or a combination of them for feedback and guidance. The real-time recognition method for eye movement gaze as described in item 1 or 2 of the patent application scope, wherein the eye movement sensing module further has a second storage unit and a warning unit, the second storage unit is used to store A preset threshold of loss rate, if one of the first eye movement data or the second eye movement data is greater than the preset threshold of loss rate, an alert function can be provided through the alert unit, and the first The storage unit and the second storage unit include a register, a cache memory, or a random access memory. According to the real-time recognition method for eye movement gaze as described in item 2 of the patent application scope, the information processing device further provides a user identification function to provide the user to log in and use the eye movement gaze real-time feedback method to adjust its attention habit. According to the real-time identification method for eye movement gaze described in item 8 of the patent application scope, the user's personal identification function includes a password protection method or a biometrics protection method. An instant recognition system for eye movement gaze, which has an information processing device and an eye movement sensing module, wherein the information processing device has an arithmetic unit and the arithmetic unit executes any one of items 1 to 9 including patent application The real-time identification method of the eye movement gaze can realize the real-time identification process of the one eye movement gaze.

Images