CN111685724B - Eye movement detection method, device, equipment and storage medium - Google Patents
Eye movement detection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN111685724B CN111685724B CN202010518480.6A CN202010518480A CN111685724B CN 111685724 B CN111685724 B CN 111685724B CN 202010518480 A CN202010518480 A CN 202010518480A CN 111685724 B CN111685724 B CN 111685724B
- Authority
- CN
- China
- Prior art keywords
- eye movement
- image data
- detected
- eye
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Heart & Thoracic Surgery (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Ophthalmology & Optometry (AREA)
- Biomedical Technology (AREA)
- Human Computer Interaction (AREA)
- Medical Informatics (AREA)
- Physics & Mathematics (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Eye Examination Apparatus (AREA)
Abstract
The embodiment of the invention discloses an eye movement detection method, which comprises the following steps: detecting an eye movement signal of a user to be detected through a DVS sensor, wherein the eye movement signal is a signal when the eyes of the user to be detected change; under the condition that an eye movement signal is detected, acquiring eye movement image data of a user to be detected, which is acquired by an image sensor, wherein the eye movement image data is image data of eyes of the user to be detected; and determining an eye movement vector according to the eye movement image data and/or the eye movement signal, wherein the eye movement vector represents the eye movement condition of the user to be tested. In addition, the embodiment of the invention also discloses an eye movement detection device, equipment and a storage medium. By adopting the invention, the accuracy of eye movement detection can be improved.
Description
Technical Field
The present invention relates to the field of eye movement devices, and in particular, to an eye movement detection method, an eye movement detection device, a computer device, and a computer readable storage medium.
Background
An eye movement device is a device for tracking and recording the eye track through machine vision and camera technology, is an important instrument for the study of psychology, advertising, industrial engineering and the like, and is also widely applied to eye movement control. The eye movement instrument is used for recording eyeball track characteristics of a person when the person processes visual information, and is widely used for research in the fields of attention, visual perception, reading and the like. In the transmitted eye movement device, the eye movement is captured mainly by an infrared camera or an RGB camera, but finer and faster movements cannot be detected due to lower frame rates of the infrared camera and the RGB camera. In addition, during detection, the infrared camera or the RGB camera needs to be in a normally-open state to capture an image, so that the power consumption of the infrared camera or the RGB camera is high. In addition, in the above-mentioned eye movement monitoring scheme based on the frame rate camera, a large amount of redundant data is generated, which causes waste of operation resources.
Disclosure of Invention
In view of the foregoing, it is desirable to provide an eye movement detection method, apparatus, computer device, and computer-readable storage medium.
An eye movement detection method, the method comprising:
detecting an eye movement signal of a user to be detected through a DVS sensor, wherein the eye movement signal is a signal when the eyes of the user to be detected change;
under the condition that an eye movement signal is detected, acquiring eye movement image data of a user to be detected, which is acquired by an image sensor, wherein the eye movement image data is image data of eyes of the user to be detected;
and determining an eye movement vector according to the eye movement image data and/or the eye movement signal, wherein the eye movement vector represents the eye movement condition of the user to be tested.
Before the step of acquiring the eye moving image data of the user to be detected, which is acquired by the image sensor, the method further comprises the following steps:
and generating a starting instruction to the image sensor to start the image sensor when the eye movement signal is detected.
Wherein the image sensor comprises an RGB camera unit and/or an infrared camera unit.
Wherein, under the condition that the eye movement signal is detected, generating a starting instruction to the image sensor so as to start the image sensor, and the method further comprises the following steps:
detecting the current ambient light intensity through a light intensity sensor;
generating a starting instruction to the RGB camera unit to start the RGB camera unit under the condition that the current ambient light intensity is larger than or equal to a preset light intensity threshold value;
and generating a starting instruction to the infrared camera unit under the condition that the current ambient light intensity is smaller than a preset light intensity threshold value so as to start the infrared camera unit.
The image sensor further comprises a depth sensor for detecting eye movement depth information of a user to be detected;
the step of determining an eye movement vector according to the eye movement image data and/or the eye movement signal further comprises:
the eye movement vector is determined according to the eye movement depth information and one or two of the eye movement image data and the eye movement signal.
The method comprises the steps of detecting an eye movement signal of a user to be detected through a DVS sensor, wherein the eye movement signal is a signal when the eyes of the user to be detected change, and the method further comprises the following steps:
detecting whether the local light intensity of the eyes of a user to be detected changes or not through the DVS sensor, collecting eye dynamic data of the local light intensity changes of the eyes, and generating eye movement signals according to the collected eye dynamic data.
Wherein the step of determining an eye movement vector according to the eye movement image data and/or the eye movement signal further comprises:
and calculating the eye movement vector according to the eye movement image data according to a preset eye movement vector calculation algorithm.
Wherein the step of determining an eye movement vector according to the eye movement image data and/or the eye movement signal further comprises:
in the case of detecting an eye movement signal, acquiring target image data corresponding to the time of the eye movement signal from among the eye movement image data;
generating target dynamic data containing image data according to the eye dynamic data and the target image data;
and analyzing the target dynamic data to determine the eye movement vector.
An eye movement detection device, the device comprising:
the eye movement signal detection module is used for detecting an eye movement signal of a user to be detected through the DVS sensor, wherein the eye movement signal is a signal when the eyes of the user to be detected change;
the image data acquisition module is used for acquiring eye moving image data of a user to be detected, which is acquired by the image sensor, under the condition that an eye moving signal is detected, wherein the eye moving image data is image data of eyes of the user to be detected;
and the eye movement vector generation module is used for determining an eye movement vector according to the eye movement image data and/or the eye movement signal, wherein the eye movement vector represents the eye movement condition of the user to be tested.
A computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of:
detecting an eye movement signal of a user to be detected through a DVS sensor, wherein the eye movement signal is a signal when the eyes of the user to be detected change;
under the condition that an eye movement signal is detected, acquiring eye movement image data of a user to be detected, which is acquired by an image sensor, wherein the eye movement image data is image data of eyes of the user to be detected;
and determining an eye movement vector according to the eye movement image data and/or the eye movement signal, wherein the eye movement vector represents the eye movement condition of the user to be tested.
A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of:
detecting an eye movement signal of a user to be detected through a DVS sensor, wherein the eye movement signal is a signal when the eyes of the user to be detected change;
under the condition that an eye movement signal is detected, acquiring eye movement image data of a user to be detected, which is acquired by an image sensor, wherein the eye movement image data is image data of eyes of the user to be detected;
and determining an eye movement vector according to the eye movement image data and/or the eye movement signal, wherein the eye movement vector represents the eye movement condition of the user to be tested.
The embodiment of the invention has the following beneficial effects:
after the eye movement detection method, the eye movement detection device, the computer equipment and the computer readable storage medium provided by the embodiment of the invention are adopted, when the eye movement of the user to be detected is monitored, whether the user to be detected generates eye movement or not is detected by the DVS sensor, a corresponding eye movement signal is detected, then under the condition that the eye movement signal is detected, the corresponding eye movement image data is acquired by the image sensor, and then the final eye movement vector is determined according to the eye movement image data and/or the eye movement signal, so as to determine the eye movement condition of the user to be detected. That is, compared with the related art in which the eye movement is monitored only by the image sensors such as the infrared camera and the RGB camera, the frame rate of the eye movement can be increased by the DVS sensor, and the accuracy of the eye movement monitoring can be improved.
Further, because the data collected by the DVS sensor is only for the object that is changed, the data of the object that is not changed is not collected, that is, redundant data corresponding to the object that is not changed can be greatly reduced in the process of performing eye movement monitoring, so that occupation of computer resources is reduced.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Wherein:
FIG. 1 is a diagram of an application environment of an eye movement detection method according to one embodiment;
FIG. 2 is a block diagram of an eye movement detection device according to one embodiment;
fig. 3 is a block diagram of a computer device for performing the eye movement detection method described above in one embodiment.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the embodiment of the invention, the eye movement detection method can detect the eye movement condition of a user based on the combination of the DVS sensor and other image sensors, so that the accuracy of eye movement detection can be improved, the power consumption can be reduced, and redundant data can be reduced.
As shown in fig. 1, in one embodiment, the operation of the eye movement detection method is based on a computer device for detecting eye movement of a user to be detected, where the computer device may be a tablet computer, a PC or other computer device, and the computer device is provided with a DVS sensor and an image sensor, and eye movement monitoring of the user is implemented through the DVS sensor and the image sensor.
Specifically, the eye movement detection method includes steps S102 to S106 shown in fig. 1:
step S102: and detecting an eye movement signal of a user to be detected through a DVS sensor, wherein the eye movement signal is a signal when the eyes of the user to be detected change.
DVS sensors (dynamic vision sensor, dynamic vision sensors) are motion sensitive sensors that capture only object boundary or contour events where there is relative motion and the illumination changes reach a certain threshold. Thus, only a small number of events are required to describe the scene content. Compared with a common image sensor, the content required to be processed by the DVS sensor is greatly reduced, so that the calculation cost can be greatly saved, and the calculation efficiency is improved.
The DVS sensor is disposed on the computer device or an eye movement monitoring device connected to the computer device, and can monitor eye movement of a user to be tested (for example, a user of a tablet computer) to detect whether the user to be tested generates eye movement. In specific implementation, whether the local light intensity change occurs in the eyes of the user to be detected is detected according to the DVS sensor, and under the condition that the local light intensity change occurs is detected, data corresponding to the local light intensity change of the eyes are acquired through the DVS sensor, namely corresponding eye dynamic data.
The data collected by the DVS sensor is 2D high-frame dynamic event data, that is, the frame rate of the DVS sensor is higher than that of a general image sensor, so that the frame rate of eye movement monitoring detection can be improved, and accordingly, fine, rapid and accurate eye movement monitoring can be detected, and the accuracy of eye movement monitoring can be improved. Moreover, the data collected by the DVS sensor is only for the object which is changed, and the data of the object which is not changed is not collected, namely, redundant data can be greatly reduced, and the occupation of computer resources is reduced.
Further, after the eye movement corresponding eye movement dynamic data is acquired through the DVS sensor, a corresponding eye movement signal can be generated according to the corresponding eye movement dynamic data, where the eye movement signal is a signal when the eyes of the user to be tested change, and is used for identifying the change of the eyes of the user to be tested. Furthermore, the eye movement signal may further include a specific change of the eye of the user to be detected, for example, data corresponding to the local eye light intensity change in the eye movement dynamic data, so as to understand a specific eye movement situation.
The technology of detecting the moving object by the DVS sensor is mature, and monitoring the eye movement by the technology corresponding to the DVS sensor may be implemented according to the DVS technology, and in this embodiment, detailed description will not be given.
Step S104: under the condition that an eye movement signal is detected, acquiring eye movement image data of a user to be detected, which is acquired by an image sensor, wherein the eye movement image data is image data of eyes of the user to be detected.
In this embodiment, the DVS sensor is normally open and is used to detect eye movement of the user to be detected in real time; alternatively, the DVS sensor is on when the computer device is in use (e.g., when the screen of a tablet is awake) and off when the computer device is off or standby. That is, in the case where the user to be measured uses the above-described computer device, the DVS sensor may monitor the eye movement of the user to be measured.
In this embodiment, when the DVS sensor does not detect the eye movement signal, it is indicated that the user to be tested does not generate eye movement, and analysis of the eye movement is not required; in the case that the DVS sensor detects an eye movement signal, it is indicated that the user to be tested generates eye movement, and further analysis of the eye movement is required.
Specifically, in the case where the DVS sensor detects an eye movement signal, it is further necessary to acquire corresponding eye movement image data by an image sensor also provided on the computer device, for example, by an infrared imaging unit or an RGB imaging unit to acquire image data corresponding to the eyes of the user to be measured as eye movement image data.
In this embodiment, the image sensor may be an RGB image capturing unit (i.e., an RGB sensor), an infrared image capturing unit (i.e., an infrared sensor), or both an RGB image capturing unit and an infrared image capturing unit. In other embodiments, the image sensor may also be other imaging units, without any limitation herein.
Under the condition that the eye movement of the user to be detected is monitored, the image sensor can be in a normally-on state, can also be in a standby or off state, and is awakened or turned on again under the condition that the eye movement signal is detected. The following description is made for two cases, respectively.
Case one: the DVS sensor and the image sensor are both in a normally open state. In the case that the DVS sensor detects an eye movement signal, determining a time corresponding to the detected eye movement signal; the image sensor acquires a corresponding image at the same time, and acquires an image corresponding to the time of the eye movement signal from the image acquired by the image sensor as the eye movement image data in step S104.
And a second case: the DVS sensor is in a normally open state, the image sensor is in a standby or off state, and is awakened or turned on again when an eye movement signal is detected. After the DVS sensor detects the eye movement signal, triggering and generating a starting instruction to the image sensor so as to start the image sensor, and then acquiring the eye movement image data corresponding to the eyes of the user to be detected through the started image sensor.
Further, in the case where the image sensor includes an RGB image capturing unit and an infrared image capturing unit, the RGB image capturing unit is suitable for use in a case where light is sufficient, and the infrared image capturing unit is suitable for use in a case where light is insufficient. Therefore, in the process of selecting the RGB image capturing unit or the infrared image capturing unit to acquire the eye moving image data, it is also necessary to determine according to the current illumination condition.
Specifically, the process of generating the start command to the image sensor to start the image sensor further includes the following steps:
detecting the current ambient light intensity through a light intensity sensor;
generating a starting instruction to the RGB camera unit to start the RGB camera unit under the condition that the current ambient light intensity is larger than or equal to a preset light intensity threshold value;
and generating a starting instruction to the infrared camera unit under the condition that the current ambient light intensity is smaller than a preset light intensity threshold value so as to start the infrared camera unit.
That is, the aforementioned computer device is further provided with a light intensity sensor, and in the process of performing eye movement monitoring, the current ambient light intensity is detected by the light intensity sensor, then the eye moving image data is collected by the RGB image capturing unit when the current ambient light intensity is greater than or equal to a preset light intensity threshold value, and the eye moving image data is collected by the infrared image capturing unit when the current ambient light intensity is less than the preset light intensity threshold value.
In addition, in the application scenario for the first case, the current ambient light intensity may be detected according to the light intensity sensor, and then whether the image sensor that is continuously turned on and detects the image data is an RGB image capturing unit or an infrared image capturing unit may be determined.
Step S106: and determining an eye movement vector according to the eye movement image data and/or the eye movement signal, wherein the eye movement vector represents the eye movement condition of the user to be tested.
After the eye movement signal is detected by the DVS sensor and the eye movement image data is acquired by the image sensor, a corresponding eye movement vector may be calculated from the eye movement image data and/or the eye movement signal. The eye movement vector represents the eye movement condition of the user to be tested, and also comprises data such as specific events, states and the like of the eye movement.
In this step, the corresponding eye movement vector may be calculated from only the eye movement image data detected by the image sensor. That is, according to a preset eye movement vector calculation algorithm, a corresponding eye movement vector is calculated from the eye moving image data.
In another embodiment, the calculation of the eye movement vector needs to take into account not only the eye movement image data detected by the image sensor but also the eye movement vector detected by the DVS sensor. That is, when the eye movement signal is detected, target image data corresponding to the time of the eye movement signal is acquired from among the eye movement image data; then generating target dynamic data containing image data according to the eye dynamic data and the target image data; and finally, analyzing the target dynamic data to determine the eye movement vector. That is, in this embodiment, dynamic event data of a 2D high frame acquired by the DVS sensor is combined with a 3D image acquired by the image sensor to obtain corresponding dynamic data with an image, and then the dynamic data is analyzed to obtain a corresponding eye movement vector.
In another embodiment, the image sensor may further include a depth sensor, or the image sensor may be replaced with a depth sensor. Wherein the depth sensor may be a TOF (Time of Flight) sensor. Specifically, the TOF sensor emits modulated near infrared light, reflects after encountering an object, and converts the distance of a photographed object by calculating a time difference or a phase difference between light emission and reflection, so as to generate depth information. That is, in this embodiment, the depth information corresponding to the eyes of the user to be tested, that is, the eye movement depth information, may be detected by the depth sensor, and whether or not eye movement and the specific situation of the eye movement are generated may be determined according to the eye movement depth information.
Further, in the above-described process of determining the eye movement vector, the eye movement vector may be determined based on the eye movement depth information and one or both of the eye movement image data and the eye movement signal. For example, an eye movement vector is determined from the eye movement depth information and the eye movement signal; for another example, an eye movement vector is determined from the eye movement depth information and the eye moving image data; for another example, the eye movement vector is determined based on the eye movement depth information, the eye movement image data, and the eye movement signal.
In the eye movement detection method, the DVS sensor is used for monitoring eye movement, the DVS sensor can be used for collecting higher frame data, finer dynamic data can be collected, and the eye movement state can be described more accurately, so that the accuracy of eye movement monitoring is improved.
Further, because the DVS sensor does not rely entirely on conventional eye movement measurement methods when detecting eye movement, some movement or oscillation of the head can be allowed without significantly affecting the measurement result; that is, in the present embodiment, the requirement for the head to be immovable for eye movement monitoring can be reduced by the DVS sensor, thereby improving the user experience.
Furthermore, in the scheme that whether the DVS sensor detects an eye movement signal or not and then the image sensor is started to detect image data under the condition that the DVS sensor detects the eye movement signal, because the data collected by the DVS sensor is only for the changed object and the data of the unchanged object are not collected, redundant data can be greatly reduced, and the corresponding image data can be continuously detected by the image sensor under the condition that the DVS sensor detects the eye movement signal, the frame rate requirement of the image sensor is reduced, the redundancy of the collected data of the image sensor is reduced, the occupation of computer resources is reduced, and the efficiency of eye movement monitoring is improved.
In another embodiment, an eye movement monitoring device is also presented.
Specifically, referring to fig. 2, fig. 2 shows a schematic structural diagram of the eye movement monitoring device.
Specifically, as shown in fig. 2, the eye movement monitoring device 10 includes:
the eye movement signal detection module 102 is configured to detect an eye movement signal of a user to be detected through a DVS sensor, where the eye movement signal is a signal when the eyes of the user to be detected change;
the image data acquisition module 104 is configured to acquire, when an eye movement signal is detected, eye movement image data of a user to be detected acquired by an image sensor, where the eye movement image data is image data of an eye of the user to be detected;
an eye movement vector generation module 106, configured to determine an eye movement vector according to the eye movement image data and/or the eye movement signal, where the eye movement vector represents an eye movement condition of the user to be tested.
In one embodiment, as shown in fig. 2, the eye movement monitoring apparatus 10 further includes an image sensor start module 108 for generating a start command to the image sensor to start the image sensor when the eye movement signal is detected.
In one embodiment, the image sensor comprises an RGB camera unit and/or an infrared camera unit.
In one embodiment, the image sensor activation module 108 is further configured to detect a current ambient light intensity via a light intensity sensor; generating a starting instruction to the RGB camera unit to start the RGB camera unit under the condition that the current ambient light intensity is larger than or equal to a preset light intensity threshold value; and generating a starting instruction to the infrared camera unit under the condition that the current ambient light intensity is smaller than a preset light intensity threshold value so as to start the infrared camera unit.
In one embodiment, the image sensor further comprises a depth sensor for detecting eye movement depth information of the user to be measured; the eye movement vector generation module 106 is further configured to determine the eye movement vector according to the eye movement depth information and one or both of the eye movement image data and the eye movement signal.
In one embodiment, the eye movement signal detection module 102 is further configured to detect whether the eye of the user to be tested has a local light intensity change through the DVS sensor, collect eye dynamic data of the eye with the local light intensity change, and generate an eye movement signal according to the collected eye dynamic data.
In one embodiment, the eye movement vector generation module 106 is further configured to calculate the eye movement vector according to the eye movement image data according to a preset eye movement vector calculation algorithm.
In one embodiment, the image data acquisition module 104 acquires target image data corresponding to the time of the eye movement signal in the eye movement image data in the case of detecting the eye movement signal; the eye movement vector generation module 106 is further configured to generate target dynamic data including image data according to the eye dynamic data and the target image data; and analyzing the target dynamic data to determine the eye movement vector.
FIG. 3 illustrates an internal block diagram of a computer device in one embodiment. The computer device may specifically be a terminal or a server. As shown in fig. 3, the computer device includes a processor, a memory, and a network interface connected by a system bus. The memory includes a nonvolatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program that, when executed by a processor, causes the processor to perform eye movement detection. The internal memory may also have stored therein a computer program which, when executed by the processor, causes the processor to perform the age identification method. It will be appreciated by those skilled in the art that the structure shown in fig. 3 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is presented comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of:
detecting an eye movement signal of a user to be detected through a DVS sensor, wherein the eye movement signal is a signal when the eyes of the user to be detected change;
under the condition that an eye movement signal is detected, acquiring eye movement image data of a user to be detected, which is acquired by an image sensor, wherein the eye movement image data is image data of eyes of the user to be detected;
and determining an eye movement vector according to the eye movement image data and/or the eye movement signal, wherein the eye movement vector represents the eye movement condition of the user to be tested.
In one embodiment, a computer-readable storage medium is provided, storing a computer program which, when executed by a processor, causes the processor to perform the steps of:
detecting an eye movement signal of a user to be detected through a DVS sensor, wherein the eye movement signal is a signal when the eyes of the user to be detected change;
under the condition that an eye movement signal is detected, acquiring eye movement image data of a user to be detected, which is acquired by an image sensor, wherein the eye movement image data is image data of eyes of the user to be detected;
and determining an eye movement vector according to the eye movement image data and/or the eye movement signal, wherein the eye movement vector represents the eye movement condition of the user to be tested.
The embodiment of the invention has the following beneficial effects:
after the eye movement detection method, the eye movement detection device, the computer equipment and the computer readable storage medium provided by the embodiment of the invention are adopted, when the eye movement of the user to be detected is monitored, whether the user to be detected generates eye movement or not is detected by the DVS sensor, a corresponding eye movement signal is detected, then under the condition that the eye movement signal is detected, the corresponding eye movement image data is acquired by the image sensor, and then the final eye movement vector is determined according to the eye movement image data and/or the eye movement signal, so as to determine the eye movement condition of the user to be detected. That is, compared with the related art in which the eye movement is monitored only by the image sensors such as the infrared camera and the RGB camera, the frame rate of the eye movement can be increased by the DVS sensor, and the accuracy of the eye movement monitoring can be improved.
Further, because the data collected by the DVS sensor is only for the object that is changed, the data of the object that is not changed is not collected, that is, redundant data corresponding to the object that is not changed can be greatly reduced in the process of performing eye movement monitoring, so that occupation of computer resources is reduced.
Those skilled in the art will appreciate that all or part of the processes in the methods of the above embodiments may be implemented by a computer program for instructing relevant hardware, where the program may be stored in a non-volatile computer readable storage medium, and where the program, when executed, may include processes in the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples only represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
Claims (9)
1. An eye movement detection method, the method comprising:
detecting an eye movement signal of a user to be detected through a DVS sensor, wherein the eye movement signal is a signal when the eyes of the user to be detected change; comprising the following steps: detecting whether the local light intensity of the eyes of a user to be detected changes or not through the DVS sensor, collecting eye dynamic data of the local light intensity changes of the eyes, and generating an eye movement signal according to the collected eye dynamic data;
under the condition that an eye movement signal is detected, acquiring eye movement image data of a user to be detected, which is acquired by an image sensor, wherein the eye movement image data is image data of eyes of the user to be detected;
determining an eye movement vector according to the eye movement image data and/or the eye movement signal, wherein the eye movement vector represents the eye movement condition of the user to be tested;
the step of determining an eye movement vector from the eye movement image data and/or the eye movement signal includes: in the case of detecting an eye movement signal, acquiring target image data corresponding to the time of the eye movement signal from among the eye movement image data; generating target dynamic data containing image data according to the eye dynamic data and the target image data; and analyzing the target dynamic data to determine the eye movement vector.
2. The eye movement detection method according to claim 1, wherein before the step of acquiring the eye moving image data of the user to be detected acquired by the image sensor, further comprising:
and generating a starting instruction to the image sensor to start the image sensor when the eye movement signal is detected.
3. The eye movement detection method according to claim 2, wherein the image sensor includes an RGB image capturing unit and/or an infrared image capturing unit.
4. The eye movement detection method according to claim 3, wherein the step of generating a start instruction to the image sensor to turn on the image sensor in the case where the eye movement signal is detected, further comprises:
detecting the current ambient light intensity through a light intensity sensor;
generating a starting instruction to the RGB camera unit to start the RGB camera unit under the condition that the current ambient light intensity is larger than or equal to a preset light intensity threshold value;
and generating a starting instruction to the infrared camera unit under the condition that the current ambient light intensity is smaller than a preset light intensity threshold value so as to start the infrared camera unit.
5. The eye movement detection method according to claim 3, wherein the image sensor further comprises a depth sensor for detecting eye movement depth information of the user to be detected;
the step of determining an eye movement vector according to the eye movement image data and/or the eye movement signal further comprises:
the eye movement vector is determined according to the eye movement depth information and one or two of the eye movement image data and the eye movement signal.
6. The eye movement detection method according to claim 1, wherein the step of determining an eye movement vector from the eye movement image data and/or eye movement signal further comprises:
and calculating the eye movement vector according to the eye movement image data according to a preset eye movement vector calculation algorithm.
7. An eye movement detection device, the device comprising:
the eye movement signal detection module is used for detecting an eye movement signal of a user to be detected through the DVS sensor, wherein the eye movement signal is a signal when the eyes of the user to be detected change; comprising the following steps: detecting whether the local light intensity of the eyes of a user to be detected changes or not through the DVS sensor, collecting eye dynamic data of the local light intensity changes of the eyes, and generating an eye movement signal according to the collected eye dynamic data;
the image data acquisition module is used for acquiring eye moving image data of a user to be detected, which is acquired by the image sensor, under the condition that an eye moving signal is detected, wherein the eye moving image data is image data of eyes of the user to be detected;
the eye movement vector generation module is used for determining an eye movement vector according to the eye movement image data and/or the eye movement signal, wherein the eye movement vector represents the eye movement condition of the user to be tested; the step of determining an eye movement vector from the eye movement image data and/or the eye movement signal includes: in the case of detecting an eye movement signal, acquiring target image data corresponding to the time of the eye movement signal from among the eye movement image data; generating target dynamic data containing image data according to the eye dynamic data and the target image data; and analyzing the target dynamic data to determine the eye movement vector.
8. A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of the method of any one of claims 1 to 6.
9. A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010518480.6A CN111685724B (en) | 2020-06-09 | 2020-06-09 | Eye movement detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010518480.6A CN111685724B (en) | 2020-06-09 | 2020-06-09 | Eye movement detection method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111685724A CN111685724A (en) | 2020-09-22 |
| CN111685724B true CN111685724B (en) | 2023-06-23 |
Family
ID=72479952
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010518480.6A Active CN111685724B (en) | 2020-06-09 | 2020-06-09 | Eye movement detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111685724B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116339510B (en) * | 2023-02-27 | 2023-10-20 | 荣耀终端有限公司 | Eye movement tracking method, eye movement tracking device, electronic equipment and computer readable storage medium |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6399766B2 (en) * | 2014-03-07 | 2018-10-03 | キヤノン株式会社 | IMAGING DEVICE, IMAGING DEVICE CONTROL METHOD, AND PROGRAM |
| KR102013935B1 (en) * | 2017-05-25 | 2019-08-23 | 삼성전자주식회사 | Method and system for detecting a dangerous situation |
| US10839547B2 (en) * | 2017-09-28 | 2020-11-17 | Samsung Electronics Co., Ltd. | Camera pose determination and tracking |
| US11202006B2 (en) * | 2018-05-18 | 2021-12-14 | Samsung Electronics Co., Ltd. | CMOS-assisted inside-out dynamic vision sensor tracking for low power mobile platforms |
| US10795435B2 (en) * | 2018-07-19 | 2020-10-06 | Samsung Electronics Co., Ltd. | System and method for hybrid eye tracker |
| CN109040711A (en) * | 2018-09-26 | 2018-12-18 | 信利光电股份有限公司 | A kind of image-pickup method, monitoring method and imaging sensor |
| CN111010560B (en) * | 2019-11-28 | 2022-03-01 | 青岛小鸟看看科技有限公司 | Anti-distortion adjusting method and device for head-mounted display equipment and virtual reality system |
-
2020
- 2020-06-09 CN CN202010518480.6A patent/CN111685724B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111685724A (en) | 2020-09-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109376667B (en) | Target detection method and device and electronic equipment | |
| CN102859565B (en) | Method and system for security system tampering detection | |
| KR102155182B1 (en) | Video recording method, server, system and storage medium | |
| CN108875837B (en) | Apparatus and method for testing target recognition device | |
| LU502731B1 (en) | Method for monitoring abnormality of power production, apparatus, computer device, and storage medium therefor | |
| CN111753649B (en) | Parking space detection method, device, computer equipment and storage medium | |
| CN111126153B (en) | Safety monitoring method, system, server and storage medium based on deep learning | |
| KR20160066927A (en) | Apparatus and method for supporting computer aided diagnosis | |
| US11501536B2 (en) | Image processing method, an image processing apparatus, and a surveillance system | |
| CN112001230A (en) | Sleeping behavior monitoring method and device, computer equipment and readable storage medium | |
| CN111685724B (en) | Eye movement detection method, device, equipment and storage medium | |
| WO2021023190A1 (en) | Proximity sensor-based noise floor adjustment processing method and apparatus, and computer device | |
| CN111260869A (en) | Method and device for extracting video frames in monitoring video and computer equipment | |
| CN112749655A (en) | Sight tracking method, sight tracking device, computer equipment and storage medium | |
| CN115190249B (en) | Camera infrared lamp dynamic light supplementing system and method | |
| US8605134B2 (en) | Video monitoring system and method | |
| US20230008356A1 (en) | Video processing apparatus, method and computer program | |
| CN116524539A (en) | Collaborative sensing method for interaction risk of signalers and drivers | |
| CN109598237A (en) | A kind of fatigue state detection method and device | |
| CN115050102A (en) | Mobile equipment standard sitting posture analysis processing method and system and storage medium thereof | |
| US20110234912A1 (en) | Image activity detection method and apparatus | |
| CN113129333B (en) | Multi-target real-time tracking method and system and electronic equipment thereof | |
| CN107886542B (en) | A kind of the target restoration methods and system of vision-based detection | |
| CN109885560B (en) | Cleaning method and device for infrared thermal image monitoring data of shale gas fracturing equipment | |
| KR102134771B1 (en) | Device and method for determining an emergency situation through object detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |