CN112006700B - Emotion recognition system and method for eye tracker - Google Patents
Emotion recognition system and method for eye tracker Download PDFInfo
- Publication number
- CN112006700B CN112006700B CN202011160004.8A CN202011160004A CN112006700B CN 112006700 B CN112006700 B CN 112006700B CN 202011160004 A CN202011160004 A CN 202011160004A CN 112006700 B CN112006700 B CN 112006700B
- Authority
- CN
- China
- Prior art keywords
- eye movement
- eye
- information
- emotion
- blink
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000008909 emotion recognition Effects 0.000 title claims abstract description 39
- 238000000034 method Methods 0.000 title claims abstract description 24
- 238000012544 monitoring process Methods 0.000 claims abstract description 112
- 230000004424 eye movement Effects 0.000 claims abstract description 103
- 230000008451 emotion Effects 0.000 claims abstract description 50
- 210000000744 eyelid Anatomy 0.000 claims abstract description 38
- 238000004458 analytical method Methods 0.000 claims abstract description 21
- 210000001747 pupil Anatomy 0.000 claims description 57
- 238000001514 detection method Methods 0.000 claims description 15
- 238000006073 displacement reaction Methods 0.000 claims description 9
- 230000001502 supplementing effect Effects 0.000 claims description 7
- 239000013589 supplement Substances 0.000 claims description 5
- 238000003384 imaging method Methods 0.000 claims description 4
- 230000000994 depressogenic effect Effects 0.000 claims description 2
- 230000000295 complement effect Effects 0.000 claims 1
- 210000001508 eye Anatomy 0.000 description 47
- 238000010586 diagram Methods 0.000 description 6
- 230000008921 facial expression Effects 0.000 description 4
- 238000011160 research Methods 0.000 description 3
- 230000000153 supplemental effect Effects 0.000 description 3
- 206010003805 Autism Diseases 0.000 description 2
- 208000020706 Autistic disease Diseases 0.000 description 2
- 206010049816 Muscle tightness Diseases 0.000 description 2
- 230000004397 blinking Effects 0.000 description 2
- 210000005252 bulbus oculi Anatomy 0.000 description 2
- 230000002996 emotional effect Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000003542 behavioural effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003340 mental effect Effects 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000009469 supplementation Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/163—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state by tracking eye movement, gaze, or pupil change
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Psychiatry (AREA)
- Pathology (AREA)
- Biomedical Technology (AREA)
- Hospice & Palliative Care (AREA)
- Psychology (AREA)
- Social Psychology (AREA)
- Physics & Mathematics (AREA)
- Developmental Disabilities (AREA)
- Biophysics (AREA)
- Child & Adolescent Psychology (AREA)
- Educational Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Eye Examination Apparatus (AREA)
Abstract
An emotion recognition system and method for an eye tracker comprise a picture playing system, an eye movement monitoring system and an emotion analysis system, can track eye movement states, perform comprehensive analysis by combining data such as eyelid and blink states, compare the data of each time to obtain weighted emotion data reference, and can be integrated on the eye tracker.
Description
Technical Field
The invention belongs to the technical field of eye movement instruments, and particularly relates to an emotion recognition system and method for an eye movement instrument.
Background
The eye tracker is an important instrument for basic research of psychology. The eye tracker is used for recording eye movement track characteristics of people when processing visual information, and is widely used for researches in the fields of attention, visual perception, reading and the like. The temporal and spatial features of eye movement are physiological and behavioral manifestations in the process of visual information extraction, which have a direct or indirect relationship with human mental activities, and are the reason why many psychologists are dedicated to eye movement research.
Most of existing equipment for emotion data analysis by combining with an eye tracker only records pupil movement data and is combined to be applied to autism, AR scene application and VR scene application.
For example, chinese patent document CN111339878A discloses a modified real-time emotion recognition method and system based on eye movement data, which collects eye movement data in real time in VR environment, i.e. can accurately and efficiently acquire rich eye movement information (eye movement time, eye movement direction, eye movement distance, gaze time, gaze frequency, pupil diameter size, and blink frequency) by tracking with an eye tracker, analyze the emotional state of the subject from the eye movement angle, and process facial expression feature vectors and eye movement feature vectors using SVR.
For example, chinese patent documents CN105868694A and CN109803109A disclose an emotion recognition scheme based on facial expression and eyeball motion, which utilizes an eye tracker to capture a focal position focused by eyes of a user, utilizes a facial expression recognition system to collect physiological signals of 8 collection positions of a human face, analyzes a muscle tension degree of each collection position according to the physiological signals of each collection position, and then combines the muscle tension degrees of the 8 collection positions to analyze and determine a current facial expression of the user through an analysis algorithm.
For example, chinese patent document CN110931111A discloses an autism assistance intervention system and method based on virtual reality and multi-modal information, which uses a built-in eye tracker to obtain eye movement data, calculates the direction and focus of eye movement, uses an extraluminal camera to photograph the lower half of the face to obtain muscle movement data of the chin and both cheeks, and performs emotion recognition through a speech feature extraction unit to obtain emotion scores of user interaction languages.
For example, chinese patent document CN111695442A discloses an online intelligent assistance system based on multi-mode fusion, in which an eye tracker detects the eye jump process of a user, extracts corresponding features from data collected by the eye tracker through a convolutional neural network, and determines the gas quality type through an SVM.
However, there is no emotion recognition solution for an eye tracker that not only tracks the eye movement state, but also performs comprehensive analysis in combination with data such as eyelid and blink states, and compares the eye movement state with the data of each time to obtain weighted emotion data reference.
Disclosure of Invention
In view of the technical problems in the prior art, an object of the present invention is to provide an emotion recognition system and method for an eye tracker, which can track the state of eye movement, perform comprehensive analysis by combining data such as eyelid and blink states, compare the data of each time to obtain weighted emotion data reference, and can be integrated into a solution of the eye tracker.
In order to achieve the purpose, the invention provides the following technical scheme: an emotion recognition system for an eye tracker, which comprises a picture playing system, an eye movement monitoring system and an emotion analysis system, the picture playing system is used for playing a user watching video picture, the eye movement monitoring system comprises a light supplementing system which is used as a light source to realize face light supplementation, a pupil tracking system which is combined with a camera device to track the position of a pupil and store tracking data, a pupil size monitoring system which monitors the size of the pupil, a blink monitoring system which monitors the blink state and an eyelid relaxation monitoring system which monitors the eyelid relaxation degree, the emotion analysis system comprises an eye movement information base system, an eye movement information comparison system for comprehensively comparing the data in the eye movement information base system and the eye movement monitoring system, and further comprises an emotion information base system and an emotion information comparison system for comprehensively comparing the data in the emotion information base system and the eye movement monitoring system.
Preferably, the pupil tracking system includes a track imaging system for recording a pupil movement track, an eye jump frequency monitoring system for monitoring a pupil jump frequency, an eye jump distance monitoring system for monitoring and recording a pupil jump distance in combination with the eye jump frequency monitoring system, and a displacement prediction system for predicting the displacement of the pupil before opening the eye in combination with the blink monitoring system and the eyelid relaxation monitoring system.
Preferably, the blink monitoring system is used for blink detection, wherein the blink monitoring system comprises a blink speed monitoring system for monitoring blink speed, a blink frequency monitoring system for monitoring blink frequency and a blink time recording system for monitoring blink time, the blink speed monitoring system and the blink frequency monitoring system are combined to record blink speed data, the blink speed and the blink frequency are measured in the same time length, the blink speed detection is averaged to be a, the blink speed exceeding A is recorded to be Aa, and the blink speed lower than A is recorded to be Ab. The blink frequency detection is averaged to obtain a mean value V, and blink frequencies above V are recorded as Va, and blink frequencies below V are recorded as Vb. Generally speaking, a fast blink speed indicates that the user is in a good active state, the follow-up picture is compact, the tension is high, the attention is relatively concentrated, and a fast blink frequency indicates that the user has high interest in the picture and the follow-up is good.
Preferably, the eyelid laxity monitoring system is used for eyelid laxity detection, wherein detection results of the eyelid laxity monitoring system are distinguished by the degree of coverage of the iris, including an uncovered state B, a partial coverage Ba, a half coverage Bb and a majority coverage Bc. By recording the degree of coverage of the iris, the eyelid laxity of the user can be judged. The greater the coverage the higher the sag.
According to another aspect of the present invention, the present invention further provides an emotion recognition method for an eye tracker, comprising the steps of:
s1, starting a picture playing system and starting a light supplementing system to supplement light on the surface of the human face, and acquiring eye movement information within a certain time through an eye movement monitoring system within a light ray range meeting the identification requirement;
s2, comparing the eye movement information obtained in the S1 with the eye movement information database system and the eye movement information comparison system to obtain basic comparison data;
and S3, comparing the basic comparison data obtained in the S2 with the emotion information database system and the emotion information comparison system, performing weight analysis with different emotion data, and obtaining analysis results with different weights.
Preferably, the data comparison content in S2 includes pupil trajectory comparison, eye jump frequency and eye jump distance in combination with a pupil tracking system, and further includes pupil size comparison in combination with a pupil size monitoring system, blink speed comparison and blink frequency comparison in combination with a blink monitoring system, and eyelid sag comparison in combination with an eyelid sag monitoring system.
According to the emotion recognition system and method for the eye movement instrument, due to the configuration of the light supplementing system, light can be supplemented to the face under the dark light condition, and the eye movement monitoring system can accurately capture eye movement data.
According to the emotion recognition system and method for the eye movement instrument, the pupil tracking system records eye movement data, and the displacement prediction system can predict and capture the approximate position of a pupil in advance in a blinking state, so that the pupil position can be conveniently and quickly positioned when an eyelid is opened; the eye movement information base system in the emotion analysis system is combined with the eye movement information comparison system to compare the eye movement information, can store the current eye movement information and facilitate comparison in the follow-up eye movement information collection process, has the function of increasing comparison information, and realizes data learning.
According to the emotion recognition system and method for the eye tracker, the emotion information base system is arranged to be combined with the emotion information comparison system, so that emotion weighting of current eye movement information can be combined, and a weight proportion is provided for reference.
According to the emotion recognition system and method for the eye tracker, the technical scheme that the emotion recognition system and the eye tracker are integrated into a whole can be realized no matter the eye tracker is an external eye tracker or an internal eye tracker, and the intelligence of the eye tracker is improved.
Drawings
FIG. 1 is a schematic block diagram illustrating an emotion recognition system for an eye tracker in accordance with an embodiment of the present invention;
FIG. 2 is a schematic block diagram illustrating a pupil tracking system for an emotion recognition system for an eye tracker in accordance with an embodiment of the present invention;
fig. 3 is a schematic block diagram illustrating a blink monitoring system for an emotion recognition system of an eye tracker according to an embodiment of the present invention.
The reference numbers in the figures are respectively: the system comprises a picture playing system 1, an eye movement monitoring system 2, an emotion analysis system 3, a light supplementing system 4, a pupil tracking system 5, a track imaging system 501, an eye jump frequency monitoring system 502, an eye jump distance monitoring system 503, a displacement prediction system 504, a pupil size monitoring system 6, a blink monitoring system 7, a blink speed monitoring system 701, a blink frequency monitoring system 702, a blink time recording system 703, an eyelid looseness monitoring system 8, an eye movement information base system 9, an eye movement information comparison system 10, an emotion information base system 11 and an emotion information comparison system 12.
Detailed Description
The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which embodiments of the invention are shown and described, and it is to be understood by those skilled in the art that the foregoing detailed description is illustrative, and that the invention is not limited to the specific embodiments shown.
Unless otherwise indicated, all references to up, down, left, right, front, back, inner and outer directions herein are to be interpreted as referring to up, down, left, right, front, back, inner and outer directions in the drawings to which the invention is directed.
FIG. 1 is a schematic block diagram of an emotion recognition system for an eye tracker; FIG. 2 is a schematic block diagram of a pupil tracking system; fig. 3 is a schematic block diagram of a blink monitoring system.
As shown in fig. 1, in an embodiment of the present invention, an emotion recognition system for an eye tracker is provided, which includes a picture playing system 1, an eye monitoring system 2, and an emotion analysis system 3, where the picture playing system 1 is configured to play a video picture watched by a user, and guide the eyeball of the user to watch the picture content interested by the user through the picture playing system. Eye movement monitoring system 2 includes and realizes facial light supplemental light system 4 as the light source, through the configuration of light supplemental light system 4, can guarantee to carry out the light supplement to face under the dim light condition, and to the occasion of seeing the shadow, light often appears inadequately, and receives the influence of broadcast picture content and the change in luminance is big, and the accuracy that detecting system detected is ensured to the light supplement through this light supplemental light system 4 this moment. The eye movement monitoring system 2 further comprises a pupil tracking system 5, the pupil tracking system 5 captures eye movement data, and can track the position of a pupil by combining with an image pickup device and store the tracking data in a memory, the eye movement monitoring system 2 further comprises a pupil size monitoring system 6 for monitoring the size of the pupil, the pupil size monitoring system 6 detects pupil size data and data of pupil size change, in a preferred embodiment, the pupil size monitoring system 6 also detects the size and the changed data of an iris and stores the data in the memory, and the eye movement monitoring system 2 further comprises a blink monitoring system 7 for monitoring a blink state and an eyelid looseness monitoring system 8 for monitoring the eyelid looseness degree. The blink monitoring system detects a blink state. The eyelid laxity monitoring system 8 is used for eyelid laxity detection, and the detection result is distinguished by the degree of coverage of the iris, including an uncovered state B, a partial coverage Ba, a half coverage Bb and a large coverage Bc. Preferably, the eyelid laxity monitoring system is used for eyelid laxity detection, wherein detection results of the eyelid laxity monitoring system are distinguished by the degree of coverage of the iris, including an uncovered state B, a partial coverage Ba, a half coverage Bb and a majority coverage Bc. By recording the degree of coverage of the iris, the eyelid laxity of the user can be judged. The greater the coverage the higher the sag.
In an embodiment of the present invention, the emotion analyzing system 3 of the emotion recognition system includes an eye movement information base system 9, and an eye movement information comparing system 10 for performing comprehensive comparison by combining data in the eye movement information base system 9 and the eye movement monitoring system 2, where the eye movement information base system 9 in the emotion analyzing system 3 compares eye movement information by combining the eye movement information comparing system 10, and can store current eye movement information to facilitate comparison in subsequent eye movement information acquisition, and has a function of increasing comparison information, thereby implementing data learning. The emotion analysis system 3 further comprises an emotion information base system 11 and an emotion information comparison system 12 which is used for performing comprehensive comparison by combining data in the emotion information base system 11 and the eye movement monitoring system 2, wherein the emotion information base system 11 is used for achieving emotion weighting of composite current eye movement information by combining the emotion information comparison system 12, and weight proportion is provided for reference.
As shown in fig. 2, the pupil tracking system 5 includes a track imaging system 501 for recording a pupil movement track, an eye jump frequency monitoring system 502 for monitoring a pupil jump frequency, an eye jump distance monitoring system 503 for monitoring and recording a pupil jump distance in combination with the eye jump frequency monitoring system 502, and a displacement prediction system 504 for predicting a pupil displacement before opening eyes in combination with the blink monitoring system 7 and the eyelid sag monitoring system 8, where the pupil tracking system 5 records eye movement data, and the displacement prediction system 504 can predict and capture an approximate position of a pupil in advance in a blink state, so as to facilitate rapid positioning of the pupil position when an eyelid is opened.
As shown in fig. 3, the blink monitoring system 7 is configured for blink detection, wherein the blink monitoring system 7 comprises a blink rate monitoring system 701 configured for monitoring blink rate, a blink rate monitoring system 702 configured for monitoring blink frequency and a blink time recording system 703 configured for monitoring blink time, wherein the blink rate monitoring system 701 and the blink rate monitoring system 702 are combined to perform blink rate data recording, and detection of which is averaged to be a, blink rates above a are recorded as Aa, and blink rates below a are recorded as Ab. The blink rate and blink frequency are measured over the same time span, and the blink rate measurements are averaged as A, with blink rates above A being recorded as Aa and blink rates below A being recorded as Ab. The blink frequency detection is averaged to obtain a mean value V, and blink frequencies above V are recorded as Va, and blink frequencies below V are recorded as Vb. Through the detection and tracking of the blink speed and the blink frequency, a computable value is given to the thinking activity state, the tensity, the interest degree of the user in the picture, the attention concentration degree and the following performance of the user, so that a quantitative index is given to the blink to provide a quantitative reference value on the emotional change of the user. Meanwhile, the method can also be used as a feature map in later period such as the neural convolution calculation.
While the emotion recognition system for an eye tracker of the present invention has been described in detail with reference to the preferred embodiments thereof, in another aspect, the present invention provides a method for emotion recognition using the emotion recognition system. This is described in detail below in conjunction with the emotion recognition system of FIGS. 1-3.
According to another embodiment of the invention, the emotion recognition method for the eye tracker comprises the following steps:
s1, starting the picture playing system 1 and starting the light supplementing system 4 to supplement light on the surface of the human face, and acquiring eye movement information within a certain time through the eye movement monitoring system 2 within a light ray range meeting the identification requirement; the eye movement information comprises five major information, namely pupil information, blink information, eyelid information, eye jump information, fixation information and the like. In a preferred embodiment, the eye movement information further comprises iris information.
S2, comparing the eye movement information obtained in the S1 with the eye movement information database system 9 and the eye movement information comparison system 10 to obtain basic comparison data; the eye movement information acquired in S1 is represented quantitatively by using the above five pieces of information about the user' S emotion change, and the data is compared with the eye movement reference information stored in the eye movement information library system 9 to form comparison data in the eye movement information comparison system 10 as basic comparison data, thereby visualizing the eye movement information. In this embodiment, the data alignment content in S2 includes pupil trajectory alignment, eye jump frequency and eye jump distance in combination with the pupil tracking system 5, and further includes pupil size alignment in combination with the pupil size monitoring system 6, blink speed alignment and blink frequency alignment in combination with the blink monitoring system 7, and eyelid sag alignment in combination with the eyelid sag monitoring system 8.
And S3, comparing the basic comparison data obtained in the S2 with the emotion information database system 11 and the emotion information comparison system 12, performing weight analysis with different emotion data, and obtaining analysis results with different weights. When the user is happy, the blinking speed becomes fast, the attention is focused, the pupil is bright and the tension of the eyelid is high, and optimistic weighting is obtained, and on the contrary, when the user is depressed, the eyelid is heavy, the eyes are not bright, the attention is distracted, and pessimistic weighting is obtained. By weighting in this manner, emotion is quantized to realize emotion recognition.
The present invention has been described in detail with reference to the specific embodiments, and it will be understood by those skilled in the art that various changes in the embodiments and/or equivalent arrangements of parts of the features of the embodiments may be made without departing from the spirit and scope of the invention, which is defined by the appended claims.
Claims (7)
1. An emotion recognition system for an eye tracker, which can be integrated with the eye tracker, comprising a picture playback system (1), an eye monitoring system (2) and an emotion analysis system (3), characterized in that: the eye movement monitoring system (2) comprises a light supplementing system (4) used as a light source to realize face light complement, a pupil tracking system (5) combined with a camera device to track pupil positions and store tracking data, a pupil size monitoring system (6) used for monitoring pupil sizes, a blink monitoring system (7) used for monitoring blink states, an eyelid looseness monitoring system (8) used for monitoring eyelid looseness, an emotion analysis system (3) comprises an eye movement information base system (9), an eye movement information comparison system (10) used for comprehensively comparing data in the eye movement information base system (9) and the eye movement monitoring system (2), an emotion information base system (11) and an emotion information comparison system (12) used for comprehensively comparing data in the emotion information base system (11) and the eye movement monitoring system (2), and the pupil tracking system (5) comprises a track imaging system (501) used for recording pupil movement tracks, The eye movement monitoring system comprises an eye movement frequency monitoring system (502) for monitoring the eye movement frequency, an eye movement distance monitoring system (503) for monitoring and recording the eye movement distance by combining the eye movement frequency monitoring system (502), and a displacement prediction system (504) for predicting the displacement of the pupil before the eye is opened by combining a blink monitoring system (7) and an eyelid relaxation monitoring system (8), wherein the eyelid relaxation monitoring system (8) is used for detecting the eyelid relaxation, and the detection result of the eyelid relaxation monitoring system (8) is distinguished by the degree of covering the iris and comprises an uncovering state B, a small part covering Ba, a half covering Bb and a large part covering Bc;
the eye movement monitoring system (2) acquires eye movement information within a certain time, the eye movement information comprises five pieces of information, namely pupil information, blink information, eyelid information, eye jump information and fixation information, and the acquired eye movement information containing the five pieces of information is combined with the eye movement information base system (9) and the eye movement information comparison system (10) to carry out data comparison to obtain basic comparison data; and comparing the obtained basic comparison data with the emotion information database system (11) and the emotion information comparison system (12), performing weight analysis with different emotion data to obtain analysis results with different weights, and quantifying the emotion to realize emotion recognition.
2. An emotion recognition system for an eye tracker in accordance with claim 1, wherein: the blink monitoring system (7) is for blink detection, wherein the blink monitoring system (7) comprises a blink rate monitoring system (701) for monitoring a blink rate, a blink frequency monitoring system (702) for monitoring a blink frequency and a blink time recording system (703) for monitoring a blink time.
3. An emotion recognition method for an eye tracker, which performs emotion recognition using the emotion recognition system for an eye tracker according to any one of claims 1 to 2, comprising the steps of:
s1, starting the picture playing system (1) and starting the light supplementing system (4) to supplement light on the surface of the human face, and acquiring eye movement information within a certain time through the eye movement monitoring system (2) within a light ray range meeting the identification requirement;
s2, comparing the eye movement information obtained in the S1 with the eye movement information database system (9) and the eye movement information comparison system (10) to obtain basic comparison data;
and S3, comparing the basic comparison data obtained in the S2 with the emotion information database system (11) and the emotion information comparison system (12), and performing weight analysis with different emotion data to obtain analysis results with different weights.
4. The emotion recognition method for an eye tracker according to claim 3, wherein: the data comparison content in the step S2 includes pupil trajectory comparison, eye jump frequency and eye jump distance in combination with the pupil tracking system (5), pupil size comparison in combination with the pupil size monitoring system (6), blink speed comparison and blink frequency comparison in combination with the blink monitoring system (7), and eyelid sag comparison in combination with the eyelid sag monitoring system (8).
5. The emotion recognition method for an eye tracker according to claim 4, wherein: the eye movement information also includes iris information.
6. The emotion recognition method for an eye tracker according to claim 5, wherein:
the eye movement information acquired in the step S1 is quantitatively characterized by using the above five pieces of information for the user emotion change, and the data is compared with the eye movement reference information stored in the eye movement information base system (9), so that comparison data is formed in the eye movement information comparison system (10) as basic comparison data.
7. The emotion recognition method for an eye tracker according to claim 4, wherein: and obtaining optimistic weighting when the user is happy, obtaining pessimistic weighting when the user is depressed, and quantifying the emotion to realize emotion recognition by weighting.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011160004.8A CN112006700B (en) | 2020-10-27 | 2020-10-27 | Emotion recognition system and method for eye tracker |
| PCT/CN2020/124841 WO2022087965A1 (en) | 2020-10-27 | 2020-10-29 | Emotion recognition system and method for use in eye tracker |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011160004.8A CN112006700B (en) | 2020-10-27 | 2020-10-27 | Emotion recognition system and method for eye tracker |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112006700A CN112006700A (en) | 2020-12-01 |
| CN112006700B true CN112006700B (en) | 2021-07-16 |
Family
ID=73528277
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011160004.8A Active CN112006700B (en) | 2020-10-27 | 2020-10-27 | Emotion recognition system and method for eye tracker |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112006700B (en) |
| WO (1) | WO2022087965A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113827238A (en) * | 2021-09-02 | 2021-12-24 | 苏州中科先进技术研究院有限公司 | Emotion evaluation method and device based on virtual reality and eye movement information |
| CN115439921A (en) * | 2022-09-22 | 2022-12-06 | 徐州华讯科技有限公司 | Image preference prediction method based on eye diagram reasoning |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100010317A1 (en) * | 2008-07-09 | 2010-01-14 | De Lemos Jakob | Self-contained data collection system for emotional response testing |
| CN106570496B (en) * | 2016-11-22 | 2019-10-01 | 上海智臻智能网络科技股份有限公司 | Emotion identification method and apparatus and intelligent interactive method and equipment |
| CN107463874A (en) * | 2017-07-03 | 2017-12-12 | 华南师范大学 | The intelligent safeguard system of Emotion identification method and system and application this method |
| CN108888281A (en) * | 2018-08-16 | 2018-11-27 | 华南理工大学 | State of mind appraisal procedure, equipment and system |
| CN109376621A (en) * | 2018-09-30 | 2019-02-22 | 北京七鑫易维信息技术有限公司 | A kind of sample data generation method, device and robot |
| CN109620262B (en) * | 2018-12-12 | 2020-12-22 | 华南理工大学 | Emotion recognition system and method based on wearable bracelet |
-
2020
- 2020-10-27 CN CN202011160004.8A patent/CN112006700B/en active Active
- 2020-10-29 WO PCT/CN2020/124841 patent/WO2022087965A1/en active Application Filing
Non-Patent Citations (2)
| Title |
|---|
| "人眼检测及动态跟踪研究综述";饶瑞琦 等;《电子世界》;20171123;全文 * |
| 《粒子滤波和卡尔曼滤波组合的瞳孔跟踪方法》;王长元;《计算机与数学工程》;20180420;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022087965A1 (en) | 2022-05-05 |
| CN112006700A (en) | 2020-12-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Itti | Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes | |
| US8808195B2 (en) | Eye-tracking method and system for screening human diseases | |
| KR102277820B1 (en) | The psychological counseling system and the method thereof using the feeling information and response information | |
| Vortmann et al. | EEG-based classification of internally-and externally-directed attention in an augmented reality paradigm | |
| CN112006700B (en) | Emotion recognition system and method for eye tracker | |
| Galdi et al. | Eye movement analysis for human authentication: a critical survey | |
| WO2017221555A1 (en) | Engagement value processing system and engagement value processing device | |
| CN111326253A (en) | Method for evaluating multi-modal emotional cognitive ability of patients with autism spectrum disorder | |
| Haque et al. | Deep convolutional neural networks and transfer learning for measuring cognitive impairment using eye-tracking in a distributed tablet-based environment | |
| US11328533B1 (en) | System, method and apparatus for detecting facial expression for motion capture | |
| Nie et al. | SPIDERS: Low-cost wireless glasses for continuous in-situ bio-signal acquisition and emotion recognition | |
| Speth et al. | Deception detection and remote physiological monitoring: A dataset and baseline experimental results | |
| CN109255309A (en) | Brain electricity and eye movement fusion method and device towards Remote Sensing Target detection | |
| CN114209324A (en) | Psychological assessment data acquisition method based on image visual cognition and VR system | |
| KR20140040474A (en) | Apparatus, method and system for video contents summarization | |
| Bruckert et al. | Where to look at the movies: Analyzing visual attention to understand movie editing | |
| Zhang et al. | Research on Dyslexia Detection based on Eye Tracking | |
| Alzahrani et al. | Eye blink rate based detection of cognitive impairment using in-the-wild data | |
| CN115429271A (en) | Autism spectrum disorder screening system and method based on eye movement and facial expression | |
| CN115439920A (en) | Consciousness state detection system and equipment based on emotional audio-visual stimulation and facial expression | |
| Wang et al. | Mgeed: a multimodal genuine emotion and expression detection database | |
| Stöckli et al. | A practical guide for automated facial emotion classification 1 | |
| KR20220158957A (en) | System for prediction personal propensity using eye tracking and real-time facial expression analysis and method thereof | |
| Wang et al. | A composite measure for the evaluation of mental workload | |
| Johari et al. | Insights Into Student Attention During Online Lectures: A Classification Approach Using Eye Data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210609 Address after: No. 2025, chengluo Avenue, Longquanyi District, Chengdu, Sichuan 610106 Applicant after: CHENGDU University Applicant after: Runtou Intelligent Science and Technology Research Institute Jiangsu Co.,Ltd. Applicant after: Hangzhou Heshi Medical Technology Co.,Ltd. Address before: No.2119, international enterprise headquarters, block B4, Jiulonghu District, Nanjing City, Jiangsu Province Applicant before: Runtou Intelligent Science and Technology Research Institute Jiangsu Co.,Ltd. Applicant before: Bellier technology development Nanjing Co.,Ltd. Applicant before: Hangzhou Heshi Medical Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |