CN114788701A - Eye movement detection and analysis system based on multichannel array electro-ocular electrodes - Google Patents

Abstract

The invention belongs to the technical field of human body action monitoring, and particularly relates to an eye movement detection and analysis system based on a multi-channel array eye electrode. The invention comprises an acquisition module, a lower computer and an upper computer; the two carry out data transmission in a wired or wireless mode; the acquisition module and the lower computer comprise front-end acquisition equipment and a signal processing hardware circuit; the upper computer is a client computer loaded with a signal analysis and information display program; and the client receives the information collected by the lower computer and displays the analysis result in a user graphical interface. The acquisition equipment is provided with a multi-channel array electro-ocular electrode for acquiring eye movement information and related physiological information, and the acquired multi-source multi-dimensional physiological signals are processed through a signal processing and analyzing method and an intelligent algorithm, so that the eye movement condition with micro expression is effectively detected. The acquisition module and the lower computer are arranged in the wearable head belt type structure; the system of the invention has richer information for analyzing the eye movement event and more comfortable user experience.

Description

Eye movement detection and analysis system based on multichannel array electro-ocular electrodes

Technical Field

The invention belongs to the technical field of human body motion monitoring, and particularly relates to an eye movement monitoring and analyzing system.

Background

Vision is one of the important perception pathways for animals. As a receptor of vision, eye movement controls human acceptance of visual information. Physiologically, the movement of each eyeball is controlled by 3 groups of antagonistic muscles, which are controlled by different nuclei in the brainstem, and the resulting different combinations of relaxation control 10 different basic movements of our eyes. Since eye movements are innervated by cerebral nerves, the manner of eye movement may also reflect some neurological health aspects to some extent, such as involuntary tremor of the eye caused by abnormalities in the nerve conduction mechanism. In addition, the daily eye movements reflect the cognitive conditions, such as attention, regions of interest, or physiological conditions, such as fatigue.

The eye movement category can be roughly classified into 5 types. Gaze (gaze) where the line of sight stays briefly at a certain point, saccade (saccade) where the line of sight jumps rapidly from one point to another, saccade (saccade) where the line of sight follows a slowly moving point, nystagmus (nytagmus) where the line of sight oscillates back and forth within a certain small range, and divergent eye movements (vergence) where the line of sight moves in different directions. Most of the human eye movements can be decomposed into a combination of these 5 eye movements. Knowing the movement of the eyes when these movements and their combinations occur can provide a data base for understanding and studying the cognitive patterns of the human. In research, such as studies of eye movement to explore the attention of a human, and studies of estimating a human region of interest by gaze stay are subjects that are often sought in cognitive science.

Therefore, the eyeball motion related information can be used in various fields, for example, human-computer interaction can be enriched by taking the human eyeball motion as an interaction means, and the efficiency of people is improved. And then, for example, the mode of the eyeball motion is acquired and analyzed to know the mode of information processing of the person, the region of interest and the like, and objective data is provided for the investigation of the user. Or from the physiological perspective, the eyeball movement characteristics of people under different conditions or different physiological characteristics of people during eyeball movement are known, and theoretical basis and research data are provided for health assessment, biological information acquisition and the like.

In summary, it is of great significance to the acquisition of eye movement data from the perspective of both practical application and theoretical research, as well as from the perspective of both physiological exploration and engineering implementation.

The research on the method for acquiring the eye movement information has been over 50 years. The most representative of these methods are iris coil method, iris reflex method, video method and electrooculography.

From the signal source of the detection method, the method can be roughly divided into two technical routes of acquiring peripheral signals to reflect the eye movement condition and directly acquiring physiological electric signals generated by a human body during the eye movement to estimate the eye movement condition. The essence of the two technical ways is different in that the acquired original data of the latter is an electric signal generated by a human body and has certain physiological significance, while the acquired original data of the former is not from the human body.

Typical techniques for reflecting eye movements by collecting peripheral signals include an iris coil method, an iris reflex method, and a video method. These methods are mainly to generate a marker on an eyeball by an apparatus and then detect a change in the marker when eye movement occurs to indirectly estimate the condition of the eye movement, for example, an iris coil method requires a contact lens with a coil to be worn on the cornea of a subject, and a video method requires an optical camera to capture an optical image of a pupil to be tested. These methods have the advantage of higher accuracy and sensitivity, but they do not meet the requirements of daily eye movement detection for portability, comfort, ease of use, and long service time due to various drawbacks, such as higher configuration difficulty, poor wearing comfort, larger resource occupation for data format processing, poor interference resistance, etc. Therefore, products developed based on the above-mentioned techniques are only suitable for laboratory environments with controlled conditions, and still cannot meet the requirement for acquiring eye movement data in daily life.

An electro-oculography (hereinafter referred to as electro-oculography) is a method for detecting a change of an electric field formed around an eyeball by a physiological current between a retina and a cornea during movement of the eyeball, and is an eye movement detection technology based on a physiological electrical signal, which is widely recognized clinically at present. Compared with other technical schemes for detecting eye movement by using peripheral signals, the technical scheme for acquiring the electro-oculogram information by the electro-oculogram has the advantages that 1) from the acquisition angle, the signal source is an electric signal generated by a human body, and the main interference source is other physiological signals such as myoelectric signals and motion artifacts generated by a detected person. Compared with the iris coil based on magnetic field information and the video method based on video information, the method is less influenced by other factors in the environment and has better stability. 2) From the perspective of signal transmission and processing, because the eye electrical signals are physiological electrical signals, compared with image signals of a video method and an iris reflection method, the data volume is extremely small, so that the resources required for processing the eye electrical signals are far less than those of the video method, and the acquisition time can be longer than that of the video method and the like. 3) From the configuration of the acquisition environment, the electrooculography requires that a plurality of electrodes be respectively placed on the epidermis around the eye orbit to acquire the electrooculogram signals. Compared with other configurations, the configuration is simpler and easier to operate, the signal is relatively more stable, the limitation on the detected person is relatively smaller, and therefore the use experience is relatively better. 4) There is no limitation on whether the eyeball is visible or not, and the movement of the eyeball when the detected person closes the eye can be detected.

The existing detection method based on the electro-oculogram signal has various electrode arrangements, and a standard lead widely accepted is shown in fig. 14. In addition, consumer-grade consumer products and research based on electro-ocular electrodes have proposed different electro-ocular electrode arrangements, such as those proposed by JINs corporation, netherlands IMEC, which are convenient to mount on the spectacle frame.

However, the existing electro-oculography method still adopts the classic electro-oculography data acquisition lead mode proposed in the last 60 th century, or the simple electrode number adjustment or electrode position adjustment based on the electro-oculography method. The fact that the change is not obtained is still that the electric potential change brought by the electric field change accompanying the eyeball rotation in a certain local part around the electrode is obtained through a single electrode, then the electric potential change of different local parts acquired by a plurality of electrodes distributed around the eye orbit is acquired and analyzed, the eyeball movement condition is inferred, and finally the movement of the sight line is inferred.

The problems with such a method are:

1. two or more electrodes are required, and accurate multi-directional eye movement detection cannot be performed by an eye electrical signal obtained by only one electrode. The reason for this problem is mainly two aspects, 1), the traditional electrode design makes the extraction mode of the clean electro-oculogram signal need to compare the signals obtained by the two electro-oculogram electrodes, so as to remove the noise in the signal to obtain the effective electro-oculogram signal. 2) The traditional electro-oculogram electrode only adopts the electric potential variation information of the electrode in the electric field formed by the eyeball when the eye moves, and the information cannot reflect the space information, so the combined analysis of the electro-oculogram signals obtained by a plurality of electrodes is needed to restore the transformation condition of the electric field generated by the eyeball in the space, and the movement condition of the eyeball is finally obtained. Thus, the prior art electro-oculography requires a large number of electrodes distributed at different positions around the eye socket of the subject to provide sufficient information.

The use of multiple electrodes presents two problems, first, the many electrode configurations of prior electro-ocular systems consist of multiple electrodes distributed around the eye orbit. Before each detection, the configuration process of a plurality of electrodes is complicated, the electrodes are not friendly to the detected person, and the problems of wire winding and the like can be encountered in the acquisition process, so that the precision is influenced. Second, electrode placement introduces human error, with the result that the placement of electrodes at the same location at different times can vary. The error can be enlarged to a certain extent by arranging the electrodes respectively, so that human errors introduced by different acquisition tasks have great difference, and the accuracy is influenced finally.

2. The acquired spatial information of the electrooculogram signals of the respective channels cannot be acquired. The essence of the electrooculogram is a field signal, and the position of each acquired signal in the electric field is important information, so that the electrooculogram can provide help for signal analysis and improve the precision. The electrodes of the existing electro-oculogram acquisition system are mostly of a unit separation type, namely, each electrode is a unit and performs acquisition work independently, and the electrodes are arranged relatively independently. Since there is no spatial information about the relative positions of the electrodes, the resulting signal is the amplitude change of the potential with time series obtained by the electrodes. The spatial relationship between the signal sequences cannot be obtained.

3. The eye movement signal is not well separated from other noise signals. The main sources of noise affecting the ocular electrical signal are: electroencephalogram signals, electromyogram signals, artifacts, and other noise. In the existing processing method, a filter is used to remove signals except for an electro-ocular signal in an original signal in a preprocessing stage before or after acquisition. However, the idea of designing such a processing method is that other signals are all useless signals during the eye movement. In fact, the occurrence of the eye movement event is often inevitably accompanied by other valuable other signals which can be used for describing the eye movement event, such as myoelectric signals, electroencephalogram signals and the like caused by facial expression actions. These multi-modal physiological signals are helpful for analyzing the eye movement related condition of the detected person in different scenes. The existing electro-oculogram acquisition system has the problems that the positions of the electrodes are mutually independent and the space has no correlation, so that other noise signals obtained by each electrode except the obtained electro-oculogram signals are different, the signals obtained by each electrode cannot be compared, and each signal component in aliasing of all the noise signals cannot be accurately analyzed.

Disclosure of Invention

The invention aims to provide an eye movement detection system which is simple and convenient to use, high in robustness and high in integration and takes a multi-channel array eye electrode as a core so as to overcome the problems caused by the limitation of the existing scheme.

The invention provides an eye movement detection and analysis system of a multi-channel array electro-ocular electrode, which collects eye movement information and related physiological information (including but not limited to electromyographic information, electroencephalographic information and motion artifact information) of a detected person through one or more multi-channel array electro-ocular electrodes arranged on the epidermis around the eye socket of the detected person (such as the external eye corner), processes multi-source multi-dimensional physiological signals collected by the multi-channel array electro-ocular electrode through a signal processing and analyzing method and an intelligent algorithm provided by the system, and can effectively detect the eye movement condition with micro expression.

The invention provides an eye movement detection and analysis system based on a multi-channel array electro-oculogram, which comprises two parts, an acquisition module, a lower computer and an upper computer. Data transmission is carried out between the two parts through wired and wireless transmission equipment; the acquisition module and the lower computer part comprise a front-end acquisition device and a signal processing hardware circuit; the upper computer part is a user client computer pre-loaded with a signal analysis program and an information display and communication program. And the results of the information acquisition and analysis received by the client from the lower computer are displayed in the graphical user interface. The overall system framework is shown in fig. 1. Wherein:

the front-end acquisition equipment has the main function of acquiring physiological electric signals. The method comprises the following steps: the multi-channel array eye electrode, the reference electrode and the grounding electrode. Wherein, the grounding electrode is arranged at the left ear or the right ear breast projection of the detected person, and the reference electrode is arranged at the eyebrow center; the multi-channel array of electro-ocular electrodes is disposed on the periorbital epidermis of the user, such as the skin outside the external canthus.

The multichannel array eye electrode is an eye electrode acquisition array comprising a plurality of contacts, and the number of the contacts integrated on a single electrode is not less than 9. When the multi-channel array electro-ocular electrode is used, for effective detection, only one multi-channel array electro-ocular electrode is needed to collect obtained data, and the detection accuracy can be improved by using a plurality of multi-channel array electro-ocular electrodes.

The plurality of contacts on the multichannel array ocular electrode are integrated on a soft base material without ductility to form the multichannel array ocular electrode. The contact points are integrated on the double-layer flexible PCB board, the contact points are designed by partial through holes, and each contact point is connected to the socket interface at the tail end of the acquisition array electrode through a copper flat cable with gold plated on the surface. The multi-channel array electrooculogram electrode and skin fixing mode can use various modes which can make the contact point contact with the epidermis around the orbit, including but not limited to fixing mode which uses double-sided adhesive combined with conductive paste.

In the invention, the multichannel array electro-ocular electrode can be arranged at any position around the eye socket, such as the skin on the outer canthus. Fig. 2 shows a single electrode disposed outside the external canthus on the right side of the subject, wherein the multichannel array eye electrode is shown as e in fig. 2.

The signal processing hardware circuit mainly comprises a power management module, a physiological electric signal acquisition module and a main control module. The signal processing hardware circuit operating principle is shown in fig. 4. Wherein:

the power management module supplies power to the received power supply, and the power supply voltage is converted through a series of operational amplifier chip combinations to meet the voltage requirements of the main control module and the physiological electric signal module.

The scheme in this example: the power supply is mainly supplied by a 5V power supply through a USB interface, a +2.5V voltage is stably generated by using a TPS73225 chip, and a-2.5V voltage is stably generated by using TPS60403 and TPS72325 chips. In addition, the LP2985 chip is used for stably generating 3.3V voltage so as to meet the voltage requirements of the main control module and the physiological electric signal module.

Equally useful alternatives are also: the TPS73225 chip is replaced by a similar uA431 chip, a TL431 chip, an LM136-2.5 LDO voltage stabilization chip and the like.

The physiological electric signal acquisition module performs relative ground differential processing on the electric signal acquired from the front-end acquisition equipment to obtain an electric potential signal of each channel. The obtained signal is used for further processing by the main control module.

The scheme in this example: an analog front end ADS1299 chip is adopted, wherein all analog front ends are grounded in common and used for collecting the potential difference of all channels to a common grounding point. The working principle of the physiological signal acquisition module is shown in fig. 3.

Equally available alternatives are also: the ADS1299 chip is replaced with a similar, e.g., RHD2000 series, acquisition chip.

The main control module is responsible for controlling signal acquisition, primarily processing information and integrating and sending the encapsulated signals to the upper computer.

The scheme in this example: the STM32F103C8T6 chip has the advantages of low cost, low power consumption, small size and the like. The main control board module performs SPI communication configuration on a plurality of ADS1299 chips in the physiological electric signal acquisition module in a daisy chain mode, and sets sampling related parameters such as sampling synchronous clock, sampling frequency, analog front-end amplification factor, data precision and the like. By reading the DRDY pin of the ADS1299, the main control module can detect and send a reading command to read the acquisition voltage value of each ADS1299 chip and upload the acquisition voltage value to the upper computer. The main control module is arranged on the analog signal part and is connected with the multichannel array eye electrode through an FPC (flexible printed circuit) interface, wherein the analog small signal wiring is as short as possible, and the collected signals are prevented from being coupled with excessive noise. The acquired signals are packaged into frames by using a Uart protocol, and whether the transmitted data packets are abnormal or not is detected by using a parity check code.

The designed hardware system transmits data to an upper computer in a wireless mode (such as Bluetooth, Wifi and the like) or a wired mode (such as USB and the like) after collecting all electro-oculogram original signals at all times and finishing digital conversion of analog signals. And after receiving and decoding the data packet, the upper computer can store the data according to the appointed format.

Equally available alternatives are also: the processing chip may replace STM32F103C8T6 with a corresponding master module, for example: the system comprises a singlechip, an FPGA, a raspberry group and a high-integration main control module; the communication protocol CAN use I C, UART, CAN and other synchronous serial data transmission standards similar to SPI; the connection method may use other topology connection methods besides the daisy chain method, such as linear, ring, and star topology. The synchronous clock can be realized by using an external crystal oscillator besides the clock in the chip; the protocol used for signal encapsulation may also use a similar data transmission protocol, such as a wireless Wifi transmission protocol.

The upper computer can be a machine which is similar to a personal computer, has the functions of calculation, storage, processing and the like and has a human-computer interaction interface. Including personal computers, tablet computers, cell phones, etc. The upper computer part comprises a signal analysis and user graphic display interface. Wherein:

the signal analysis module comprises a signal preprocessing module, a signal analysis module and an intelligent algorithm module. Wherein:

the signal preprocessing module carries out preprocessing work such as band-pass filtering, power frequency removing, artifact eliminating, bad track detecting and damaged data removing on the original signal obtained by the main control module, and then transmits the processed signal to the signal analyzing module.

The signal analysis module firstly analyzes the independent components of the preprocessed signals to analyze the electro-oculogram signals and other physiological signals (such as electromyogram signals) related to the eye movement events, and then sends the analyzed signals of various sources to an intelligent algorithm for further analysis and detection of the eye movement events.

The intelligent algorithm module extracts time domain information, frequency domain information and spatial information of the analyzed signals (electro-oculogram signals, myoelectricity signals and the like) of different sources, and sends the signals into a detection model generated based on the intelligent algorithm module to finally obtain a detection result of the eye movement event.

The user graphic display interface is used for information display interaction, and comprises the steps of organizing analysis and detection results into information which is easy to understand by a user, and presenting the information to a detected person in a graphic form. The data that may be presented includes: the movement track of the sight line, the eyeball rotation condition classification, the micro expression during the eye movement and the like.

The innovation points of the system mainly comprise:

in the first part, in the aspect of electrode design, the invention provides a design scheme of an ocular electrode of a multi-channel array, which is convenient to use and high in integration level. In the scheme, only one electrode is arranged on one part of the epidermis around the eye socket of the detected person, such as the outer side of the external canthus, and the collection is carried out. Compared with the conventional electro-oculogram acquisition system which needs a plurality of electrode arrangements distributed around a plurality of eyes, the scheme of the invention improves the usability and the comfort level. The method provides high user experience and a high user acceptance basis for long-time eye movement detection in daily scenes of a system based on the electro-oculogram as a data source.

In the second part, in the aspect of source signal type selection, the signals acquired by the electrodes proposed in the first part are divided into separable multi-source signals. The physiological electric signals of various different sources when the eye movement event occurs, such as the eye electric signals and the electromyographic signals of the electrode coverage area, acquired by the scheme of the invention can be analyzed one by one through a signal processing method with high interpretability. The information which can be used for analyzing the eye movement event is richer than the existing eye movement event detection technology.

In the third part, in the aspect of information dimensionality of the obtained electro-oculogram signals, because the electrode design adopts a form of a multi-channel array electrode, the electro-oculogram signals of all electric shocks can be obtained during acquisition, and meanwhile, the space information among all signals is also reserved, and compared with the traditional electro-oculogram event detection technology, the electro-oculogram event detection method has richer information dimensionality. By the intelligent algorithm, the eye movement event can be accurately detected by only one electrode.

And combining the second part and the third part, using eye movement signals acquired by the multichannel array electro-ocular electrodes, and obtaining the sight gaze position condition of the detected person when the detected person rotates the eyeball and the facial state condition of the eyeball during movement after signal processing and intelligent algorithm processing.

Aiming at the defects of the existing electrooculogram system, the invention provides a scheme of arranging a multi-channel array electrooculogram electrode around the orbit to collect physiological signals related to the eye movement events. The scheme has a simple arrangement mode, and the facial actions of the detected person are influenced to a small extent. The physiological electric signals generated when the eye movement of a detected person occurs are collected by the high-resolution array electrode, and the spatial information among all the collected information is kept, so that the data dimension for establishing the eye movement detection model is richer. The data acquired by the acquisition scheme comprises a plurality of physiological electric signals including the electro-oculogram, so that different related physiological electric signals of a plurality of eye movement events are analyzed by a signal processing method such as blind source separation and the like, and the eye movement condition of the detected person can be restored from different angles.

Drawings

FIG. 1 is a block diagram of an eye movement detection and analysis system according to the present invention.

FIG. 2 is a schematic diagram of an electrode arrangement using a single multi-channel array of electro-ocular electrodes disposed outside the right external canthus.

Fig. 3 is a schematic diagram of a physiological signal acquisition module.

Fig. 4 is a schematic diagram of a circuit subsection of signal processing hardware.

FIG. 5 is a diagram of the overall system of the embodiment.

Fig. 6 shows a subject wearing a multichannel array electro-oculogram data acquisition headband.

Fig. 7-1 is a hardware outline diagram.

Fig. 7-2 is a hardware explosion diagram (1).

Fig. 7-3 is a hardware explosion diagram (2).

Fig. 8 is a schematic diagram of data preprocessing of an upper computer of a personal computer with an electro-ocular detection probe signal.

Fig. 9 is a schematic diagram of signal analysis and intelligent algorithm of the electro-oculogram detection head with a personal computer upper computer.

Fig. 10 is a comparison of decomposition results of looking jump to the right in different expression states.

FIG. 11 is a comparison of spatial information of each channel of the multi-channel array electro-ocular electrode with actual eye movement.

Fig. 12 shows the results of eye movement recognition in 8 directions.

FIG. 13 is an illustration of a user graphical interface.

Figure 14 is a typical electrooculogram electrode arrangement.

The reference numbers in the figures: a is an operation state indicator lamp, b is a reference electrode, c is an electrode insulation silica gel cushion, d is a physiological electric signal acquisition module, e is a multichannel array eye electric electrode, f is a power management module, g is a headband hardware circuit arrangement frame, h is a grounding electrode, i is an adjustable head bracket, j is a circuit protection shell, k is a main control module, l is a battery, m is a headband control key, and n is a USB interface.

Detailed Description

The invention provides an eye movement detection and analysis system of a multi-channel array electro-ocular electrode, which is designed into a complete hardware and software system comprising a multi-channel array electro-ocular data acquisition head band and a personal computer upper computer terminal. The acquisition module and the lower computer are designed into a wearable head band type structure, and the functions of acquiring, amplifying, controlling and the like of physiological electric signals related to the eye movement events are integrated. The upper computer adopts a personal computer terminal, and the whole system diagram is shown in figure 5.

The wearable electro-oculogram data acquisition module is designed into a head band type. The base material of the headband is a plastic material with toughness. The head band is integrated with a multi-channel array eye electrode plate, a reference electrode, a grounding electrode, a signal processing hardware circuit and other accessory circuits. The signal processing hardware circuit comprises a power management module, a main control module and a physiological electric signal acquisition module. The multichannel array ocular electrode, the reference electrode, the grounding electrode and the signal processing hardware circuit are all connected through FPC leads

Before using, at first by the person of being detected with the bandeau wear on the head for the bandeau just covers the eyebrow bone, makes multichannel array eye electrode piece can cover the outer canthus outside of right eye. Then three adjustable head frame head supports are adjusted to adapt to the head size of the detected person, so that the head support can be worn stably and used after being comfortable. The wearing method is shown in fig. 6.

The original electro-oculogram signals are acquired by the multi-channel array electro-oculogram electrode plate and transmitted to the signal processing hardware circuit through the flexible FPC conducting wire. The signal processing hardware circuit carries out a series of operations such as synchronization, amplification, digital-to-analog conversion and the like on the original data acquired by the data of each channel and uploads the original data to a PC upper computer in a USB high-speed wired transmission mode. Or the acquired original data can be wirelessly transmitted to a PC upper computer for analysis through a Bluetooth module.

Design of wearable head band type structure

In this embodiment, the acquisition module and each part of the lower computer are designed and arranged in a wearable head band type structure, the whole appearance of the wearable head band type structure is as shown in fig. 7-1, and fig. 7-2 is a hardware explosion diagram of the head band. The headband support structure comprises two parts, namely a headband hardware circuit mounting frame g and an adjustable head frame head support i. Wherein:

the headband hardware circuit mounting frame g is divided into a left mounting frame g-1, a right mounting frame g-2 and a front forehead g-3. The headband hardware circuit mounting frame g is made of non-toxic and harmless materials with good elasticity and impact resistance, and circuit units and sensor units are carried on the inner side and the outer side of the headband hardware circuit mounting frame. The circuit mounted on the headband hardware circuit mounting frame g includes: the right mounting frame g-2 is provided with an acquisition unit consisting of a multi-channel array electro-ocular electrode e and a physiological electric signal acquisition module d, and a main control module k consisting of a headband control key m, an operation state indicator lamp a and a USB interface n, and a transmission, control and feedback unit. The battery l is arranged on the left placing frame g-1, and the power management module f and the grounding electrode h form a supporting circuit. A reference electrode b is arranged on the front guard g-3. All the positions near the electrodes, which are in contact with the skin, are provided with insulating silicon bases c. All electronic components are wrapped by a circuit protection shell j. The specific positions of the parts are shown in fig. 7-2.

The number of the adjustable head frame head supports i is 3, and the adjustable head frame head supports i are distributed above the forehead, above the hindbrain and above the hindbrain. The adjustable head frame head support i is adjustable in design and comprises an adjustable metal frame i-1 and a head support i-2. The length of the adjustable head frame head support i can be adjusted by adjusting the length of the head support i-2 metal frame in the head support so as to adapt to the head sizes of different detected persons. The head band formed by the head band hardware circuit arrangement frame g and the adjustable head band head support i can provide better inward pressure, so that the sensor units on the inner side of the head band can be well pressed on the skin surface of a detected person. The reference electrode b positioned in the center of the forehead g-3 and the multichannel array eye electric electrode e positioned at the position of the outer eye of the right eye are carried on an insulating silicon base c, and the position of an electrode contact is slightly higher than the surface of the silicon base. 3 insulating silica gel base c guarantees that the electrode contact can have better contact nature with skin, and insulating and close skin's soft material also provides comfortable wearing experience and prevents the contact of circuit and skin.

The multi-channel array electro-ocular electrode e probe is a whole piece of PCB. The device is characterized in that 64 circular contact probes with the size of 2 mm are integrated on the device, and the distance between every two probes is 3 mm. The round electric shock probe is made of copper, and the surface of the round electric shock probe is plated with gold. The whole hardware design of the headband guarantees the requirements of convenient use, reliable fixation, comfortable wearing and long-term use.

(II) Circuit design

In consideration of wearing comfort and use convenience of the whole embodiment, the whole circuit design adopts a distributed modular design and comprises a power management module, a physiological electric signal acquisition module and a main control module. And all modules of the circuit are connected through the FPC.

(1) Power supply management module

The module uses various low dropout linear regulator chips to perform voltage conversion on a 5V power supply supplied by a USB so as to meet the power supply requirements of other modules of a hardware system. The function of the circuit is to convert 5V input voltage into stable +2.5V voltage, -2.5V voltage and +3.3V voltage to ensure the normal operation of other modules.

(2) Physiological electric signal acquisition module

The module is connected with the multichannel array eye electrode probe through an FPC interface. Each signal amplification circuit is designed by adopting a plurality of parallel 8-channel bioelectricity amplification quantization analog front ends ADS1299, and the functions of the signal amplification circuit are to amplify the acquired original multi-channel ocular electrical signals and perform analog-digital conversion so as to ensure that the signals for analysis have higher signal-to-noise ratio. The amplified signals are transmitted to the main control module through FPC wiring.

(3) Master control module

All components in the main control module are integrated on a whole PCB. The main chip STM32F103C8T6 chip undertakes the functions of controlling sampling related parameters such as sampling synchronous clock, sampling frequency, analog front end amplification factor, data precision and the like, storing the acquired data and transmitting the data to the upper computer of the personal computer through the Bluetooth module according to a transmission protocol. A high-performance microprocessor in the main control module and a Bluetooth module interface. The high-performance microprocessor adopts a parallel multi-SPI interface to simultaneously control a plurality of analog front ends, provides a unified sampling clock to output to the analog front ends, and transmits data to the upper computer of the personal computer while acquiring all electro-oculogram original signals at all times. And the upper computer of the personal computer decodes the received data packet and stores the data packet in the personal computer according to an appointed data form.

Software and intelligent algorithm on PC upper computer

The data processed by the main control module is imported into a PC (personal computer) in a wireless or wired mode by using Bluetooth or USB (universal serial bus), the original signals are preprocessed by upper computer analysis software, then the obtained data is analyzed in a combination task by using an intelligent algorithm, and the obtained analysis result is displayed in a user graphical interface.

In the invention, the system has the following working procedures:

when the system is used by a detected person for the first time, the system needs to be calibrated once. After the detected person wears the equipment, corresponding actions need to be completed according to prompts, and the acquired data are transmitted to an upper computer for storage and then used for eye movement event condition classification based on eye electrical signals. After the model training personalized calibration is completed, the user can use the device to detect and analyze the condition of the eye movement event.

The testee can browse and manage the records and reports by using client software installed on the PC side.

(1) Data pre-processing

The data preprocessing aims to filter useless information and ensure the quality of data sent to intelligent algorithm training so as to improve the accuracy of algorithm prediction and save the computing power of a personal computer upper computer and storage resources of a storage medium.

The preprocessing module firstly sends original data into a band-pass filter to filter and obtain 0.1-900Hz electric signals, and then a 50Hz notch filter is used for removing power frequency interference corresponding to 50Hz and frequency multiplication of the 50Hz notch filter, so that pure eye electric signals are obtained. The artifact elimination work is to process data by using operations such as wavelet transformation motion artifact algorithm, bad track detection, damaged data elimination and the like. The basic process flow is shown in fig. 8.

(2) Signal analysis and intelligent algorithm

The signal analysis module adopted in this embodiment focuses on collecting and decomposing the electric signals related to the eye movement event according to the usage scenario of this embodiment, so as to obtain the electro-ocular components and other expression-related components. Wherein, the electro-ocular component is a multi-channel electro-ocular signal, and the other expression related components are aliasing of the electromyographic signal and other signals. The multichannel eye electrical signals and other expression related components are sent to an intelligent algorithm module, so that the classification analysis of the eye movement condition and the classification analysis of the expression during the eye movement can be obtained. The whole intelligent algorithm module comprises two main parts, namely an eye electric signal processing sub-module and an expression related signal processing sub-module. The electro-oculogram signal processing part comprises down-sampling, a feature discriminator and voting decision, and the eye movement direction classification corresponding to the signals can be obtained by processing the multi-channel electro-oculogram signals. The expression-related signal processing part comprises feature extraction and integrated learning, and the expression and emotion classification corresponding to signals can be obtained by analyzing the electromyographic signals and other signals in the expression-related components. The intelligent algorithm flow of the embodiment is shown in fig. 9.

(2.1) Signal analysis

In the signal analysis module, the main method is an ica (independent Component analysis) independent Component analysis method. ICA is a common signal separation method used to extract original independent signals from mixed data. Among many ICA algorithms, the fixed point algorithm (FastICA) is often used because of its fast convergence rate and good separation effect. The FastICA algorithm has the form of kurtosis-based, likelihood maximum-based, negative entropy maximum-based and the like, and the FastICA based on the negative entropy maximum is used in the invention. The algorithm takes the maximum negative entropy as a search direction and can sequentially extract independent sources.

In the invention, the FastICA with the maximum negative entropy is used for acquiring electro-oculogram components, expression and action related electromyography components and the like. Because signals of various different sources are collected by the electrode and subjected to aliasing, the obtained data with a certain time duration are subjected to decentralization to enable the mean value of the data to be 0, then whitening processing is carried out on the data, and then optimized solution is carried out according to the maximum principle of negative entropy to obtain independent components in the data. And then, the electro-ocular component and the myoelectric component can be distinguished through a kurtosis tool, so that more accurate signal characteristic expression is obtained, and data preparation is made for subsequent different tasks.

(2.2) Intelligent Algorithm-expression-dependent Signal processing

And the expression related signal processing is finished by the expression related signal processing submodule. RMS, Mean and SSC three types of characteristics of n channels after low-pass filtering of the analyzed electromyographic signal components are extracted by a signal analysis module, and each action is converted into a column of 3-x-n dimensional characteristic vectors. After all the action features of the multiple types of expressions are extracted, model training is carried out through an integrated learning classifier and ten-fold cross validation, and the final classification accuracy is obtained. Three types of features were used: RMS can reflect the power of the signal, Mean can depict the average amplitude of the signal, SSC is helpful to reflect the strength condition of the muscle, and physiological information for distinguishing three expression actions is provided. The integrated learning classifier is mainly combined with a plurality of base learners to obtain more excellent generalization performance than a single learner.

(2.3) Intelligent Algorithm-Ocular Electrical Signal processing

The electro-ocular signal processing is completed by an electro-ocular signal processing submodule. Based on the previous verification experiment result, the data is downsampled in the task of judging the eye movement angle, the final judgment precision is not influenced, and the complexity of the algorithm can be greatly reduced. The found electro-ocular component signal is thus down-sampled four times.

Through previous verification experiments and literature research, the eye movement angle is in the range of 30 degrees, and the eye movement angle has strong linear characteristics. Therefore, the invention sends the data after down sampling to the linear discriminator to train, in order to obtain the discrimination model. In the decision phase, voting is performed by using a linear discrimination result based on a channel. If linear discrimination is carried out on n data acquired by the multichannel array electro-ocular electrode with the specification of n (n > = 9) channels, the category with the most channel data discrimination result is selected as the final discrimination result. Based on the discrimination mode, the accuracy of data discrimination can be improved. And the method has stronger robustness against the conditions that poor contact of partial electrodes, channel signal data interference and the like occur in the using process of the equipment.

(IV) Experimental setup and results

In order to test the feasibility of the implementation of the embodiment, the system scheme of the embodiment is used for collecting the eye jump motion data of the tested eye in 8 directions (horizontal right, horizontal left, vertical upward, vertical downward, 45 degrees above the right, 45 degrees above the left, 45 degrees below the right and 45 degrees below the left) under 3 expression states (no expression, glaring and biting), and the signal processing method of the invention is combined with an intelligent algorithm to analyze the eye movement events from multiple dimensions such as eye movement classification, expression classification and the like.

(1) Demonstration of decomposition experiment results of components of eye movement events

As can be seen from fig. 10, the electrical signals generated by the eye jump motion at the same eye angle in different expression states collected by the electrode designed by the present invention are different. The left image shows original data acquired by all channels when a task of controlling eyeballs to jump to the right is executed in two different expression states of tooth biting and eye glaring. The right diagram shows the eye movement components and the expression-related components of the two expression state information which are decomposed after the eye movement event independent component decomposition processing. It can be seen that the decomposed electro-oculogram components are similar because the eye movements in the two states are similar, and the decomposed expression-related components are different because the expression states in the two states are different.

(2) Display of comparison results of eye movement images and corresponding channel spatial feature heat maps under different eye movement conditions

The lower graph in fig. 11 is a heat map plotting performed by calculating the mean value of each channel according to a 20ms window after arranging the data obtained by all 9 channels according to the physical space position, and using the mean value of each channel. The upper panel of fig. 11 shows the eye movements tested in the experiment. The upper and lower diagrams in fig. 11 are in one-to-one correspondence. It can be seen that the variation of the spatial characteristics of the data collected by each channel of the collection method provided by the invention has a corresponding relationship with the eye movement of the detected person.

(3) Eye movement condition classification result display

And finally, the eye movement identification accuracy rate of 8 angles reaches 96.8%. As shown in fig. 12. The original signals acquired by the method are preprocessed by data, and then higher accuracy is obtained on models such as linear discrimination, integrated subspace discrimination, neural networks and the like. In addition, the classification task of eye jump (no expression state jump vision, squinting expression state jump vision and biting expression state jump vision) under the three expression states of the experiment can reach 95.1% of accuracy rate.

A graphical user interface: and providing a browsing report for a user, browsing the recorded condition, and carrying out management operation on the information through a graphical interface to play the display result, generate the report, manage the report and the like.

Fig. 13 is an illustration of an example of the user interface of the present embodiment.

The above-described devices and operations will be familiar to and understood by those of ordinary skill in the art.

The foregoing description of specific examples has been presented by way of illustration and is not to be construed as limiting. Further, many changes and modifications may be made within the scope of the embodiments of the present invention without departing from the spirit thereof, and the invention includes such changes and modifications.

Compared with the prior art, the invention has the advantages that:

1. the invention provides an electrode design scheme for collecting physiological signals related to eye movement events of a detected person by using a multi-channel array electro-ocular electrode, reserves the spatial information of each electro-ocular signal, and provides a more multidimensional signal basis for eye movement detection by using one electrode compared with the prior art;

2. the invention provides an eye movement detection and analysis system scheme which can detect the eye movement of a detected person by using an eye electric signal acquired by one electrode arranged around the eye socket of the detected person. Therefore, the detection setting is simpler and more convenient, and meanwhile, the artificial operation errors are less introduced;

3. the invention provides a signal preprocessing and signal analyzing scheme for separating an electro-oculogram signal, other eye movement related physiological electrical signals (such as an electromyogram signal, an electroencephalogram signal and the like) and a noise signal by a multi-channel array electro-oculogram electrode. The acquisition method, the signal processing and the intelligent algorithm expand the detection of the acquired signals from the simple eye movement to the detection of the relevant conditions during the eye movement. Therefore, compared with the existing eye movement method which only can detect the eye movement, the method provided by the invention has wider application scenes.

Claims (8)

1. An eye movement detection and analysis system based on a multichannel array electro-oculogram electrode is characterized by comprising two parts: the system comprises an acquisition module, a lower computer and an upper computer, wherein data transmission is carried out between the acquisition module and the lower computer through wired and wireless transmission equipment; the acquisition module and the lower computer part comprise front-end acquisition equipment and a signal processing hardware circuit; the upper computer part is a user client computer pre-loaded with a signal analysis program and an information display and communication program; the client acquires and analyzes the information received from the lower computer, and the result is displayed in a graphical user interface; wherein:

the front-end acquisition equipment is used for acquiring physiological electric signals; the method comprises the following steps: the multi-channel array eye electrode, the reference electrode and the grounding electrode; the grounding electrode is arranged at the left ear or the right ear breast projection of the detected person, and the reference electrode is arranged at the eyebrow center; the multichannel array electro-ocular electrodes are arranged at periorbital epidermis of a user; the multichannel array eye electrode is an eye electrode acquisition array comprising a plurality of contacts, and the number of the contacts integrated on a single electrode is not less than 9;

the signal processing hardware circuit is used for processing the physiological electric signals acquired by the front-end acquisition equipment; the method comprises the following steps: the physiological electric signal acquisition module is connected with the power management module; wherein:

the power supply management module supplies power to the received power supply, and the power supply voltage is converted through a series of operational amplifier chip combinations to meet the voltage of the main control module and the physiological electric signal module;

the power management module supplies power by using a 5V power supply of a USB interface, stably generates +2.5V voltage by using a TPS73225 chip, and stably generates-2.5V voltage by using TPS60403 and TPS72325 chips; in addition, a 3.3V voltage is stably generated by using an LP2985 chip so as to meet the voltage requirements of the main control module and the physiological electric signal module;

the physiological electric signal acquisition module performs relative differential processing on the electric signal obtained from the front-end acquisition equipment to obtain an electric potential signal of the channel; the obtained signals are used for further processing by the main control module;

the physiological electric signal acquisition module adopts an analog front end ADS1299 chip, wherein all the analog front ends are grounded and are used for acquiring the potential difference of all channels to a common grounding point;

the main control module is in charge of controlling signal acquisition, primarily processing information and integrally sending the encapsulated signals to an upper computer;

the main control module adopts an STM32F103C8T6 chip; the method comprises the following steps that a main control module carries out SPI communication configuration on a plurality of ADS1299 chips in a physiological electric signal acquisition module in a daisy chain mode, and sets sampling related parameters such as a sampling synchronous clock, sampling frequency, analog front-end amplification factor and data precision; by reading DRDY pins of the ADS1299, the main control module detects and sends a reading command to read the acquired voltage value of each ADS1299 chip and uploads the acquired voltage value to the upper computer; the main control module is arranged on the analog signal part and is connected with the multichannel array eye electrode through an FPC (flexible printed circuit) interface; the acquired signals are packaged into frames by using a Uart protocol, and a parity check code is used for detecting whether the transmitted data packets are abnormal or not;

the upper computer is a machine with the functions of calculation, storage, processing and the like and a human-computer interaction interface.

2. The eye movement detection and analysis system according to claim 1, wherein the plurality of contacts of the multi-channel array of electro-ocular electrodes are integrated on a flexible but non-stretchable substrate to form a multi-channel array of electro-ocular electrodes; the contact uses a partial through hole design, and each contact is connected to a socket interface at the tail end of the acquisition array electrode through a copper flat cable plated with gold on the surface; the multichannel array electro-ocular electrode and the skin are fixed in a fixing mode of combining double faced adhesive tape with conductive paste.

3. The eye detection analysis system of claim 1, wherein the upper computer portion comprises a signal analysis and user graphical display interface; wherein:

the signal analysis module comprises a signal preprocessing module, a signal analysis module and an intelligent algorithm module; wherein:

the signal preprocessing module is used for performing band-pass filtering, power frequency removal, artifact elimination, bad channel detection and damaged data elimination preprocessing on the original signal obtained by the main control module, and then transmitting the processed signal to the signal analysis module;

the signal analysis module is used for analyzing independent components of the preprocessed signals to analyze out electro-oculogram signals and other physiological signals related to the eye movement events, and then sending the analyzed signals of various sources to the intelligent algorithm module for further analysis and detection of the eye movement events;

the intelligent algorithm module extracts time domain information, frequency domain information and spatial information of the analyzed signals of the different sources, and sends the signals to a detection model generated based on an artificial intelligence algorithm to finally obtain a detection result of the eye movement event;

the user graphical display interface is used for information display interaction, and comprises information which is easy to understand for a user and is formed by analyzing a detection result, and the information is presented to a detected person in a graphical mode; the data that may be presented includes: the movement track of the sight line, the eyeball rotation condition classification and the micro expression during the eye movement.

4. The eye movement detection and analysis system according to claim 3, wherein in the signal analysis module, FastICA based on the maximum negative entropy is used, and the algorithm sequentially extracts independent sources with the maximum negative entropy as a search direction; comprises the steps of obtaining electro-oculogram components, expression and action related myoelectricity components by using FastICA; the method comprises the steps that signals of various different sources are mixed and collected by an electrode, the obtained data with a certain time duration are subjected to decentralization to enable the mean value of the data to be 0, then whitening processing is carried out on the data, and then optimization solving is carried out according to the negative entropy maximum principle to obtain independent components in the data; and then, distinguishing an electro-oculogram component and an electro-myoelectricity component by a kurtosis tool to obtain more accurate signal characteristic expression so as to prepare data for subsequent different tasks.

5. The eye movement detection and analysis system according to claim 4, wherein the intelligent algorithm module comprises an electro-ocular signal processing sub-module and an expression-related signal processing sub-module; the electro-ocular signal processing sub-module is used for processing an electro-ocular signal and comprises down-sampling, a feature discriminator and voting decision; the submodule processes the multichannel electro-oculogram signals to obtain eye movement direction classifications corresponding to the signals; the expression-related signal processing submodule is used for processing expression-related signals and comprises feature extraction and integrated learning, and the module analyzes and processes electromyographic signals and other signals in the expression-related components to obtain expression and emotion classifications corresponding to the signals.

6. The eye movement detection and analysis system according to claim 5, wherein the expression-related signal processing sub-module processes the expression-related signals, and comprises extracting the three types of RMS, Mean and SSC of n channels after the electromyographic signal components are low-pass filtered from the signal analysis module, and converting each action into a column of 3 x n-dimensional feature vectors; after all the action features of the multiple types of expressions are extracted, model training is carried out through an integrated learning classifier by using ten-fold cross validation, and final classification is obtained; three types of features were used: the power of the RMS response signal, the average amplitude of the Mean profile signal, and the SSC response to the exertion of the muscle provide physiological information that distinguishes between three expressive movements.

7. The eye movement detection and analysis system according to claim 5, wherein the electro-ocular signal processing sub-module processes the electro-ocular signals and performs a four-time down-sampling on the found electro-ocular component signals; sending the down-sampled data to a linear discriminator for training to obtain a discrimination model; in the decision-making stage, voting is carried out by using a channel-based linear discrimination result, n data acquired by the multichannel array electro-ocular electrodes of the n channels are linearly discriminated, and the category with the most channel data discrimination result is selected as a final discrimination result.

8. The eye movement detection and analysis system according to any one of claims 1 to 7, wherein the acquisition module and the lower computer are disposed in a wearable headband structure comprising two parts, namely a headband hardware circuit mounting frame (g) and an adjustable head frame head support (i); wherein:

the headband hardware circuit mounting frame (g) is divided into a left mounting frame (g-1), a right mounting frame (g-2) and a front forehead (g-3); the right placing frame (g-2) is used for installing an acquisition unit consisting of a multichannel array electro-ocular electrode (e) and a physiological electric signal acquisition module (d), and a main control module (k) consisting of a headband control key (m), an operation state indicator lamp (a) and a USB interface (n) to form a transmission, control and feedback unit; the left placing frame (g-1) is used for installing a supporting circuit consisting of a battery (l), a power management module (f) and a grounding electrode (h); the front guard (g-3) is used for installing a reference electrode (b); all the positions near the electrodes, which are in contact with the skin, are provided with insulating silicon bases (c); all electronic components are wrapped by a circuit protection shell (j); the number of the adjustable head frame head supports (i) is 3, and the adjustable head frame head supports are distributed above the forehead, above the hindbrain and at the hindbrain; the adjustable head frame head support (i) is adjustable in design and consists of an adjustable metal frame (i-1) and a head support (i-2); the length of the adjustable head frame head support (i) can be adjusted by adjusting the length of the metal frame of the head support (i-2) in the head support so as to adapt to the head sizes of different detected persons; the headband composed of the headband hardware circuit mounting frame (g) and the adjustable head frame and head support (i) can provide better inward pressure, so that each sensor unit on the inner side of the headband can be better pressed on the skin surface of a detected person; a reference electrode (b) positioned in the center of the forehead (g-3) and a multichannel array eye electric electrode (e) positioned at the position of the outer eye of the right eye are carried on an insulating silicon base (c), and the position of an electrode contact is slightly higher than the surface of the silicon base; 3 insulating silica gel bases (c) ensure that the electrode contact has good contact with the skin.

Images