CN117332256A

CN117332256A - Intention recognition method, device, computer equipment and storage medium

Info

Publication number: CN117332256A
Application number: CN202311288253.9A
Authority: CN
Inventors: 许启刚
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2023-10-07
Filing date: 2023-10-07
Publication date: 2024-01-02

Abstract

The application provides an intention recognition method, an intention recognition device, computer equipment and a storage medium, and belongs to the technical field of computers. The method comprises the following steps: for any one of a plurality of visual stimulation modes, performing visual stimulation on a first object to obtain a plurality of sample brain electrical signals induced by the visual stimulation mode, wherein each visual stimulation mode is used for representing a corresponding intention; performing feature analysis on the plurality of sample electroencephalogram signals by adopting an intention recognition mode to obtain a prediction intention of the first object; and adjusting the intention recognition mode based on the intention corresponding to the visual stimulus mode and the predicted intention, wherein the adjusted intention recognition mode is used for predicting the intention of a second object, and the second object comprises the first object. According to the technical scheme, the intention recognition mode is continuously optimized, so that the predicted intention of the intention recognition mode is more and more accurate, and the accuracy of the intention recognition can be effectively improved.

Description

Intention recognition method, device, computer equipment and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to an intent recognition method, apparatus, computer device, and storage medium.

Background

In recent years, brain-computer interfaces have become increasingly popular. The brain-computer interface is a technology for communicating with external environment equipment by utilizing brain electrical signals generated by human brain activities. How to achieve accurate human-computer interaction through brain-computer interfaces is the focus of research in the art.

At present, the following methods are generally adopted: collecting P300 waves induced for many times by visual stimulus through a brain-computer interface; then, carrying out average treatment on the P300 waves obtained for many times; and then, the P300 wave after processing is identified through a signal processing and intention identifying module, so that the intention of a user is judged, and further man-machine interaction is realized.

However, the waveform of the P300 signal in the above technical solution is small, and is susceptible to individual differences, fatigue, attention, and other factors, and the stability of the signal is relatively poor, so that the accuracy of intention recognition is low.

Disclosure of Invention

The embodiment of the application provides an intention recognition method, an intention recognition device, computer equipment and a storage medium, which are used for continuously optimizing an intention recognition mode so as to ensure that the predicted intention of the intention recognition mode is more and more accurate and effectively improve the accuracy of the intention recognition. The technical scheme is as follows:

In one aspect, there is provided an intention recognition method, the method including:

for any one of a plurality of visual stimulation modes, performing visual stimulation on a first object to obtain a plurality of sample brain electrical signals induced by the visual stimulation mode, wherein each visual stimulation mode is used for representing a corresponding intention;

performing feature analysis on the plurality of sample electroencephalogram signals by adopting an intention recognition mode to obtain a prediction intention of the first object;

and adjusting the intention recognition mode based on the intention corresponding to the visual stimulus mode and the predicted intention, wherein the adjusted intention recognition mode is used for predicting the intention of a second object, and the second object comprises the first object.

In another aspect, there is provided an intention recognition apparatus, the apparatus including:

the system comprises a signal acquisition module, a first object acquisition module and a second object acquisition module, wherein the signal acquisition module is used for carrying out visual stimulation on a first object in any one of a plurality of visual stimulation modes to obtain a plurality of sample brain electrical signals induced by the visual stimulation modes, and each visual stimulation mode is used for representing corresponding intention;

the characteristic analysis module is used for carrying out characteristic analysis on the plurality of sample electroencephalogram signals in an intention recognition mode to obtain the predicted intention of the first object;

The adjusting module is used for adjusting the intention recognition mode based on the intention corresponding to the visual stimulus mode and the predicted intention, and the adjusted intention recognition mode is used for predicting the intention of a second object, wherein the second object comprises the first object.

In some embodiments, the signal acquisition module comprises:

the display unit is used for displaying the visual stimulus modes and the intentions corresponding to each visual stimulus mode;

the display unit is also used for sequentially displaying the plurality of visual stimulus modes at different flicker frequencies respectively;

and the acquisition unit is used for acquiring a plurality of sample brain electrical signals corresponding to the visual stimulation mode in the process of displaying any visual stimulation mode.

In some embodiments, the display unit is configured to display, for any one of a plurality of visual stimulus modes, the visual stimulus modes with different flicker frequencies respectively; for each flicker frequency, acquiring a sample brain electrical signal based on a visual stimulation mode displayed at the flicker frequency; and selecting the flicker frequency corresponding to the sample brain electrical signal with the highest signal amplitude as the flicker frequency of the visual stimulation mode.

In some embodiments, the acquiring unit is configured to acquire a plurality of sample electroencephalograms in a process of displaying any visual stimulus mode; based on the functions of the brain regions in the human brain, a plurality of sample brain electrical signals acquired in the brain regions related to vision are acquired from the plurality of sample brain electrical signals.

In some embodiments, the visual stimulus mode display style includes at least one of the following:

the visual stimulation mode comprises a plurality of stripes, and at least one of the directions and the colors of the plurality of stripes are changed according to the flicker frequency of the visual stimulation mode;

the visual stimulus mode comprises a plurality of colors, and the colors are changed according to the flicker frequency of the visual stimulus mode;

the visual stimulation mode comprises a plurality of shapes, and the shapes are changed according to the flicker frequency of the visual stimulation mode;

the position of the visual stimulation mode changes according to the flicker frequency of the visual stimulation mode.

In some embodiments, the feature analysis module comprises:

the first processing unit is used for preprocessing the plurality of sample electroencephalograms, and the preprocessing is used for removing noise in the plurality of sample electroencephalograms;

And the second processing unit is used for carrying out feature analysis on the preprocessed plurality of sample electroencephalogram signals by adopting the intention recognition mode to obtain the predicted intention of the first object.

In some embodiments, the first processing unit is configured to perform band-pass filtering on any sample electroencephalogram signal to obtain an intermediate electroencephalogram signal, where the band-pass filtering is used to eliminate interference caused by a device for acquiring the sample electroencephalogram signal; and performing independent component analysis on the intermediate electroencephalogram signals to obtain preprocessed sample electroencephalogram signals, wherein the independent component analysis is used for eliminating interference caused by the first object when the sample electroencephalogram signals are acquired.

In some embodiments, the adjusting module is configured to adjust the intent recognition mode with a goal of maximizing a correlation between a sample electroencephalogram signal and a stimulation signal of the visual stimulation mode, where the predicted intent is the same as an intent corresponding to the visual stimulation mode, and the stimulation signal is used to represent a flicker frequency and a display style of the visual stimulation mode; and when the predicted intention is different from the intention corresponding to the visual stimulation mode, aiming at minimizing the correlation degree between the sample brain electrical signal and the stimulation signal of the visual stimulation mode, and adjusting the intention recognition mode.

In some embodiments, the signal acquisition module is further configured to perform visual stimulation on the first object for any one of the plurality of visual stimulation modes to obtain a plurality of sample biosignals induced by the visual stimulation mode, where the plurality of sample biosignals are at least one of an electrocardiosignal and an electromyographic signal;

the characteristic analysis module is used for carrying out characteristic analysis on the plurality of sample electroencephalogram signals and the plurality of sample biological signals by adopting the intention recognition mode to obtain the predicted intention of the first object.

In some embodiments, the signal acquisition module is further configured to acquire a plurality of electroencephalogram signals of the second subject;

the feature analysis module is further configured to perform feature analysis on the plurality of electroencephalogram signals based on the adjusted intent recognition manner, so as to obtain a predicted intent of the second object.

In another aspect, a computer device is provided, the computer device including a processor and a memory for storing at least one segment of a computer program loaded and executed by the processor to implement the intent recognition method in embodiments of the present application.

In another aspect, a computer readable storage medium having stored therein at least one segment of a computer program loaded and executed by a processor to implement a method for identifying intent as in embodiments of the present application is provided.

In another aspect, a computer program product is provided, comprising a computer program stored in a computer readable storage medium, the computer program being read from the computer readable storage medium by a processor of a computer device, the computer program being executed by the processor to cause the computer device to perform the method of intent recognition provided in each of the above aspects or in various alternative implementations of each of the aspects.

The embodiment of the application provides an intention recognition method, which sequentially carries out visual stimulation on a first object by adopting a plurality of visual stimulation modes, acquires a plurality of sample electroencephalograms of the first object in the process based on any visual stimulation, carries out feature analysis on the plurality of sample electroencephalograms by the intention recognition mode so as to predict the intention of the first object, and then compares the intention corresponding to the visual stimulation mode with the predicted intention, adjusts the intention recognition mode, so that the intention predicted by the intention recognition mode is more and more similar to the intention corresponding to the visual stimulation mode, namely, the intention recognition mode is optimized continuously, the intention predicted by the intention recognition mode is more and more accurate, and then predicts the intention of a second object by the optimized intention recognition mode, thereby effectively improving the accuracy of the intention recognition, and being beneficial to improving the efficiency and the user experience of man-machine interaction.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic view of an implementation environment of an intent recognition method provided in accordance with an embodiment of the present application;

FIG. 2 is a flow chart of an intent recognition method provided in accordance with an embodiment of the present application;

FIG. 3 is a schematic illustration of a vision forming process provided in accordance with an embodiment of the present application;

FIG. 4 is a flow chart of another intent recognition method provided in accordance with an embodiment of the present application;

FIG. 5 is a schematic diagram of a primary interface of an intent recognition system provided in accordance with an embodiment of the present application;

FIG. 6 is a schematic diagram of a visual stimulus interface provided in accordance with an embodiment of the present application;

FIG. 7 is a performance diagram of a visual stimulus interface provided in accordance with an embodiment of the present application;

FIG. 8 is a schematic diagram of an international 10-20 standard lead electrode placement standard provided in accordance with an embodiment of the present application;

FIG. 9 is a schematic diagram of an operating primary window provided in accordance with an embodiment of the present application;

FIG. 10 is a schematic illustration of a pretreatment provided in accordance with an embodiment of the present application;

FIG. 11 is a flow chart of a pre-process provided in accordance with an embodiment of the present application;

FIG. 12 is a schematic diagram of another visual stimulus interface provided in accordance with an embodiment of the present application;

FIG. 13 is a schematic diagram of an electroencephalogram signal provided according to an embodiment of the present application;

Fig. 14 is a waveform diagram of an electroencephalogram signal according to an embodiment of the present application;

FIG. 15 is a graph showing the extraction of characteristics of an electroencephalogram under stimulation at a stimulation frequency of 9HZ according to an embodiment of the present application;

fig. 16 is a feature extraction result of an electroencephalogram signal according to an embodiment of the present application;

FIG. 17 is a schematic diagram of yet another visual stimulus interface provided in accordance with an embodiment of the present application;

FIG. 18 is a block diagram of an intent recognition system provided in accordance with an embodiment of the present application;

FIG. 19 is a block diagram of an intent recognition device provided in accordance with an embodiment of the present application;

FIG. 20 is a block diagram of another intent recognition device provided in accordance with an embodiment of the present application;

fig. 21 is a block diagram of a terminal according to an embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

The terms "first," "second," and the like in this application are used to distinguish between identical or similar items that have substantially the same function and function, and it should be understood that there is no logical or chronological dependency between the "first," "second," and "nth" terms, nor is it limited to the number or order of execution.

The term "at least one" in this application means one or more, and the meaning of "a plurality of" means two or more.

It should be noted that, information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data for analysis, stored data, presented data, etc.), and signals referred to in this application are all authorized by the user or are fully authorized by the parties, and the collection, use, and processing of relevant data is required to comply with relevant laws and regulations and standards of relevant countries and regions. For example, reference herein to both a sample brain electrical signal of a first subject and a brain electrical signal of a second subject are acquired with sufficient authorization.

In order to facilitate understanding, terms related to the present application are explained below.

An Electroencephalogram (EEG) is an electrical signal generated by brain neuron activity and may be acquired by an Electroencephalogram helmet or other Electroencephalogram acquisition device. The electroencephalogram signals can be classified according to the frequency fluctuation range, and the electroencephalogram signals of each wave band can be active in different physiological activities of a human body. The electroencephalogram signals can be classified into 5 categories according to the above classification characteristics, as shown in the following table 1:

TABLE 1

Event-related potential (ERP) is an electroencephalogram signal related to a particular Event or stimulus. Event-related potentials may reflect physiological responses and cognitive processes of the brain to external stimuli.

An Electro-OculoGram (EOG) is an electrical signal generated by analyzing eye movement.

Human-machine interaction (HCI) refers to the process of information exchange and control between a person and a computer or other intelligent device.

Brain-computer interface technology (Brain-Computer Interface, BCI) achieves human-computer interaction by analyzing Brain electrical signals.

Independent component analysis (Independent Component Analysis, ICA) is a signal separation and feature extraction method that can separate a mixed signal into independent components for noise and artifact removal.

The typical correlation analysis (Canonical Correlation Analysis, CCA) is mainly used for analyzing and researching the correlation between every two variables, and is also a dimension reduction technology.

Steady state visual evoked potential (Steady State Visual Evoked Potential, SSVEP) is a steady state brain electrical response elicited by presentation of frequency modulated visual stimuli, commonly used in brain-machine interface studies and clinical examinations.

Convolutional neural networks (Convolutional Neural Network, CNN) are a deep learning algorithm that is mainly used to process data with a grid structure, such as images and videos, and achieve efficient feature extraction through local connection, weight sharing, and spatial dimension reduction.

A recurrent neural network (Recurrent Neural Network, RNN) is a neural network structure capable of processing sequence data by sharing parameters between time steps to capture long-term dependencies in the sequence.

Magnetoencephalography (MEG) is a non-invasive imaging technique used to measure weak magnetic fields generated by brain neural activity.

Stereoscopic electrophysiology (stereo Electro Encephalon Graphy, sEEG) is an invasive electroencephalography technique that records brain electrical activity by implanting electrodes into specific areas of the brain, commonly used for epileptic focus localization and functional brain region mapping.

Cochlear electrography (ECoG) is an invasive electroencephalogram technique that records brain electrical activity by placing electrodes on the surface of the cerebral cortex for research into brain function and clinical diagnosis.

Functional Near infrared spectroscopy (fNIRS) is a non-invasive brain imaging technique that studies brain activity by measuring changes in cerebral cortical blood oxygen saturation.

The intention recognition method provided by the embodiment of the application can be executed by computer equipment. In some embodiments, the computer device is a terminal or a server. In the following, taking a computer device as an example, an implementation environment of the intent recognition method provided in the embodiment of the present application will be described, and fig. 1 is a schematic diagram of an implementation environment of the intent recognition method provided in the embodiment of the present application. Referring to fig. 1, the implementation environment includes a terminal 101 and a server 102. The terminal 101 and the server 102 can be directly or indirectly connected through wired or wireless communication, and the present application is not limited herein.

In some embodiments, terminal 101 is, but is not limited to, a smart phone, tablet, notebook, desktop, smart speaker, smart watch, smart voice interaction device, smart home appliance, vehicle-mounted terminal, smart wheelchair, and unobstructed facility, etc. The terminal 101 is installed with an application supporting man-machine interaction. The application may be a game-type application, a virtual reality-type application, a multimedia-type application, or a communication-type application, etc. Illustratively, the terminal 101 is a terminal used by a user. The terminal 101 is configured with a signal acquisition instrument. The signal acquisition instrument can be used for acquiring biological signals such as brain electrical signals, electrocardiosignals or electromyographic signals and the like, and the embodiment of the application is not limited. The terminal 101 can analyze the collected bio-signals to identify the user's intention. Then, the terminal 101 can perform a corresponding operation according to the intention of the user.

Those skilled in the art will recognize that the number of terminals may be greater or lesser. Such as the above-mentioned terminals may be only one, or the above-mentioned terminals may be several tens or hundreds, or more. The number of terminals and the device type are not limited in the embodiment of the present application.

In some embodiments, the server 102 is a stand-alone physical server, can be a server cluster or a distributed system formed by a plurality of physical servers, and can also be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs (Content Delivery Network, content delivery networks), basic cloud computing services such as big data and artificial intelligence platforms, and the like. The server 102 is configured to provide background services for applications that support human-machine interaction. In some embodiments, the server 102 takes on primary computing work and the terminal 101 takes on secondary computing work; alternatively, the server 102 takes on secondary computing work and the terminal 101 takes on primary computing work; alternatively, a distributed computing architecture is used for collaborative computing between the server 102 and the terminal 101.

Fig. 2 is a flowchart of an intention recognition method provided according to an embodiment of the present application, and referring to fig. 2, description is given in the embodiment of the present application taking a terminal execution example. The intention recognition method includes the steps of:

201. for any one of a plurality of visual stimulation modes, the terminal performs visual stimulation on the first object to obtain a plurality of sample brain electrical signals induced by the visual stimulation modes, and each visual stimulation mode is used for representing a corresponding intention.

In the embodiments of the present application, when the eyes of a person are stimulated by light, the occipital lobe region of the brain will produce a corresponding electrical response. This electrical response, which is produced by the visual stimulus, becomes an electroencephalogram signal. Brain electrical signals reflect the functional state from retinal ganglion cells to the visual cortex and are often used to study human mental activities, behaviors, sensory functions, etc. The excitation intensity of the human visual pathway can be observed through the peak value of the brain electrical signal. The time from the reception of visual stimulus to the generation of peaks in the visual cortex can reflect the propagation speed of nerve excitation in the visual pathway. The brain electrical signal belongs to the visual evoked potential (Visual Evoked Potential, VEP). The waveform of the visual evoked potential has a main relation with age, and the amplitude of the brain electrical signal is roughly 40% -50% higher than that of the brain electrical signal in the single eye stimulation.

The principle of vision is that vision is formed by light emitted by a light source, and the light is refracted through an intraocular refraction system (comprising cornea, vitreous body, crystalline lens and aqueous humor) to enter visual sense organs, namely left and right eyes of a person and acts on retina containing photoreceptors and a nerve tissue network to cause visual sensation. Referring to fig. 3, fig. 3 is a schematic diagram of a vision forming process according to an embodiment of the present application. Photoreceptors can be classified into two major categories, rod cells and cone cells, according to their shape. When the light is darker, the light sensitivity of the video rod cell is higher, but the video rod cell cannot be finely spatially resolved and cannot participate in color vision, and the total number of the video rod cells is more than one hundred million. Cone cells are active in brighter environments. In contrast to the effects of rod cells, there are approximately 600 to 700 tens of thousands in the human retina. Their distribution is non-uniform. Rod cells and cone cells can transform visual information into neural information.

After the formation of the visual sensation, the visual sensation is transmitted out of the eye through the optic nerve fiber, is transmitted to the visual center of the cerebral cortex through the visual channel to be processed, and finally forms vision. During this entire transmission process, the axons of ganglion cells, the optic nerve fibers, are active, which are present mainly at the site of the visual intersection. Approximately half of 100 ten thousand optic nerve fibers were cast into the ipsilateral lateral thalamus and half crossed to the contralateral side. Mainly to the lateral knee, only a small portion to the upper hill. In the upper hill, visual information, somatosensory information and auditory information are combined to coordinate sensory response with the related movements of the ear, eyes and head. The projections of the nerve cells of the lateral knee make up the line of sight projected onto the primary visual cortex and then onto the higher visual center.

The overall process of human vision formation can be summarized as: the light passes through cornea and pupil of human eye, and is refracted by lens and vitreous body, then forms image on retina, and uses nerve treatment to produce nerve impulse, and uses optic nerve to transfer the visual information to brain visual centre, and finally forms vision in visual centre.

The visual stimulus means may be a background change means or an entity change means, which is not limited by the embodiments of the present application. Background variation means visual stimulation by varying the background. For example, the background is black and white stripes, and the terminal continuously changes the distribution of the black and white stripes, that is, the black and white stripes alternate. Entity change means visual stimulus by changing the way of an entity. For example, the entity is an apple and the terminal displays an image of the apple to visually stimulate the first object to induce the first object to produce an intent for the apple. The visual stimulus mode can be implemented by means of the built-in function of the psychropy software package, which is not limited by the embodiment of the present application.

The terminal displays a plurality of visual stimulus modes in the visual stimulus display interface. Each visual stimulus pattern corresponds to an intention. The intention may be up, down, left or right, etc. for controlling the movement of the target in a certain direction, and the embodiment of the present application is not limited to the intention. The terminal performs visual stimulation on the first object by displaying a visual stimulation mode to the first object, so that the first object is induced to generate a plurality of sample brain electrical signals. Then, the terminal can collect a plurality of sample brain-computer signals through brain-computer interface technology.

There are various ways to collect the electroencephalogram signals. Firstly, electroencephalogram acquired by using an electrode cap is the most common neuroimaging method in the research of brain-computer interface technology. Secondly, electrodes are implanted into cortex electroencephalogram and depth electrodes acquired on the surface of the brain through surgery, and functional magnetic resonance imaging for measuring cerebral blood flow changes related to different mental activities is performed. Functional magnetic resonance imaging cannot measure electrical activity because cortical electroencephalograms and deep electrodes can cause trauma to the subject. Therefore, the electroencephalogram mode is adopted to acquire the electroencephalogram signals, and the embodiment of the application is the most reasonable acquisition mode.

202. And the terminal adopts an intention recognition mode to perform feature analysis on the plurality of sample electroencephalogram signals so as to obtain the predicted intention of the first object.

In the embodiment of the application, for any sample brain electrical signal, the terminal adopts an intention recognition mode to perform feature analysis on the sample brain electrical signal, so as to obtain an analysis result of the sample brain electrical signal. Then, the terminal determines a predicted intention of the first object based on the analysis results of the plurality of sample brain electrical signals. The intention recognition method may be an independent component analysis algorithm, a convolutional neural network, or a deep learning method such as a convolutional neural network, which is not limited in the embodiment of the present application. The deep learning method has stronger feature extraction and classification capability, and compared with the traditional machine learning algorithm, the accuracy and stability of intention recognition can be improved.

203. The terminal adjusts an intention recognition mode based on the intention corresponding to the visual stimulus mode and the predicted intention, wherein the adjusted intention recognition mode is used for predicting the intention of a second object, and the second object comprises the first object.

In the embodiment of the application, the terminal determines the accuracy of predicting the intention by taking the intention corresponding to the visual stimulus mode as a reference. Then, the terminal adjusts the intention recognition mode according to the accuracy of the predicted intention. For example, the terminal may adjust parameters for performing feature analysis in the intent recognition manner, which is not limited in the embodiments of the present application. Then, the terminal can apply the adjusted intention recognition mode to various human-computer interaction scenes. In the man-machine interaction scene, the terminal can recognize the intention of the second object through the adjusted intention recognition mode. The second object may be the same as or different from the first object, which is not limited in the embodiment of the present application. That is, during the application process, an electroencephalogram signal acquisition system is used to acquire an electroencephalogram signal of the user. The brain electrical signals are converted into external information communication and control commands through recognition and decoding, so that bidirectional interaction with a user is realized. The system can be applied to the fields of wheelchair control, game interaction, virtual reality and the like, and provides more convenient and natural interaction experience for users. Meanwhile, the system can be optimized and adjusted according to actual requirements and scenes, so that performance and user experience are improved.

Fig. 4 is a flowchart of another intention recognition method provided according to an embodiment of the present application, referring to fig. 4, in the embodiment of the present application, an example will be described in which the method is executed by a terminal. The intention recognition method includes the steps of:

401. the terminal displays a plurality of visual stimulus modes and intentions corresponding to each visual stimulus mode.

In the embodiment of the application, the intentional identification system is operated on the terminal. The interface in the intention recognition system can be designed based on the python language, and the PyQt framework is used for constructing the interface and interacting and using the system by a user. PyQt is a program framework implemented by the python language based on a graphics program, which contains many different functional modules and classes that can provide many operations. The intent recognition system supports cross-platform and may run on operating systems such as Windows, linux and Mac OS.

For example, fig. 5 is a schematic diagram of a main interface of an intent recognition system according to an embodiment of the present application. Referring to fig. 5, the consciousness recognition system includes two parts, an on-line experiment and an off-line experiment. The off-line experiment refers to debugging an intention recognition mode capable of accurately predicting the intention based on a sample electroencephalogram signal of the first object. The first object may be referred to as a subject or tester. The online experiment refers to predicting the intention of each object by using a debugged intention recognition mode, and is equivalent to an application process. Common to both are the use of the emoiv epoc+14 channel EEG brain electrical signal acquisition system and DSI-stream data acquisition software. The difference is that the off-line experiment stores the collected brain electrical signals in the computer in an EDF file format, and the on-line experiment uses a socket to transmit data in real time. The signal processing module pre-processes the original signal and then carries out characteristic molecules, the step is mainly carried out by using a typical association analysis algorithm, the intention of the object is identified, and finally the result is displayed. The embodiment of the application provides an operation interface and an interaction design friendly to a user (object), simplifies setting and operation flow, and reduces the use threshold and learning cost of the user. This enables more users to conveniently use the system for human-computer interaction.

The intent recognition system includes a visual stimulus interface. The terminal may display a plurality of visual stimulus modes and intentions corresponding to each visual stimulus mode on the visual stimulus interface. The visual stimulus pattern may be user-defined. The embodiment of the application does not limit the display modes of the visual stimulus mode, and the following exemplarily describes various display modes. Optionally, the display mode of the visual stimulus mode includes at least one of the following:

first, the visual stimulus pattern includes a plurality of stripes. At least one of the direction and the color of the plurality of stripes is varied according to the flicker frequency of the visual stimulus pattern. For example, the visual stimulus mode is cross bar and vertical bar staggered display, left and right diagonal bar staggered display, straight grid and inclined grid staggered display, checkerboard overturning or the like.

Second, the visual stimulus means comprises a plurality of colors. The plurality of colors are transformed according to the flicker frequency of the visual stimulus pattern. For example, the visual stimulus is black and white interlaced display.

Third, visual stimuli include a variety of shapes. The various shapes are transformed according to the flicker frequency of the visual stimulus pattern. For example, the visual stimulus is a square circular interlaced display.

Fourth, the position of the visual stimulus means is changed according to the flicker frequency of the visual stimulus means. For example, visual stimulation is performed in such a manner that the position of the target for stimulation is changed continuously to form a trajectory. The trajectories corresponding to different visual stimulus modes are different.

The human eye vision system is also limited in its ability to resolve image details, with the human eye resolving more brightness details than color details. And when the brightness is changed, the effect of edge enhancement is generated, and the human eyes feel brighter in the bright places and darker in the dark places. This effect can be explained by the mach effect, which causes the error perception threshold near the edge pixels to be 3 to 4 times higher than the far edge threshold. The direction change of the black and white stripes and the checkerboard pattern of the checkerboard pattern are selected for the background of the visual stimulus according to the characteristics of the human eye visual system summarized above. The background change mode can stimulate the vision of human eyes and generate strong brain electric signals.

For example, fig. 6 is a schematic diagram of a visual stimulus interface provided in accordance with an embodiment of the present application. Referring to fig. 6, the visual stimulus patterns in the visual stimulus interface are eight in total, arranged in two rows, four in one row. Each visual stimulus pattern blinks at a steady frequency. The flicker frequencies of different visual stimulus modes are different, each line of visual stimulus modes has four conversion modes, and the four conversion modes are as follows in sequence: the horizontal bars and the vertical bars are displayed in a staggered manner, the left diagonal bars and the right diagonal bars are displayed in a staggered manner, the straight grids and the inclined grids are displayed in a staggered manner, and the checkerboard is turned over. The flicker frequency of the visual stimulus pattern of different rows is different.

In some embodiments, the terminal is capable of performing performance analysis on the visual stimulus interface after operation. To reduce the frame dropping rate of the interface when running, the visual stimulus interface is preferably run on a highly configured computer and if running on a notebook computer, a power plug is required or on a better configured desktop. The method is helpful to the accuracy of the subsequently acquired brain electrical signals, and is beneficial to subsequent research experiments. Because the frame dropping rate is higher, the flicker frequency of the visual stimulus mode is not accurate enough, and the acquired brain electrical signals have deviation.

For example, fig. 7 is a performance diagram of a visual stimulus interface provided in accordance with an embodiment of the present application. Referring to fig. 7, the left-hand abscissa is used to represent the sequence number of successive data frames during processing. The ordinate of the left graph is used to represent the time required to process each data frame. The abscissa of the right graph is used to represent time. The ordinate of the right graph is used to indicate the number of data frames generated at a particular point in time. The abrupt location in the figure indicates that a large number of data frames need to be processed when stimulated at this time. This generally corresponds to an active state in the brain electrical signal, for example when stimulated externally, visually, audibly, etc., the brain will react more strongly, resulting in more data frames. Whereas the smoother portion indicates that during this time the number of processed data frames is relatively small, which generally corresponds to a plateau in the electroencephalogram signal. In steady state, brain activity is stable, and is less affected by external stimulus, so the number of generated data frames is relatively small. Frame loss rate=dropped/frames=1/180=0.556%, for indicating that 1 frame is lost out of 180 Frames, the frame loss rate is 0.556%. The frame loss rate is an index for measuring the quality of the acquired brain electrical signals, and a lower frame loss rate means that the acquired brain electrical signals are more complete. Average processing time per frame mean=16.7 ms, standard deviation s.d. =1.93 ms.99% C (frame) =11.74-21.71, used to indicate that at a 99% confidence level we can predict that the processing time per frame signal is between 11.74ms and 21.71 ms.

A good visual stimulus interface should have a low frame dropping rate. The high frame dropping rate can cause inaccurate stimulation flicker frequency, thereby affecting the accuracy of the acquired brain electrical signals. For example, if one stimulus interface loses 10 frames out of 180 frames, the frame dropping rate is 10/180=5.56%, which may affect the accuracy of the stimulus and the user experience. In the data in FIG. 7, the average processing time per frame is 16.7ms, which means that the refresh rate is about 1s/16.7ms≡60Hz. In general, a 60Hz refresh rate may be considered good because it can meet the needs of most application scenarios. The conditions of high frame loss rate, overlong average processing time of each frame, large standard deviation, low confidence level and the like are all poor visual stimulation interfaces.

402. The terminal sequentially displays a plurality of visual stimulation modes at different flicker frequencies, and a plurality of sample brain electrical signals corresponding to the visual stimulation modes are acquired in the process of displaying any visual stimulation mode.

In the embodiment of the application, the terminal sequentially displays a plurality of visual stimulus modes at different flicker frequencies. The terminal can enable the first object to observe each visual stimulus mode in turn. That is, the terminal only displays one visual stimulation mode at a time, so as to avoid mutual interference of brain electrical signals generated by multiple stimulations at the same time. The terminal may set an appropriate rest time to alleviate fatigue of the first subject between changing visual stimulus patterns. Alternatively, to eliminate the effect of the stimulation sequence, the terminal may employ a display sequence of randomized visual stimulation patterns. This helps to avoid the influence of the stimulation sequence and fatigue on the acquired brain electrical signals. The embodiments of the present application are not limited in the manner in which the various visual stimulus modes are displayed.

For the setting of the flicker frequency, its frequency range must be within a range recognizable to the human eye. However, within this range, not all frequencies will produce a very pronounced brain electrical signal. The choice of frequency is limited by visual frequency limits, and the visual system is limited differently for stimuli of varying complexity. As the visual system produces an adaptation-back effect after repeated visual stimuli of the same kind, i.e. the way in which neurons of the visual system are active changes after repeated stimuli. Therefore, for different visual stimulation modes, different flicker frequencies are adopted for stimulation, so that human eyes can more accurately recognize different visual stimulation modes, and respective corresponding intentions are induced. The method is beneficial to improving the accuracy of the brain electrical signals.

In some embodiments, in determining the flicker frequency, the terminal may select the frequency to which the human eye is most sensitive as the flicker frequency. Accordingly, the flicker frequency is determined as follows: for any one of the visual stimulation modes, the terminal respectively adopts different flicker frequencies to display the visual stimulation mode. Then, for each flicker frequency, the terminal acquires a sample brain electrical signal based on the visual stimulus pattern displayed at the flicker frequency. Then, the terminal selects the flicker frequency corresponding to the sample brain electrical signal with the highest signal amplitude as the flicker frequency of the visual stimulation mode. According to the scheme provided by the embodiment of the application, the flicker frequency corresponding to the sample electroencephalogram signal with the highest signal amplitude is selected and used as the flicker frequency of the visual stimulation mode, so that the frequency sensitive to human eyes can be selected for stimulation, and the subsequent acquisition of the electroencephalogram signal with better quality is facilitated.

The frequency selection range is mainly concentrated between 4-50HZ, and can be divided into low frequencies below 15HZ, medium frequencies between 15-30HZ and high frequencies above 30 HZ. The lower frequency can induce stronger brain electrical signals, but easily causes more serious visual fatigue and even causes photosensitive epileptic seizure, which is unfavorable for the body health of the first object. High frequency stimulation is less favorable than low frequency for inducing steady state visual evoked potentials, but does not pose a threat to human health.

For any one of multiple visual stimulation modes, the terminal performs visual stimulation on the first object to obtain multiple sample brain electrical signals induced by the visual stimulation mode. According to the embodiment of the application, the Emotiv EPOC Flex Saline electroencephalogram signal acquisition system can be used for acquiring the electroencephalogram signal of the first object. The acquisition of the electroencephalogram signals is generally carried out by using an electroencephalogram cap of international 10-20 standard. The electrodes on the Emotiv EPOC Flex Saline electroencephalogram acquisition system can be configured at any 10-20 locations or on the ears.

For example, FIG. 8 is a schematic diagram of an international 10-20 standard lead electrode placement standard provided in accordance with an embodiment of the present application. Referring to fig. 8, electrode names are distinguished according to prefixes, F, fp, T, C, O, P respectively representing frontal lobe, temporal lobe, central zone, occipital lobe, and parietal lobe. The numerical suffix odd indicates the left hemisphere and even indicates the right hemisphere. The head skin of the first subject is cleaned and the keratinous tissue is removed locally prior to wearing the electroencephalogram cap. When the electroencephalogram cap is worn, the mandible of a first subject is fixed, after the electroencephalogram cap is worn, the positions of electrodes which are needed to be used are positioned, the electrodes are placed, and in the process, the condition that each electrode is vertically contacted with the scalp needs to be determined. Then, an appropriate amount of electrode paste was injected into each of the mounted electrodes using a syringe, and the electrode paste was brought into sufficient contact with the scalp using gentle rotation of the flat-headed needle. The electrode paste is an electrolyte solution, does not irritate the skin of the first subject, and has the effects of reducing the gap generated between the electrode and the scalp and enhancing the conductivity. Wearing the electrode to perform resistance test on the connected electrode. And after all the data are ready, the data are acquired by matching with data acquisition software. The used electroencephalogram cap after data acquisition needs to be cleaned by distilled water and dried in the shade.

When the brain electrical information acquired in the brain electrical cap is recorded, the terminal can use DSI-STREAMER data acquisition software. The software can confirm whether the scalp is in good contact with the sensor during the process of collecting data. The method is characterized in that when the data source window tab of the main window is operated to start recording, the status lamp is changed from a yellow connection status to a green data flow status. Referring to fig. 9, fig. 9 is a schematic diagram of an operation main window according to an embodiment of the present application. The terminal displays the acquired brain electricity information in real time and stores the brain electricity information in a EDF (European Data Format) file format into a computer. The EDF format file may store a multi-channel file, and each signal may be at a different sampling frequency, with a header and one or more data records within the file. The header contains some conventional information (object identification, start time, etc.) and the specifications (calibration, sampling rate, filtering, etc.) of each signal. Encoded as ASCII characters. The data record contains samples that are small end 16-bit integers. The acquisition software is provided with an impedance driver which generates a tiny safe current of 110-130hz to the scalp connected with the electroencephalogram cap when the program is activated so as to measure the impedance between the electrode and the scalp.

The connection state of the sensor is shown in real time in fig. 9: the first state 901 represents that the connection is normal; the second state 902 represents an edge operation; the first state 903 represents an operational failure or an operational error. After the latter two states occur, the diagnostic sub tab needs to be checked to check for faults. Edge operation means that the contact between the sensor and the scalp may not be sufficiently tight, but not yet completely disengaged. For example, the electrodes may not be in intimate contact with the scalp due to insufficient hair or conductive gel, etc. In this state, the acquisition of the brain electrical signal may be disturbed to some extent, but measurement may still be performed. An operational error refers to a severe poor contact between the sensor and the scalp and may even be completely detached. For example, the electrode may lose contact with the scalp due to improper installation, loosening, or complete drying of the conductive gel. In this state, the acquisition of the electroencephalogram signal is severely disturbed, and even effective measurement cannot be performed.

In the process of displaying any visual stimulation mode, the terminal acquires a plurality of sample brain electrical signals corresponding to the visual stimulation mode. The plurality of sample brain electrical signals may be steady-state visual evoked potentials, which is not limited in the embodiments of the present application.

In some embodiments, the terminal is capable of screening out a sample electroencephalogram signal related to vision from a plurality of acquired sample electroencephalograms signals so as to enable accurate feature analysis to be performed subsequently. Correspondingly, the process of acquiring a plurality of sample brain electrical signals corresponding to the visual stimulation mode by the terminal comprises the following steps: in the process of displaying any visual stimulation mode, the terminal acquires a plurality of sample brain electrical signals. Then, the terminal acquires a plurality of sample brain electrical signals acquired in the brain region related to vision from the plurality of sample brain electrical signals based on the functions of the respective brain regions in the human brain. Because brain electricity induced by visual stimulus is mainly generated on occipital lobes, the embodiment of the application can select sample brain electricity signals collected by the P3, P4, O1 and O2 electrodes from the international 10-20 standard lead electrodes. According to the scheme provided by the embodiment of the application, the sample brain electrical signals related to vision can be screened out, and the follow-up more accurate prediction of intention is facilitated.

In some embodiments, in addition to the electroencephalogram signals, the terminal can also acquire other biological signals to combine the electroencephalogram signals to predict intent. Correspondingly, for any one of a plurality of visual stimulation modes, the terminal performs visual stimulation on the first object to obtain a plurality of sample biological signals induced by the visual stimulation mode, wherein the plurality of sample biological signals are at least one of electrocardiosignals and electromyographic signals. Electrocardiographic signals are also a type of biological signal. The terminal can fuse various sample biological signals so as to improve accuracy and stability of intention recognition. The multi-signal fusion can provide richer physiological information, and is beneficial to improving the efficiency and user experience of human-computer interaction.

In some embodiments, the terminal may acquire multimodal evoked brain electrical signals in addition to the visually evoked brain electrical signals. Correspondingly, the terminal stimulates the first object based on a visual stimulation mode, an auditory induction mode and a tactile induction mode to obtain a plurality of sample brain electrical signals. The multi-mode induced electroencephalogram signals can provide richer interaction information, and are beneficial to improving accuracy and instantaneity of intention recognition.

403. And the terminal adopts an intention recognition mode to perform feature analysis on the plurality of sample electroencephalogram signals so as to obtain the predicted intention of the first object.

In the embodiment of the application, some irresistible interference factors can be suffered in the acquisition process of the electroencephalogram signals. Signals generated by these interference factors are superimposed on the acquired brain electrical signals, creating artifacts (noise). These artifacts affect the accuracy of the subsequent analysis of the characteristics of the electroencephalogram signals, so that preprocessing is required before the electroencephalogram signals are analyzed to filter out the artifacts. Correspondingly, the terminal adopts an intention recognition mode to perform feature analysis on a plurality of sample electroencephalogram signals, and the process for obtaining the predicted intention of the first object comprises the following steps: the terminal preprocesses a plurality of sample brain electrical signals. The preprocessing is used for removing noise in the plurality of sample electroencephalogram signals. And then, the terminal adopts an intention recognition mode to perform feature analysis on the preprocessed plurality of sample electroencephalograms to obtain the predicted intention of the first object. According to the scheme provided by the embodiment of the application, before the characteristic analysis is carried out on the sample brain electrical signal, the sample brain electrical signal can be preprocessed so as to remove noise in the sample brain electrical signal, and the accuracy and stability of follow-up intention recognition are improved.

In some embodiments, the electroencephalogram signal is a steady-state visual evoked potential. Steady-state visual evoked potentials have high time-varying sensitivity, and their interfering components can be divided into the following categories: the terminal is interfered by power frequency of voltage in the form of electromagnetic wave; myoelectric disturbances (EMG signals) caused by muscle movement; electrode interference caused by poor electrode contact or poor conductivity of the electrode material; electrocardiographic interference with a frequency of about 1HZ caused by heart beat; the first subject blinks and eye rotation causes ocular artifacts upon signal acquisition. Of course, the pretreatment of the interfering components is different for different steady-state visual evoked potentials.

For example, fig. 10 is a schematic diagram of a pretreatment provided according to an embodiment of the present application. Referring to fig. 10, because the most significant effect on steady state visual evoked potentials is ocular artifacts (EOG signals), the primary purpose of the pre-processing is to remove ocular artifacts. Because of the apparent EOG artifacts in the signal, an independent component analysis algorithm may be selected.

In some embodiments, during the preprocessing, the terminal performs band-pass filtering on the sample electroencephalogram signals to obtain intermediate electroencephalogram signals. The band-pass filtering is used for eliminating interference brought by equipment for acquiring the sample brain electrical signals. And then, the terminal performs independent component analysis on the intermediate electroencephalogram signal to obtain a preprocessed sample electroencephalogram signal. The independent component analysis is used for eliminating interference brought by the first object when the sample electroencephalogram signal is acquired. According to the scheme provided by the embodiment of the application, the band-pass filtering and the independent component analysis are sequentially carried out on the sample electroencephalogram signals, so that the interference caused by equipment and objects in the electroencephalogram signals can be removed, the components of the electroencephalogram signals are purer, and the accuracy and the stability of the follow-up intention recognition are improved.

For example, fig. 11 is a flow chart of a preprocessing provided according to an embodiment of the present application. Referring to fig. 11, the terminal firstly removes the unnecessary electrodes, and because brain electricity induced by visual stimulus is mainly generated on occipital lobes, the electrodes P3, P4, O1 and O2 in the international 10-20 standard lead electrode placement mode are mainly used, and signals of other electrodes need to be filtered. The signals above 50HZ and below 3HZ are then filtered using band pass filtering. Bandpass filtering may filter out frequency components of a particular range by attenuating other range frequency components to very low levels. Finally, the independent component analysis algorithm is used for component analysis.

Because the electroencephalogram signals acquired by each electrode during data acquisition are not necessarily signals generated by the position of the electrode, the signals can be influenced by electroencephalogram signals at other positions, namely, the electric influences of different sources. Whereas the independent component analysis algorithm was originally applied to blind source signal separation. Independent component analysis may perform some linear decomposition of the data into statistically independent components, and then filter out unwanted artifacts from these independent components, thereby obtaining the desired electroencephalogram signal. The basic source signal can be recovered from the linear mixed signal using independent component analysis. The mode of independent component analysis is: x=as. Wherein the observation signal x is formed by mixing independent signals s through a mixing matrix A. A. s is unknown and independent component analysis can be done by estimating a and s with x known. s can be known by s=wx. Where W is the inverse of A.

There is a specialized independent component analysis software package in MNE-Python, which is an open source Python module for processing, analyzing and visualizing functional neuroimage data (EEG, MEG, sEEG, ECoG and fNIRS). The mn.preprocessing.ica interface specially preprocesses signals, and the ICA independent component analysis algorithm is sensitive to low-frequency drift and needs to filter low-frequency signals well before use. In the examples of the present application, frequencies below 3HZ are filtered out.

In some embodiments, the terminal is capable of acquiring stimulation signals for each visual stimulation modality during the profiling process. The stimulation signal is used to represent the flicker frequency and display pattern of the visual stimulation pattern. And then, the terminal adopts typical association analysis to respectively determine the relativity of the sample brain electrical signals and each stimulation signal. Then, the terminal uses the intention of the visual stimulus pattern corresponding to the maximum correlation as the predicted intention of the first object. According to the scheme provided by the embodiment of the application, through typical association analysis, the correlation degree between the brain electrical signal and the visual stimulation mode is analyzed, so that the visual stimulation mode from which the brain electrical signal originates can be accurately analyzed, and the intention of the first object can be accurately determined.

In some embodiments, the terminal may calculate the correlation between the sample brain electrical signal and the stimulation signal through the following formula one.

Equation one:

wherein X is used for representing a sample brain electrical signal; y is used for representing the stimulation signal of the visual stimulation mode; cov (X, Y) is used to represent the covariance between the sample brain electrical signal and the stimulus signal; d (X) is used to represent the variance of the sample brain electrical signal; d (Y) is used to represent the variance of the stimulus signal; ρ (X, Y) is used to represent the correlation between the sample brain electrical signal and the stimulation signal, and the value range of ρ is [ -1,1]. The closer the value of the correlation is to 1, the higher the linear correlation between x, Y, and vice versa. Correlation of one-dimensional data may be analyzed using ρ, but high-dimensional data may not directly use ρ. The typical correlation analysis is realized by reducing the dimensions of the multidimensional X, Y to one-dimensional X 'and Y', and then analyzing the correlation coefficients of the X 'and Y'.

For the two sets of data X, Y, the sample matrices are respectively n1×m and n2×m, n1 and n2 are feature dimensions, and m is the number of samples. And projecting the two matrixes to obtain two linear coefficient vectors a and b and one-dimensional vectors X 'and Y'. X' =a ^T X、Y'＝b ^T Y。

Normalizing the original data:

ρ(X′,Y′)＝cov(a ^T X,b ^T Y)＝E((a ^T X)(b ^T Y) ^T )＝a ^T E(XY ^T )b

D(X′)＝D(a ^T X)＝a ^T E(XX ^T )a

D(Y′)＝D(b ^T Y)＝b ^T E(YY ^T )b

The average value of X, Y at this time is 0:

D(X)＝cov(X,X)＝E(XX ^T ),D(Y)＝cov(Y,Y)＝E(YY ^T )

cov(X,Y)＝E(XY ^T ),cov(Y,X)＝E(YX ^T )

let S _XY = cov (X, Y), ρ can be converted into:

/>

in the embodiment of the application, the typical association analysis algorithm needs to use four electrodes of O1, O2, P3 and P4, and a PZ electrode is used for reference. Because the accuracy of the typical correlation analysis algorithm is relatively high compared to the fast fourier transform (Fast Fourier Transform, FFT), the typical correlation analysis algorithm is selected as the feature analysis algorithm in the pattern of intent recognition of the present system in the embodiments of the present application.

In some embodiments, the terminal may perform feature analysis on the plurality of sample electroencephalograms and the plurality of sample biological signals in an intent recognition manner to obtain the predicted intent of the first object. The multi-signal fusion can provide richer physiological information, and is beneficial to improving the efficiency and user experience of human-computer interaction.

404. The terminal adjusts an intention recognition mode based on the intention corresponding to the visual stimulus mode and the predicted intention, wherein the adjusted intention recognition mode is used for predicting the intention of a second object, and the second object comprises the first object.

In the embodiment of the application, when the predicted intention is the same as the intention corresponding to the visual stimulation mode, the terminal aims at maximizing the correlation between the sample electroencephalogram signal and the stimulation signal of the visual stimulation mode, and adjusts the intention recognition mode. The stimulation signal is used to represent the flicker frequency and display pattern of the visual stimulation pattern. When the predicted intention is different from the intention corresponding to the visual stimulation mode, the terminal aims at minimizing the correlation between the sample brain electrical signal and the stimulation signal of the visual stimulation mode and adjusts the intention recognition mode. According to the scheme provided by the embodiment of the application, under the condition that the prediction is correct, the intention recognition mode can be further adjusted, so that the correlation between stimulus signals corresponding to the real intention, which are calculated based on the intention recognition mode, is larger and larger, namely, the accuracy of the intention recognition mode is further improved; in the case of a prediction error, the intention recognition method may be further adjusted such that the correlation between the stimulus signals corresponding to the erroneous intention calculated based on the intention recognition method becomes smaller and smaller, that is, the erroneous recognition of the corrected intention recognition method.

The processing of steps 401 to 404 is the processing of the offline experiment. In some embodiments, during an in-line experiment, the terminal may collect multiple sets of data for analysis by multiple first objects (subjects) to verify the accuracy of the pattern of intent recognition. After the accuracy of the intention recognition mode is verified by the on-line experiment, the intention recognition mode of the on-line experiment can be applied to the on-line experiment. The off-line data acquisition needs to be used in DSI-STREAMER data acquisition software, equipment is debugged after an electroencephalogram cap is worn, and the data is acquired after an electrode is well contacted with a scalp. The first object needs to focus on each specific frequency-changing stimulus in the visual stimulus mode, each stimulus focuses on about ten seconds, and the electroencephalogram signals generated by each stimulus are collected and respectively stored as EDF file format data. And preprocessing and characteristic analysis are performed on the electroencephalogram signals under the line, and the result is displayed.

For example, the flicker frequencies selected in the embodiment of the present application are 8HZ, 9HZ, 10HZ, and 11HZ, and the disk grid is turned over, the horizontal and vertical stripes are alternately displayed, the straight grid is alternately displayed, and the left and right diagonal stripes are alternately displayed, which correspond to the four control intentions respectively. The visual stimulus mode is a circle with each radius being 100px, and the vertical-horizontal spacing of each visual stimulus mode is 200px. Referring to fig. 12, fig. 12 is a schematic diagram of another visual stimulus interface provided according to an embodiment of the present application.

The terminal can transmit data in real time by means of a TCP/IP socket of the DSI-stream data acquisition software, and convert the data packet byte array into a corresponding format described in a TCP/IP socket protocol document, as shown in Table 2.

TABLE 2

And the terminal adopts a typical association analysis algorithm to perform characteristic analysis and displays the result of the electroencephalogram signal on an interface. Please refer to fig. 13. Fig. 13 is a schematic diagram of an electroencephalogram signal according to an embodiment of the present application. The electroencephalogram signals acquired through the plurality of electrodes are shown in fig. 13.

In the process of on-line experiments, in order to verify the accuracy of feature extraction, the terminal firstly collects and analyzes the electroencephalogram signals on the most common black and white square flicker with different frequencies in an off-line mode. Because less frequent stimulation is detrimental to the physical health of the subject, higher frequency stimulation cannot trigger a characteristic significant brain electrical signal (e.g., SSVEP signal). The terminal narrows the frequency range setting to 8-15HZ. The subsequent terminal tests the visual stimulus with the background of stripe change, when the visual stimulus mode of alternately displaying the straight grid and the inclined grid is tested, the test is sequentially carried out by adopting different flicker frequencies, and the result shows that the visual stimulus mode can not generate SSVEP signals with obvious characteristics, and the test result is shown in figure 14. Fig. 14 is a waveform diagram of an electroencephalogram signal according to an embodiment of the present application.

Therefore, the visual stimulus mode of alternate display of the straight grid and the inclined grid is omitted in the embodiment of the application. Since the checkerboard is found to be more stimulated to induce a stable SSVEP signal in the offline experiments, the terminal changes the visual stimulation mode of the checkerboard adopting radial contraction motion, and the visual stimulation mode rotates at a certain frequency for the induction of the electroencephalogram signal in the same angle. The evoked electroencephalogram signals are quite stable, and obvious frequency characteristics of the electroencephalogram signals can be extracted.

It should be noted that, when setting the stimulation frequency (flicker frequency), frequencies with common multiples need to be avoided, because a certain error may be generated due to poor performance of the visual stimulation interface. That is, two peaks are generated at the time of feature extraction, as shown in fig. 15, fig. 15 is a result of feature extraction of an electroencephalogram signal under stimulation frequency induction of 9HZ according to an embodiment of the present application.

As a result, it was found that the stimulation accuracy was relatively low at a frequency of 13HZ or higher, and therefore, experiments were conducted using a frequency of 8 to 11HZ in the examples of the present application. The 12HZ is removed because the frequency of the human brain, which is the alpha wave produced by the human at rest, is approximately 12HZ without any visual stimulus. To avoid this specificity, the 12HZ frequency is omitted. The experimental results are shown in fig. 16, and fig. 16 is a feature extraction result of an electroencephalogram signal according to an embodiment of the present application. The display frequency is floated with an up-down error of about 0.5. And the experimental result shows that the feature extraction accuracy is higher in the checkerboard overturning mode. In view of the above results, the visual stimulus interface is shown in fig. 17 after the terminal changes the visual stimulus mode. Fig. 17 is a schematic diagram of yet another visual stimulus interface provided in accordance with an embodiment of the present application. The embodiment of the application adopts a visual stimulation mode design, is similar to a modularized design, and can conveniently perform function expansion and optimization.

In some embodiments, the off-line experiment ends and the terminal applies the adjusted intent recognition approach to the on-line experiment. Correspondingly, the terminal acquires a plurality of electroencephalogram signals of the second object. And then, the terminal performs feature analysis on the plurality of electroencephalogram signals based on the adjusted intention recognition mode to obtain the predicted intention of the second object. According to the scheme provided by the embodiment of the application, by adopting the visually induced electroencephalogram signals, the system can induce obvious signal characteristics in a short time, and signal processing and intention recognition can be performed in real time. This helps to improve the real-time nature of human-machine interaction, enabling the user to interact with the device more quickly. The system adopts the brain electrical signals induced by vision as an interaction basis, and can realize a man-machine interaction mode without manual operation. Such an interaction means is of great significance to users who are inconvenient to act or who cannot use the conventional interaction means. For example, the intention recognition method provided by the embodiment of the application can be applied to the fields of wheelchair control, game interaction, virtual reality, barrier-free facility operation and the like, and has a wide application prospect.

For example, fig. 18 is a frame diagram of an intent recognition system provided in accordance with an embodiment of the present application. Referring to fig. 18, a subject wears a device for acquiring brain electrical signals (e.g., an emoiv epoc+14 channel brain electrical signal acquisition meter). The terminal then displays a visual stimulus interface. The visual stimulus interface displays visual stimulus modes. The terminal performs visual stimulation on the object in a visual stimulation mode. Then, the terminal acquires the brain electrical signals acquired by the brain electrical signal acquisition instrument. Then, the terminal preprocesses the electroencephalogram signals. Then, the terminal adopts a consciousness recognition mode to perform feature analysis on the preprocessed electroencephalogram signals, so that the intention of the object is determined. Then, the terminal displays the intention of the object. The terminal may display the text corresponding to the intention, or directly display the execution process controlled by the intention, which is not limited in the embodiment of the present application.

In some embodiments, in addition to bio-signal based intent recognition, the terminal may also be based on context aware technologies (e.g., environmental sensors, behavioral analysis, etc.) to enable more intelligent and natural human-machine interaction. Context awareness can help the system to better understand the needs and environment of the user, improving adaptability and flexibility of interaction.

Fig. 19 is a block diagram of an intent recognition device provided in accordance with an embodiment of the present application. The intention recognition device is for performing the steps when the above-described intention recognition method is performed, and referring to fig. 19, the intention recognition device includes: a signal acquisition module 1901, a feature analysis module 1902, and an adjustment module 1903.

The signal acquisition module 1901 is configured to perform visual stimulation on the first object for any one of a plurality of visual stimulation modes, so as to obtain a plurality of sample brain electrical signals induced by the visual stimulation modes, where each visual stimulation mode is used for representing a corresponding intention;

the feature analysis module 1902 is configured to perform feature analysis on a plurality of sample electroencephalograms by using an intention recognition manner to obtain a predicted intention of the first object;

the adjustment module 1903 is configured to adjust an intent recognition mode based on the intent and the predicted intent corresponding to the visual stimulus mode, where the adjusted intent recognition mode is used to predict the intent of a second object, and the second object includes the first object.

In some embodiments, fig. 20 is a block diagram of another intent recognition device provided in accordance with an embodiment of the present application. Referring to fig. 20, a signal acquisition module 1901 includes:

a display unit 19011 for displaying a plurality of visual stimulus modes and intentions corresponding to each visual stimulus mode;

The display unit 19011 is further configured to sequentially display multiple visual stimulus modes at different flicker frequencies, respectively;

and an acquisition unit 19012, configured to acquire a plurality of sample electroencephalograms corresponding to the visual stimulation mode during displaying any visual stimulation mode.

In some embodiments, with continued reference to fig. 20, a display unit 19011 is configured to display visual stimulus patterns with different flicker frequencies for any one of a plurality of visual stimulus patterns, respectively; for each flicker frequency, acquiring a sample brain electrical signal based on a visual stimulation mode displayed by the flicker frequency; and selecting the flicker frequency corresponding to the sample brain electrical signal with the highest signal amplitude as the flicker frequency of the visual stimulation mode.

In some embodiments, with continued reference to fig. 20, an acquiring unit 19012 is configured to acquire a plurality of sample electroencephalogram signals during displaying any of the visual stimulus modes; based on the functions of the individual brain regions in the human brain, a plurality of sample brain electrical signals acquired in the brain regions related to vision are acquired from the plurality of sample brain electrical signals.

In some embodiments, the visual stimulus mode display pattern includes at least one of:

The visual stimulus mode comprises a plurality of stripes, and at least one of the directions and the colors of the plurality of stripes are changed according to the flicker frequency of the visual stimulus mode;

the visual stimulus mode comprises a plurality of colors, and the colors are converted according to the flicker frequency of the visual stimulus mode;

the visual stimulus mode comprises a plurality of shapes, and the shapes are changed according to the flicker frequency of the visual stimulus mode;

the position of the visual stimulus pattern changes according to the flicker frequency of the visual stimulus pattern.

In some embodiments, with continued reference to fig. 20, a feature analysis module 1902 includes:

a first processing unit 19021, configured to perform preprocessing on the plurality of sample electroencephalograms, where the preprocessing is used to remove noise in the plurality of sample electroencephalograms;

the second processing unit 19022 is configured to perform feature analysis on the preprocessed plurality of sample electroencephalograms by using an intention recognition manner, so as to obtain a predicted intention of the first object.

In some embodiments, with continued reference to fig. 20, the first processing unit 19021 is configured to bandpass filter, for any sample electroencephalogram signal, the sample electroencephalogram signal to obtain an intermediate electroencephalogram signal, where the bandpass filter is used to eliminate interference caused by a device for acquiring the sample electroencephalogram signal; and performing independent component analysis on the intermediate electroencephalogram signal to obtain a preprocessed sample electroencephalogram signal, wherein the independent component analysis is used for eliminating interference caused by a first object when the sample electroencephalogram signal is acquired.

In some embodiments, with continued reference to fig. 20, an adjustment module 1903 is configured to adjust the intent recognition mode with a goal of maximizing a correlation between the sample electroencephalogram signal and a stimulation signal of the visual stimulation mode, where the intent of the prediction is the same as the intent of the visual stimulation mode, the stimulation signal being used to represent a flicker frequency and a display pattern of the visual stimulation mode; when the predicted intention is different from the intention corresponding to the visual stimulation mode, the intention recognition mode is adjusted with the aim of minimizing the correlation between the sample brain electrical signal and the stimulation signal of the visual stimulation mode.

In some embodiments, with continued reference to fig. 20, the signal acquisition module 1901 is further configured to perform visual stimulation on the first object for any one of a plurality of visual stimulation modes to obtain a plurality of sample biosignals induced by the visual stimulation mode, where the plurality of sample biosignals are at least one of an electrocardiograph signal and an electromyographic signal;

The feature analysis module 1902 is configured to perform feature analysis on the plurality of sample electroencephalograms and the plurality of sample biological signals by using an intention recognition manner, so as to obtain a predicted intention of the first object.

In some embodiments, with continued reference to fig. 20, the signal acquisition module 1901 is further configured to acquire a plurality of brain electrical signals of a second subject;

the feature analysis module 1902 is further configured to perform feature analysis on the plurality of electroencephalogram signals based on the adjusted intent recognition manner, so as to obtain a predicted intent of the second object.

The embodiment of the application provides an intention recognition device, which sequentially carries out visual stimulation on a first object by adopting a plurality of visual stimulation modes, obtains a plurality of sample electroencephalograms of the first object in the process based on any visual stimulation, carries out feature analysis on the plurality of sample electroencephalograms by adopting an intention recognition mode so as to predict the intention of the first object, and then compares the intention corresponding to the visual stimulation mode with the predicted intention, adjusts the intention recognition mode, so that the intention predicted by the intention recognition mode is more and more similar to the intention corresponding to the visual stimulation mode, namely, the intention recognition mode is optimized continuously, the intention predicted by the intention recognition mode is more and more accurate, and then predicts the intention of a second object by the optimized intention recognition mode, thereby effectively improving the accuracy of the intention recognition, and being beneficial to improving the efficiency and the user experience of man-machine interaction.

It should be noted that: the intent recognition device provided in the above embodiment only illustrates the division of the above functional modules when running an application, and in practical application, the above functional allocation may be performed by different functional modules according to needs, i.e., the internal structure of the device is divided into different functional modules to perform all or part of the functions described above. In addition, the intention recognition device and the intention recognition method provided in the above embodiments belong to the same concept, and detailed implementation processes of the intention recognition device and the intention recognition method are detailed in the method embodiments, and are not repeated here.

In the embodiment of the present application, the computer device may be configured as a terminal or a server, and when the computer device is configured as a terminal, the technical solution provided in the embodiment of the present application may be implemented by the terminal as an execution body, and when the computer device is configured as a server, the technical solution provided in the embodiment of the present application may be implemented by the server as an execution body, and also the technical solution provided in the present application may be implemented by interaction between the terminal and the server, which is not limited in this embodiment of the present application.

Fig. 21 is a block diagram of a terminal 2100 provided according to an embodiment of the present application. The terminal 2100 may be a portable mobile terminal such as: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, motion picture expert compression standard audio plane 3), an MP4 (Moving Picture Experts Group Audio Layer IV, motion picture expert compression standard audio plane 4) player, a notebook computer, or a desktop computer. Terminal 2100 may also be referred to by other names of user devices, portable terminals, laptop terminals, desktop terminals, and the like.

In general, the terminal 2100 includes: a processor 2101 and a memory 2102.

The processor 2101 may include one or more processing cores, such as a 4-core processor, an 8-core processor, or the like. The processor 2101 may be implemented in hardware in at least one of a DSP (Digital Signal Processing ), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array ). The processor 2101 may also include a main processor, which is a processor for processing data in an awake state, also called a CPU (Central Processing Unit ); a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 2101 may incorporate a GPU (Graphics Processing Unit, image processor) for taking care of rendering and drawing of content that the display screen is required to display. In some embodiments, the processor 2101 may also include an AI (Artificial Intelligence ) processor for processing computing operations related to machine learning.

Memory 2102 may include one or more computer-readable storage media, which may be non-transitory. Memory 2102 may also include high speed random access memory as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 2102 is used to store at least one computer program for execution by processor 2101 to implement the intent recognition method provided by the method embodiments herein.

In some embodiments, terminal 2100 may further optionally include: a peripheral interface 2103 and at least one peripheral. The processor 2101, memory 2102, and peripheral interface 2103 may be connected by a bus or signal line. The individual peripheral devices may be connected to the peripheral device interface 2103 by buses, signal lines or circuit boards. Specifically, the peripheral device includes: at least one of radio frequency circuitry 2104, a display screen 2105, a camera assembly 2106, audio circuitry 2107, and a power supply 2108.

The peripheral interface 2103 may be used to connect at least one Input/Output (I/O) related peripheral device to the processor 2101 and the memory 2102. In some embodiments, the processor 2101, memory 2102, and peripheral interface 2103 are integrated on the same chip or circuit board; in some other embodiments, either or both of the processor 2101, memory 2102, and peripheral interface 2103 may be implemented on separate chips or circuit boards, which is not limited in this embodiment.

The Radio Frequency circuit 2104 is used for receiving and transmitting RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 2104 communicates with a communication network and other communication devices via electromagnetic signals. The radio frequency circuit 2104 converts an electrical signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electrical signal. In some embodiments, the radio frequency circuit 2104 includes: antenna systems, RF transceivers, one or more amplifiers, tuners, oscillators, digital signal processors, codec chipsets, subscriber identity module cards, and so forth. The radio frequency circuitry 2104 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the world wide web, metropolitan area networks, intranets, generation mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and/or WiFi (Wireless Fidelity ) networks. In some embodiments, the radio frequency circuit 2104 may also include NFC (Near Field Communication ) related circuitry, which is not limited in this application.

The display screen 2105 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display 2105 is a touch screen, the display 2105 also has the ability to collect touch signals at or above the surface of the display 2105. The touch signal may be input to the processor 2101 as a control signal for processing. At this point, the display 2105 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display 2105 may be one, provided on the front panel of the terminal 2100; in other embodiments, the display 2105 may be at least two, respectively disposed on different surfaces of the terminal 2100 or in a folded design; in other embodiments, the display 2105 may be a flexible display disposed on a curved surface or a folded surface of the terminal 2100. Even more, the display 2105 may be arranged in a non-rectangular irregular pattern, i.e. a shaped screen. The display 2105 may be made of LCD (Liquid Crystal Display ), OLED (Organic Light-Emitting Diode) or other materials.

The camera assembly 2106 is used to capture images or video. In some embodiments, the camera assembly 2106 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal and the rear camera is disposed on the rear surface of the terminal. In some embodiments, the at least two rear cameras are any one of a main camera, a depth camera, a wide-angle camera and a tele camera, so as to realize that the main camera and the depth camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize a panoramic shooting and Virtual Reality (VR) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 2106 may also include a flash. The flash lamp can be a single-color temperature flash lamp or a double-color temperature flash lamp. The dual-color temperature flash lamp refers to a combination of a warm light flash lamp and a cold light flash lamp, and can be used for light compensation under different color temperatures.

The audio circuitry 2107 may include a microphone and a speaker. The microphone is used for collecting sound waves of users and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 2101 for processing, or inputting the electric signals to the radio frequency circuit 2104 for realizing voice communication. For the purpose of stereo acquisition or noise reduction, a plurality of microphones may be provided at different portions of the terminal 2100, respectively. The microphone may also be an array microphone or an omni-directional pickup microphone. The speaker is used to convert electrical signals from the processor 2101 or the radio frequency circuit 2104 into sound waves. The speaker may be a conventional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only the electric signal can be converted into a sound wave audible to humans, but also the electric signal can be converted into a sound wave inaudible to humans for ranging and other purposes. In some embodiments, the audio circuit 2107 may also include a headphone jack.

The power supply 2108 is used to supply power to the respective components in the terminal 2100. The power source 2108 may be alternating current, direct current, disposable battery, or rechargeable battery. When the power source 2108 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, terminal 2100 can further include one or more sensors 2109. The one or more sensors 2109 include, but are not limited to: an acceleration sensor 2110, a gyro sensor 2111, a pressure sensor 2112, an optical sensor 2113, and a proximity sensor 2114.

The acceleration sensor 2110 can detect the magnitudes of accelerations on three coordinate axes of the coordinate system established with the terminal 2100. For example, the acceleration sensor 2110 may be used to detect components of gravitational acceleration in three coordinate axes. The processor 2101 may control the display screen 2105 to display the user interface in either a landscape view or a portrait view based on gravitational acceleration signals acquired by the acceleration sensor 2110. The acceleration sensor 2110 may also be used for the acquisition of motion data of a game or a user.

The gyro sensor 2111 may detect a body direction and a rotation angle of the terminal 2100, and the gyro sensor 2111 may collect a 3D motion of the user on the terminal 2100 in cooperation with the acceleration sensor 2110. The processor 2101 may implement the following functions based on the data collected by the gyro sensor 2111: motion sensing (e.g., changing UI according to a tilting operation by a user), image stabilization at shooting, game control, and inertial navigation.

Pressure sensor 2112 may be provided on a side frame of terminal 2100 and/or on a lower layer of display 2105. When the pressure sensor 2112 is provided at a side frame of the terminal 2100, a grip signal of the user to the terminal 2100 may be detected, and left-right hand recognition or quick operation may be performed by the processor 2101 according to the grip signal collected by the pressure sensor 2112. When the pressure sensor 2112 is provided at the lower layer of the display screen 2105, the processor 2101 controls the operability control on the UI interface according to the pressure operation of the user on the display screen 2105. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

The optical sensor 2113 is used to collect the ambient light intensity. In one embodiment, the processor 2101 may control the display brightness of the display screen 2105 based on the intensity of ambient light collected by the optical sensor 2113. Specifically, when the intensity of the ambient light is high, the display brightness of the display screen 2105 is turned high; when the ambient light intensity is low, the display brightness of the display screen 2105 is turned down. In another embodiment, the processor 2101 may also dynamically adjust the shooting parameters of the camera assembly 2106 based on the intensity of ambient light collected by the optical sensor 2113.

The proximity sensor 2114, also called a distance sensor, is typically provided on the front panel of the terminal 2100. The proximity sensor 2114 is used to collect a distance between the user and the front surface of the terminal 2100. In one embodiment, when the proximity sensor 2114 detects that the distance between the user and the front surface of the terminal 2100 becomes gradually smaller, the processor 2101 controls the display 2105 to switch from the bright screen state to the off screen state; when the proximity sensor 2114 detects that the distance between the user and the front surface of the terminal 2100 gradually increases, the processor 2101 controls the display 2105 to switch from the off-screen state to the on-screen state.

It will be appreciated by those skilled in the art that the structure shown in fig. 19 does not constitute a limitation of the terminal 2100, and more or less components than those illustrated may be included, or some components may be combined, or a different arrangement of components may be employed.

The present application also provides a computer readable storage medium having stored therein at least one section of a computer program loaded and executed by a processor of a computer device to implement the operations performed by the computer device in the intent recognition method of the above embodiments. For example, the computer readable storage medium may be Read-Only Memory (ROM), random-access Memory (Random Access Memory, RAM), compact disc Read-Only Memory (Compact Disc Read-Only Memory, CD-ROM), magnetic tape, floppy disk, optical data storage device, and the like.

Embodiments of the present application also provide a computer program product comprising a computer program stored in a computer readable storage medium. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program so that the computer device performs the intention recognition method provided in the above-described various alternative implementations.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The foregoing description of the preferred embodiments is merely exemplary in nature and is in no way intended to limit the invention, since it is intended that all modifications, equivalents, improvements, etc. that fall within the spirit and scope of the invention.

Claims

1. A method of intent recognition, the method comprising:

2. The method of claim 1, wherein visually stimulating the first subject for any one of a plurality of visual stimuli to obtain a plurality of sample brain electrical signals induced by the visual stimulus, comprising:

displaying the multiple visual stimulus modes and the intentions corresponding to each visual stimulus mode;

sequentially displaying the plurality of visual stimulus modes at different flicker frequencies respectively;

and in the process of displaying any visual stimulation mode, acquiring a plurality of sample brain electrical signals corresponding to the visual stimulation mode.

3. The method of claim 2, wherein the flicker frequency is determined in the following manner:

for any one visual stimulation mode of a plurality of visual stimulation modes, respectively adopting different flicker frequencies to display the visual stimulation modes;

For each flicker frequency, acquiring a sample brain electrical signal based on a visual stimulation mode displayed at the flicker frequency;

and selecting the flicker frequency corresponding to the sample brain electrical signal with the highest signal amplitude as the flicker frequency of the visual stimulation mode.

4. The method according to claim 2, wherein the step of obtaining a plurality of sample brain electrical signals corresponding to any visual stimulus mode during the displaying of the visual stimulus mode includes:

in the process of displaying any visual stimulation mode, acquiring a plurality of sample brain electrical signals;

based on the functions of the brain regions in the human brain, a plurality of sample brain electrical signals acquired in the brain regions related to vision are acquired from the plurality of sample brain electrical signals.

5. The method of claim 1, wherein the visual stimulus mode display pattern comprises at least one of:

6. The method according to claim 1, wherein the performing feature analysis on the plurality of sample electroencephalogram signals by using an intention recognition method to obtain the predicted intention of the first object includes:

preprocessing the plurality of sample electroencephalograms, wherein the preprocessing is used for removing noise in the plurality of sample electroencephalograms;

and performing feature analysis on the preprocessed plurality of sample electroencephalograms by adopting the intention recognition mode to obtain the predicted intention of the first object.

7. The method of claim 6, wherein the preprocessing the plurality of sample brain electrical signals comprises:

for any sample brain electrical signal, carrying out band-pass filtering on the sample brain electrical signal to obtain an intermediate brain electrical signal, wherein the band-pass filtering is used for eliminating interference brought by equipment for collecting the sample brain electrical signal;

and performing independent component analysis on the intermediate electroencephalogram signals to obtain preprocessed sample electroencephalogram signals, wherein the independent component analysis is used for eliminating interference caused by the first object when the sample electroencephalogram signals are acquired.

8. The method according to claim 6, wherein the performing feature analysis on the preprocessed plurality of sample electroencephalograms by using the intention recognition method to obtain the predicted intention of the first object includes:

the method comprises the steps of obtaining stimulation signals of all visual stimulation modes, wherein the stimulation signals are used for representing flicker frequency and display patterns of the visual stimulation modes;

adopting typical association analysis to respectively determine the relativity of the sample brain electrical signals and each stimulation signal;

and taking the intention of the visual stimulus mode corresponding to the maximum correlation degree as the predicted intention of the first object.

9. The method of claim 1, wherein the adjusting the intent recognition mode based on the intent of the predicted intent corresponding to the visual stimulus mode comprises:

when the predicted intention is the same as the intention corresponding to the visual stimulation mode, aiming at maximizing the correlation degree between the sample brain electrical signal and the stimulation signal of the visual stimulation mode, adjusting the intention recognition mode, wherein the stimulation signal is used for representing the flicker frequency and the display mode of the visual stimulation mode;

And when the predicted intention is different from the intention corresponding to the visual stimulation mode, aiming at minimizing the correlation degree between the sample brain electrical signal and the stimulation signal of the visual stimulation mode, and adjusting the intention recognition mode.

10. The method according to claim 1, wherein the method further comprises:

for any one of the multiple visual stimulation modes, performing visual stimulation on the first object to obtain multiple sample biological signals induced by the visual stimulation mode, wherein the multiple sample biological signals are at least one of electrocardiosignals and electromyographic signals;

performing feature analysis on the plurality of sample electroencephalogram signals by adopting an intention recognition mode to obtain a predicted intention of the first object, wherein the method comprises the following steps:

and performing feature analysis on the plurality of sample electroencephalogram signals and the plurality of sample biological signals by adopting the intention recognition mode to obtain the predicted intention of the first object.

11. The method according to claim 1, wherein the method further comprises:

acquiring a plurality of electroencephalogram signals of the second object;

and carrying out feature analysis on the plurality of electroencephalogram signals based on the adjusted intention recognition mode to obtain the predicted intention of the second object.

12. An intent recognition device, the device comprising:

13. A computer device, characterized in that it comprises a processor and a memory for storing at least one piece of computer program, which is loaded by the processor and which performs the method of identifying an intention as claimed in any one of claims 1 to 11.

14. A computer readable storage medium, characterized in that the computer readable storage medium is for storing at least one piece of computer program for executing the intention recognition method of any one of claims 1 to 11.

15. A computer program product comprising a computer program which, when executed by a processor, implements the method of intention recognition as claimed in any one of claims 1 to 11.