CN117290709B

CN117290709B - Method, system, device and storage medium for continuous dynamic intent decoding

Info

Publication number: CN117290709B
Application number: CN202311585811.8A
Authority: CN
Inventors: 胡方扬; 魏彦兆; 李宝宝
Original assignee: Xiaozhou Technology Co ltd
Current assignee: Xiaozhou Technology Co ltd
Priority date: 2023-11-27
Filing date: 2023-11-27
Publication date: 2024-02-02
Anticipated expiration: 2043-11-27
Also published as: CN117290709A

Abstract

The application discloses a method, a system, equipment and a storage medium for continuous dynamic intention decoding, which are used for acquiring continuous electroencephalogram signals of a user in real time by utilizing an electrode array; interpolation processing is carried out on the continuous brain electrical signals to obtain target brain electrical signals; sliding window processing is carried out on the target electroencephalogram signal based on a preset time window, so that an electroencephalogram sequence matrix is obtained; extracting a plurality of features from an electroencephalogram feature matrix, and constructing a feature word set; determining a feature word weight sequence and a feature word vector matrix according to the feature word set; and inputting the feature word weight sequence and the feature word vector matrix into a preset cascade neural network for classification, and obtaining a classification result. The method for stably and accurately decoding the complex dynamic intention of the user is achieved by collecting, processing and analyzing continuous electroencephalogram signals and combining the feature extraction and the cascade neural network classification method.

Description

Method, system, device and storage medium for continuous dynamic intent decoding

Technical Field

The present application relates to the field of artificial intelligence, and in particular, to a method, system, apparatus, and storage medium for continuous dynamic intent decoding.

Background

With the development of information technology, man-machine interaction modes are continuously innovated so as to meet the requirements of people on more complex and intelligent interaction. An electroencephalogram (EEG) -based intent decoding technique has received attention as an interactive way to directly reflect the intent of a user. The EEG signals may capture brain activity contactlessly, reflecting the intention and cognitive state of the user. The existing EEG-based intention decoding technology mainly adopts a traditional machine learning method, and realizes intention recognition of several classifications by analyzing simple and stable electroencephalogram modes. However, the methods have low accuracy rate of intention recognition of complex dynamic changes, low stability and difficult adaptation to complex human-computer interaction scenes. The intention of the dynamic change corresponds to complex and changeable EEG modes, overlap exists between continuous intention, EEG signals are easy to be interfered by various types, the traditional classifier has weak new mode identification capability and poor generalization performance, the intention change cannot be tracked in real time, and the factors all make accurate and stable decoding of the dynamic intention face a plurality of challenges. Therefore, dynamic intent decoding is still a key difficulty of research, and a new technical scheme is necessary to be provided to realize stable and accurate decoding of complex dynamic intent of a user so as to further expand application of EEG technology in the field of intelligent interaction.

Disclosure of Invention

The embodiment of the application provides a method, a system, equipment and a storage medium for decoding continuous dynamic intention, which can solve the technical problem of how to stably and accurately decode complex dynamic intention of a user.

In a first aspect, embodiments of the present application provide a method for continuous dynamic intent decoding, including:

collecting continuous electroencephalogram signals of a user in real time by utilizing an electrode array;

performing interpolation processing on the continuous electroencephalogram signals to obtain target electroencephalogram signals;

sliding window processing is carried out on the target electroencephalogram signal based on a preset time window, so that an electroencephalogram sequence matrix is obtained;

extracting a plurality of features from the electroencephalogram feature matrix to construct a feature word set;

determining a feature word weight sequence and a feature word vector matrix according to the feature word set;

and inputting the characteristic word weight sequence and the characteristic word vector matrix into a preset cascade neural network for classification, and obtaining a classification result.

In some implementations of the first aspect, using the electrode array, continuous brain electrical signals of the user are acquired in real time, including:

according to the equipment specification parameters of the electrode array, adjusting the acquisition parameters of the electrode array to the highest acquisition rate and the highest resolution supported by the electrode array;

Starting the electrode array to collect continuous electroencephalogram signals of the user in real time, and monitoring the signal quality of the continuous electroencephalogram signals in real time based on a preset signal quality threshold, wherein the preset signal quality threshold comprises a resistance signal threshold, an impedance signal threshold and a noise signal threshold;

and if the signal quality of the continuous electroencephalogram signals does not accord with the preset signal quality threshold, carrying out abnormal repair on the electrode array.

In some implementations of the first aspect, interpolating the continuous electroencephalogram signal to obtain a target electroencephalogram signal includes:

detecting a missing segment in the continuous electroencephalogram signal, and performing linear interpolation on the missing segment to obtain linear interpolation;

performing wavelet transformation on the linear interpolation result, and extracting a low-frequency component wavelet signal and a high-frequency component wavelet signal;

performing Fourier inverse transformation and wavelet transformation on the high-frequency component wavelet signals to obtain high-frequency interpolation signals;

performing frequency multiplication on the low-frequency component wavelet signal to obtain a low-frequency interpolation signal;

and superposing the high-frequency interpolation signal and the low-frequency interpolation signal to obtain a target brain electrical signal.

In some implementations of the first aspect, performing sliding window processing on the target electroencephalogram signal based on a preset time window to obtain an electroencephalogram sequence matrix, including:

Performing adaptive segmentation on the target electroencephalogram signal to obtain an adaptive signal segment sequence; the adaptive signal segment sequence comprises a plurality of electroencephalogram signal segments;

presetting a candidate time window length set, wherein the candidate time window length set comprises a plurality of candidate time window lengths;

using a dynamic sliding window length algorithm, and automatically adjusting the time window length according to the content of signals in each electroencephalogram segment based on the preset candidate time window length set to obtain a target time window length corresponding to each electroencephalogram segment;

and carrying out sliding window segmentation on the electroencephalogram signal segments with the corresponding relation by utilizing the target time window length to obtain an electroencephalogram sequence matrix.

In some implementations of the first aspect, adaptively segmenting the target electroencephalogram signal to obtain an adaptive signal segment sequence includes:

calculating the derivative of each time point on the smoothed target electroencephalogram signal, and detecting derivative mutation points;

analyzing the baseline change before and after the derivative mutation and the spectral characteristic change, and judging the state change point corresponding to the target electroencephalogram signal after the smoothing treatment;

dividing at the detected state change point to obtain a self-adaptive signal paragraph;

For each segmented paragraph, calculating paragraph characteristics corresponding to each segmented paragraph;

calculating the feature similarity between the segmented paragraphs according to the paragraph features corresponding to each segmented paragraph;

and according to the feature similarity, merging the segments, and forming an adaptive signal segment sequence according to the segment merged segments.

In some implementations of the first aspect, extracting a plurality of features from the electroencephalogram feature matrix, constructing a feature vocabulary, includes:

for each electroencephalogram sequence matrix, carrying out feature extraction on the electroencephalogram sequence matrix based on a preset feature extraction algorithm to obtain single feature data corresponding to the electroencephalogram sequence matrix, wherein the feature data comprises time domain feature data, frequency domain feature data and/or statistical feature data;

for each two electroencephalogram sequence matrixes, performing feature extraction on a plurality of electroencephalogram sequence matrixes based on a preset correlation feature extraction algorithm to obtain correlation feature data among the plurality of electroencephalogram sequence matrixes, wherein the correlation feature data comprises time domain correlation feature data, frequency domain correlation feature data, mutual information feature data and/or correlation coefficient feature data, and the correlation feature data and the single feature data are used as electroencephalogram feature data of the continuous electroencephalogram signals

And constructing a characteristic word set according to the electroencephalogram characteristic data.

In some implementations of the first aspect, the preset cascade neural network includes an input layer, a three-level convolutional neural network, and an output layer;

inputting the feature word weight sequence and the feature word vector matrix into a preset cascade neural network for classification to obtain a classification result, wherein the classification result comprises the following steps:

receiving a characteristic word weight sequence and a characteristic word vector matrix according to the input layer;

classifying the characteristic word weight sequence and the characteristic word vector matrix according to the three-level convolutional neural network to obtain a classification result;

and outputting the classification result according to the output layer.

In a second aspect, embodiments of the present application provide a system for continuous dynamic intent decoding, comprising:

the acquisition module is used for acquiring continuous electroencephalogram signals of a user in real time by utilizing the electrode array;

the interpolation module is used for carrying out interpolation processing on the continuous electroencephalogram signals to obtain target electroencephalogram signals;

the sliding window module is used for carrying out sliding window processing on the target electroencephalogram signals based on preset time windows to obtain an electroencephalogram sequence matrix of each preset time window;

the extraction module is used for extracting a plurality of features from the electroencephalogram feature matrix and constructing a feature word set;

The determining module is used for determining a characteristic word weight sequence and a characteristic word vector matrix according to the characteristic word set;

In a third aspect, the present application also provides a computer device comprising a processor and a memory for storing a computer program which, when executed by the processor, implements the method for continuous dynamic intent decoding as defined in the first aspect.

In a fourth aspect, the present application also provides a computer readable storage medium storing a computer program which, when executed by a processor, implements the method for continuous dynamic intent decoding as described in the first aspect.

Compared with the prior art, the application has the following beneficial effects:

by utilizing the electrode array, continuous brain electrical signals of a user are collected in real time, and dynamic changes of brain activities can be captured in a non-contact mode, so that a continuous data source is provided for subsequent intention decoding; interpolation processing is carried out on the continuous electroencephalogram signals to obtain target electroencephalogram signals, so that missing parts in the electroencephalogram signals can be filled, and the integrity and quality of data are improved; based on a preset time window, sliding window processing is carried out on the target electroencephalogram signals to obtain an electroencephalogram sequence matrix, so that analysis and feature extraction of the electroencephalogram signals are facilitated; extracting a plurality of features from the electroencephalogram feature matrix, and constructing a feature word set for subsequent intention classification; determining a feature word weight sequence and a feature word vector matrix according to the feature word set to be used as an input preset cascade neural network; and inputting the feature word weight sequence and the feature word vector matrix into a preset cascade neural network for classification to obtain a classification result, and classifying continuous dynamic intentions to obtain a classification result. The method for stably and accurately decoding the complex dynamic intention of the user is achieved by collecting, processing and analyzing continuous electroencephalogram signals and combining the feature extraction and the cascade neural network classification method.

In addition, the preset cascade neural network of the embodiment of the application adopts a three-level deep network structure, so that characteristics of different semantic layers can be extracted, the recognition accuracy of complex dynamic intentions is remarkably improved, dynamic incremental learning is carried out by using the cascade network, feature expression is continuously optimized, classification results are corrected, the stability of intended decoding is greatly improved, continuous real-time intended decoding is supported, and the complex man-machine interaction requirements are better met.

Drawings

FIG. 1 is a flow diagram of a method for continuous dynamic intent decoding as illustrated in an embodiment of the present application;

FIG. 2 is a schematic diagram of a system for continuous dynamic intent decoding as illustrated in an embodiment of the present application;

fig. 3 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.

Referring to fig. 1, fig. 1 is a flowchart of a method for continuous dynamic intent decoding according to an embodiment of the present application. The method for continuous dynamic intent decoding of the embodiments of the present application can be applied to computer devices including, but not limited to, smart phones, notebook computers, tablet computers, desktop computers, physical servers, cloud servers, and the like. As shown in fig. 1, the method for continuous dynamic intent decoding of the present embodiment includes steps S101 to S106, which are described in detail below:

step S101, continuous brain electrical signals of a user are acquired in real time by utilizing an electrode array.

In this step, a flexible, adjustable electrode array is used as an electroencephalogram signal acquisition device, which can be easily attached to the scalp surface to prevent electrode movement and signal interruption, ensure stability of electrode position, so as to maintain continuity of signals during long-time acquisition and provide a stable signal acquisition environment. Each electrode is connected to a signal amplifier and transmits a signal through a guide wire into a receiver.

Optionally, in addition to the electroencephalogram signal, multimodal fusion is performed in combination with other various biological signals, such as eye movements, facial expressions, heart rate, etc. Through integrating and decoding the signals acquired by different sensors, the intention and the internal state of the user can be more comprehensively understood, and the accuracy and the adaptability of brain-computer interaction are improved.

In some embodiments, the step S101 includes:

In the embodiment, the high sampling rate and the high resolution can better capture and record the tiny change of the brain electrical signal, can provide richer and more accurate signal information, and ensures the fidelity of continuous signals. During the acquisition process, the quality of the electroencephalogram signals is monitored in real time, including the quality of detection electrodes and the stability of measurement signals. For example, by measuring the indexes of resistance, impedance, noise level, etc., problems that may cause signal interruption are found and handled in time.

Alternatively, the exception repair may include the steps of:

checking electrode connection: ensuring that all electrodes are in good contact with the skin without loosening or falling off. If the electrode connection is bad, the connection problem of the electrode is repaired.

Removing noise interference: noise signal interference is removed by filtering techniques such as low pass filtering and notch filtering.

Repositioning the electrode positions: if the signal quality problem is due to an electrode position error, the electrode position is repositioned to ensure the accuracy of the signal acquisition.

And (3) adjusting acquisition parameters: and readjusting acquisition parameters according to the equipment specification parameters and the signal quality requirements so as to optimize the signal quality.

The embodiment can improve the quality of continuous brain electrical signals acquired by the electrode array through abnormal repair so as to facilitate subsequent analysis and processing.

Step S102, interpolation processing is carried out on the continuous brain electrical signals, and target brain electrical signals are obtained.

In this embodiment, because a signal loss may occur in the signal acquisition process, in order to ensure signal integrity, to improve accuracy of subsequent intended decoding, interpolation processing is performed on the continuous electroencephalogram signal.

Alternatively, the interpolation process may be linear interpolation, spline interpolation, fourier interpolation, or Kriging interpolation. The linear interpolation is carried out according to the complete data points before and after the missing data, the signal change is linear, and the interpolation is carried out according to the linear relation of time or position. Spline interpolation interpolates by using polynomial functions between the multiple data points to approximate the true signal variation. The Fourier interpolation converts the signal into a frequency domain, and interpolation calculation is carried out according to frequency components of the frequency domain. The Kriging interpolation predicts signal values for unknown locations by analyzing the spatial distribution of known data.

Optionally, to reduce subsequent amounts of computation and noise interference, interference or noise components in the signal are removed or suppressed by signal filtering to extract the useful signal. Signal filtering methods include, but are not limited to, low pass filtering, high pass filtering, band reject filtering, median filtering, and adaptive filtering.

In some embodiments, the step S102 includes steps S1021 to S1025, which are specifically as follows:

and S1021, detecting a missing segment in the continuous electroencephalogram signal, and performing linear interpolation on the missing segment to obtain linear interpolation.

Optionally, detecting a missing segment in the continuous electroencephalogram signal, and performing linear interpolation on the missing segment to obtain linear interpolation, which specifically includes: firstly, setting a reasonable amplitude threshold for input continuous electroencephalogram signal data, and judging whether the amplitude of each sampling point is lower than the threshold. If the amplitude of a sample point is below a threshold, it is marked as a missing point. By traversing all the sampling points, all the missing points in the signal can be detected, and the missing points are connected into a section, namely the missing section needing interpolation. Then, for each determined missing segment, the last valid sample point at the beginning and the first valid sample point at the end of the segment are selected as two reference points for linear interpolation. Based on the values of the two reference points, a linear function can be established that passes through the two points. And then, in the missing section, sequentially taking the position of each sampling point needing interpolation, substituting the position into a linear function to calculate the corresponding amplitude value, and taking the amplitude value as the interpolation result of the point. After interpolation of all sampling points is carried out, linear interpolation signals of the whole missing section are obtained. And finally, traversing all the missing segments, and sequentially carrying out linear interpolation on each segment to finally obtain a linear interpolation result in the whole signal range.

Step S1022, wavelet transformation is carried out on the linear interpolation result, and a low-frequency component wavelet signal and a high-frequency component wavelet signal are extracted.

It is understood that the linear interpolation result is subjected to wavelet transformation, and a low-frequency component wavelet signal A1 and a high-frequency component wavelet signal A2 are extracted.

Optionally, wavelet transformation is performed on the linear interpolation result, and a low-frequency component wavelet signal and a high-frequency component wavelet signal are extracted, which specifically comprises: and selecting proper wavelet functions and parameters thereof according to the frequency range and analysis requirements of the electroencephalogram signals, and constructing a group of wavelet bases. The wavelet basis set is used as the basis function of wavelet transformation, and can carry out multi-level decomposition on signals. And then, performing wavelet transformation on the linear interpolation result by using the constructed wavelet base. The wavelet transformation performs multilayer differentiation on the signal, sequentially extracts contributions of different frequencies to the signal, and decomposes the signal into a low-frequency part and a multilayer high-frequency part. The low frequency part reflects the periodic components in the signal and the high frequency part reflects the details and transient components in the signal. Then, the low frequency component of the lowest frequency layer is taken as the low frequency wavelet signal to be extracted from the wavelet transformation result, and the high frequency component of the higher frequency layer is taken and combined into the high frequency wavelet signal to be extracted. Through the wavelet transformation process, the linear interpolation result is effectively decomposed into a low-frequency wavelet signal and a high-frequency wavelet signal, so that the two frequency components can be conveniently and respectively subjected to interpolation processing.

And step S1023, performing Fourier inverse transformation and wavelet transformation on the high-frequency component wavelet signals to obtain high-frequency interpolation signals.

It will be appreciated that the high frequency component wavelet signal A2 is subjected to inverse fourier transform and wavelet transform to obtain a high frequency interpolation signal B2'.

Optionally, performing inverse fourier transform and wavelet transform on the high-frequency component wavelet signal to obtain a high-frequency interpolation signal, which specifically includes: firstly, an input high-frequency component signal A2 needing interpolation is firstly subjected to Fourier inverse transformation, and is converted from a frequency domain to a time domain, so that a high-frequency time domain signal B2 after preliminary interpolation is obtained. Then, in order to supplement the possible loss of high frequency details in the signal B2, the signal B2 may be further subjected to wavelet decomposition, where db wavelet is adopted, and 3-4 layers of high frequency details signals with different frequency ranges, called D1, D2, D3, etc. may be decomposed; at the same time, each layer of decomposition will simultaneously obtain an approximation signal A3 at a low frequency. Next, each high-frequency detail signal is extracted separately, and a reconstruction process of wavelet transformation is performed. The reconstruction method may employ a sampled wavelet transform reconstruction algorithm that recovers the detail components of the signal by preserving the conjugate symmetry properties of the wavelet transform coefficients. Wavelet reconstruction is performed on each high-frequency detail signal D1, D2, D3 and the like, so as to obtain reconstructed high-frequency detail signals which are called RD1, RD2 and RD3. Then, all the reconstructed high-frequency detail signals RD1, RD2 and RD3 obtained in the previous step are injected into a layer-by-layer wavelet reconstruction process together with the low-frequency approximation signal A3. After wavelet reconstruction, an enhanced high-frequency time domain signal with complete interpolation called B2' can be synthesized. Through the series connection of the Fourier transform and the wavelet transform, the basic time-frequency structure is obtained by utilizing the Fourier transform, and the high-frequency details are extracted and reconstructed by the wavelet transform, so that the finally obtained high-frequency interpolation signal B2' is richer, more accurate and complete.

And step S1024, frequency multiplication is carried out on the low-frequency component wavelet signals to obtain low-frequency interpolation signals.

It will be appreciated that the low frequency component wavelet signal A1 is multiplied to obtain a low frequency interpolation signal B1 to supplement the missing low frequency component.

Optionally, the low-frequency component wavelet signal is multiplied to obtain a low-frequency interpolation signal, and the method specifically comprises the characteristic of periodicity and stability because A1 is a low-frequency signal extracted by wavelet decomposition. The varying characteristics of the missing segments can be seen as the absence of some sort of low frequency periodic signal. To restore this low frequency periodicity, interpolation can be performed using frequency multiplication. First a reference period is set, which can be set according to the periodic nature of the active signal in A1. And then carrying out periodic continuation on the missing segment according to the reference period, namely, carrying out periodic extension on the effective low-frequency periodic signal replication, and taking the result as an interpolation result of the missing segment. After such period extension, the missing segments can be filled with periodic signals. And finally, combining the processed low-frequency signals A1 to form a low-frequency interpolation signal B1 with the low-frequency interpolation completed. It should be noted that: the purpose of the inverse fourier transform is to convert the signal from the frequency domain back to the time domain. A1 is a time domain signal as a low frequency component extracted after wavelet transformation. The change of the low-frequency signal is relatively slow, and a relatively accurate low-frequency time domain interpolation result can be obtained through linear interpolation, so that the inverse Fourier transform of A1 is not needed.

Step S1025, superposing the high-frequency interpolation signal and the low-frequency interpolation signal to obtain a target electroencephalogram signal.

It can be understood that, by superposing the high-frequency interpolation signal B2' and the low-frequency interpolation signal B1, an electroencephalogram signal which is continuous in one time dimension and complete in frequency component can be obtained.

In this embodiment, since the electroencephalogram signal has both periodic and unsteady characteristics, that is, includes a stable low-frequency periodic signal and a highly variable high-frequency unsteady signal. In order to accurately recover the two types of signal characteristics, the embodiment combines linear interpolation and Fourier interpolation, namely, the low-frequency signal A1 extracted after wavelet decomposition is interpolated by adopting a linear interpolation method to recover the periodic low-frequency part of the electroencephalogram signal. The linear interpolation can utilize the periodic stability of the low-frequency signal to realize accurate interpolation. And performing Fourier inverse transformation on the high-frequency signal A2 extracted after wavelet decomposition to realize preliminary interpolation, thereby obtaining an initial high-frequency interpolation signal B2. To further improve the interpolation quality of the high frequency part, the signal B2 is continuously subjected to wavelet decomposition, and a plurality of high frequency detail component signals are extracted. And then, respectively carrying out wavelet transformation reconstruction on each high-frequency detail signal to recover the high-frequency detail. And synthesizing the reconstructed high-frequency detail signal and the reconstructed low-frequency signal, and performing wavelet inversion to obtain a complete high-frequency interpolation signal B2'. The serial application of the Fourier transform and the wavelet transform can better recover the unsteady state characteristic of the high-frequency signal. And finally, the low-frequency interpolation signal and the high-frequency interpolation signal are overlapped, so that the periodicity of the low-frequency part is reserved, the unsteady state characteristic of the high-frequency part is recovered, and a more accurate electroencephalogram signal interpolation result is obtained.

Step S103, sliding window processing is carried out on the target electroencephalogram signal based on a preset time window, and an electroencephalogram sequence matrix is obtained.

In this embodiment, the original electroencephalogram signal is unstable, and also fluctuates in the same state. To analyze the different states, a relatively stable electroencephalogram paragraph needs to be extracted. Fixed time window segmentation presents time scale selection difficulties, and window intervals often contain different steady states at the same time. The self-adaptive segmentation can be carried out according to the state change points of the electroencephalogram signals, so that paragraphs with longer duration and relatively stable internal characteristics are obtained. And relatively stable paragraphs in the electroencephalogram signals are obtained through self-adaptive segment segmentation so as to better reflect different stable states. The paragraphs better preserve certain brain electrical states, are beneficial to the subsequent extraction of time features and frequency domain features of the paragraphs, and are used for expressing certain states and carrying out pattern recognition. Compared with a fixed window, the self-adaptive segmentation provides better segment extraction, and lays a foundation for subsequent recognition analysis.

In some embodiments, the step S103 includes steps S1031 to S1034, which are specifically as follows:

step S1031, carrying out self-adaptive segmentation on the target electroencephalogram signal to obtain a self-adaptive signal segment sequence; the adaptive signal segment sequence includes a plurality of electroencephalogram signal segments.

Optionally, the step S1031 includes steps S1031-1 to S1031-6, specifically as follows:

and step S1031-1, calculating the derivative of each time point on the target electroencephalogram signal after the smoothing processing, and detecting a derivative mutation point.

Illustratively, the derivative is calculated point by point on the smoothed electroencephalogram signal. The derivative may be calculated approximately by a central difference method, specifically, the smoothed electroencephalogram signal is expressed as a time sequence { x (i) }, i represents the i-th sampling point, and the derivative approximation (i) of the i-th point is calculated as (x (i+1) -x (i-1))/(2 x sampling intervals). The sampling interval is a time interval of data acquisition. The smoothing process can adopt methods such as moving average smoothing or Savitzky-Golay filtering, and the like, so that the influence of noise on derivative calculation can be restrained. And setting a proper threshold value as a derivative mutation judgment threshold value, and marking a point as a derivative mutation point when the absolute value of the derivative of the point is larger than the threshold value. And judging point by point on the whole signal sequence, and detecting all derivative mutation points. These abrupt points are possible state change points, but it is necessary to further judge the front and rear characteristics thereof to confirm whether they are actual state change points.

And step S1031-2, analyzing the baseline change and the spectral characteristic change before and after derivative mutation, and judging the state change point corresponding to the target electroencephalogram signal after smoothing.

Illustratively, after detecting the derivative mutation point, it is necessary to further determine whether there is a significant change in the signal characteristics before and after it to confirm whether it is a state change point. The method mainly comprises the steps of analyzing and judging from the two aspects of baseline change and spectrum characteristic change, calculating signal average values in a certain time window before and after a derivative mutation point, representing a baseline level, judging whether the average values have significant differences, and supporting the point as a state change point if obvious baseline level jump exists. Meanwhile, power spectrum characteristics in windows before and after the mutation point are respectively calculated, and whether characteristic parameters such as total power, main frequency band energy distribution, dominant frequency and the like are obviously changed before and after the mutation point is judged. If there is a significant change in the spectral content, this point is also supported as a state change point. And finally, whether obvious differences appear at the same time in the two aspects of the comprehensive baseline change and the spectrum change or not is confirmed, and whether the derivative mutation point corresponds to a real transition point of the brain electrical state or not is confirmed.

Step S1031-3, dividing at the detected state change point to obtain an adaptive signal paragraph.

Illustratively, at each detected state change point (the derivative mutation point identified by step S1031-2), the electroencephalogram signal is segmented, and the signal between two adjacent state change points is segmented into an independent paragraph with the state change point as a boundary. Thus, the signals are segmented and segmented at the state transformation moment, and a series of electroencephalogram signal paragraph sequences with different lengths and relatively stability can be obtained. Because the segmentation is performed at the moment when the signal state is obviously changed, compared with a fixed time window, the segmentation mode is performed more adaptively according to the signal state change characteristics, so that a series of signal paragraphs which are more in line with the time structure characteristics of the electroencephalogram signals are obtained.

Step S1031-4, for each segmented paragraph, calculating paragraph characteristics corresponding to each segmented paragraph.

Illustratively, for each electroencephalogram segment obtained in step S1031-3, relevant features of the segment are analyzed and extracted for subsequent judgment of segment similarity and determination of whether merging of segments is required. The characteristics which can be analyzed comprise the time duration of the paragraph, the time duration of the state maintenance, the time domain mean value and variance of the paragraph signals, the signal intensity distribution, the frequency domain power spectrum characteristics of the paragraph, the wavelet transformation coefficients of the paragraph and the time-frequency characteristics. The features can express the feature signature of each paragraph in the form of vectors, and judge the similarity between two paragraphs to find out paragraphs with similar features.

Step S1031-5, calculating the feature similarity between the segmented paragraphs according to the paragraph features corresponding to each segmented paragraph.

Illustratively, the distance or correlation between the features of each paragraph obtained in the previous step can be calculated to find out the paragraphs with high similarity of the features. For segments with very short duration or unstable features, it is stated that they are not a steady state, likely a false segmentation caused by noise or artifacts.

Step S1031-6, according to the feature similarity, combining the segments, and forming an adaptive signal segment sequence according to the segments after the segment combination.

In this embodiment, to avoid over-segmentation, the too short or unstable segments may be merged into the adjacent segments with the highest similarity, so that the segmentation is more reasonable. This step is repeated until each segment meets the time and feature stability requirements. Finally, an adaptive electroencephalogram signal segmentation result after optimization processing is obtained, and segment error segmentation conditions possibly generated by segmentation only according to state points are improved.

Step S1032, presetting a candidate time window length set, wherein the candidate time window length set includes a plurality of candidate time window lengths.

It should be noted that, based on the adaptive segmentation result in step S1031, sliding window processing is performed on each adaptive segment to obtain an electroencephalogram sequence matrix divided at a certain time interval. In steps S1032 to S1034, each adaptive segmentation is further segmented into a plurality of sub-segments using a sliding window operation, and intent classification is performed as input. The sliding window size can be automatically adjusted according to the signal content in each segment by using a dynamic sliding window length algorithm, namely, a plurality of candidate sliding window lengths are set, the optimal length of the current sliding window is calculated according to the signal complexity of the previous sliding window, and the process is repeated to dynamically adjust the length of each sliding window, so that the self-adaptive optimization of the sliding window processing is realized.

Optionally, before the dynamic sliding window length algorithm starts, a plurality of candidate sliding window lengths need to be preset as candidate sets for calculating the current sliding window optimal length subsequently. The candidate sliding window length may be set as a fixed array of values, where the values are set according to the actual situation, typically selecting several representative time interval lengths. For example, the candidate sliding window length array may be set to [50ms,100ms,150ms,200ms ], which several values cover the possible typical sliding window range.

Step S1033, using a dynamic sliding window length algorithm, based on the preset candidate time window length set, automatically adjusting the time window length according to the signal content in each electroencephalogram segment, so as to obtain a target time window length corresponding to each electroencephalogram segment.

Optionally, the step S1033 includes:

after the adaptive segmentation is completed, a plurality of electroencephalogram signal segmented data are obtained. In order to calculate the optimal length of each sliding window, the signal complexity of each segment needs to be determined. For this purpose, for the first segment, electroencephalogram data is collected, three characteristic parameters of time domain entropy, frequency domain entropy and peak factor are calculated, and a complexity vector P of the segment is formed. 1) Time domain entropy, which reflects the complexity of the signal time sequence. By calculating the probability of each sample occurrence, the information entropy p1= - Σp (i) logp (i) is obtained, wherein p (i) =n (i)/N, N (i) represents the occurrence number of the ith sampling value in the window, and N is the window length. 2) And (3) frequency domain entropy, namely performing FFT on the signals, converting the signals into a frequency domain, and calculating the information entropy of the frequency spectrum to reflect the complexity of the frequency domain. P2= - Σp (f) lovp (f), where P (f) = |x (f) |2/Σ|x (f) |2, |x (f) | is the FFT magnitude of the f-th frequency component. 3) Peak factor, the degree of abrupt change in the response signal waveform. P3=max (x)/RMS (x), max (x) represents the maximum value of the signal waveform x, i.e., the maximum peak value in the waveform curve. RMS (x), which is the root mean square value of x, is an index for evaluating the smoothness of a waveform curve. RMS (x) =sqrt (Σ (x (i) 2)/N), where x (i) represents the i-th sampling point, N is the total sampling point number, and sqrt represents the open root number. RMS reflects the energy level of the waveform as a whole. The three parameters are combined into a three-dimensional vector P= [ P1, P2, P3], wherein P1-P3 respectively correspond to the three features. The complexity vector P may thus reflect the complexity of the current window signal from both the time and frequency domain angles.

After the complexity vector P of the first segment is calculated, the complexity vector P is used to determine the optimal length l of the first sliding window. The specific calculation process is to set the candidate sliding window length set to l= [ L1, L2,..ln ], where li represents the i-th candidate length. Calculating the linear correlation between P and each candidate length li, resulting in a correlation array r= [ R1, R2,..rn ], where ri represents the correlation coefficient of P and li, which can be calculated using pearson correlation coefficients. In the correlation array R, finding the maximum value rm and the corresponding index m, and Lm is the sliding window length which is most matched with P. Lm is taken as the optimal length of the first sliding window, i.e. l=lm. If a plurality of correlation values are the same and all maximum, taking the average value of the candidate lengths as l, or selecting a better value according to actual needs. Through this process, the complexity vector P of the first segment may be matched, and the optimal length l of the first sliding window may be determined from the candidate set.

Step S1034, performing sliding window segmentation on the electroencephalogram signal segments with the corresponding relationship by using the target time window length, to obtain an electroencephalogram sequence matrix.

Optionally, in the previous step, the optimal sliding window length l1 of the first sliding window has been calculated from the signal complexity vector P1 of the first segment. Then, the determined sliding window length l1 is used as the size of a first sliding window, and the first segmentation segment is subjected to sliding window segmentation to obtain a first electroencephalogram signal sliding window w1. And repeating the steps 3 and 4, namely carrying out signal complexity analysis on the first sliding window w1 to obtain a complexity vector P1, carrying out matching calculation based on signals of the second division section to determine the optimal sliding window length l2 of the second sliding window w2, taking the l2 as the size of the second sliding window w2, and carrying out sliding window division on the second division section to obtain the second sliding window w2. And calculating the optimal length l3 of the third sliding window w3 based on the third segmentation segment. And so on, each time the optimal length of the current sliding window is calculated according to the signal complexity of the previous sliding window. In this way, the size of each sliding window can be dynamically optimized, enabling adaptation of the sliding window length.

In the embodiment of the application, the following beneficial effects can be achieved:

improving the adaptivity of feature extraction: the dynamic sliding window can be adjusted according to the complexity of the signal, enough information is captured by using a larger window in a complex area, the fineness is improved by using a small window in a simple area, and the flexibility of feature extraction is improved as a whole. The dependence of manual setting is reduced: the window size is automatically optimized by the algorithm, the optimal value is not required to be set by artificial prediction, the pre-judging error is reduced, and the window size is closer to the real optimal point. Reducing the risk of overfitting: the fixed window is easy to excessively adapt to a certain feature scale, and the dynamic window can enhance the adaptability of the model to multi-scale features and improve generalization capability. Treatment of different individual differences: the complexity distribution of the signals collected by different individuals may be different, and the dynamic window can adapt to specific situations.

It should be noted that, in this embodiment, the sliding window operation is performed through a plurality of time windows, so that the correlation between a plurality of window sequences is extracted subsequently, so that the correlation between continuous intentions is extracted, and further, the accuracy of continuous dynamic intent decoding can be improved.

Step S104, extracting a plurality of features from the electroencephalogram feature matrix, and constructing a feature word set.

Optionally, the step S104 includes:

for each two electroencephalogram sequence matrixes, carrying out feature extraction on a plurality of electroencephalogram sequence matrixes based on a preset correlation feature extraction algorithm to obtain correlation feature data among the plurality of electroencephalogram sequence matrixes, wherein the correlation feature data comprises time domain correlation feature data, frequency domain correlation feature data, mutual information feature data and/or correlation coefficient feature data, and the correlation feature data and the single feature data are used as electroencephalogram feature data of the continuous electroencephalogram signals;

In this embodiment, optionally, the time domain feature extraction algorithm is mainly applied to the time domain signal, and is used to describe the temporal feature of the signal. Time domain feature extraction algorithms include, but are not limited to:

mean (Mean) = (x1+x2+ … +xn)/n, where xi represents the i-th sample point and n represents the number of sample points.

Variance (Variance): σ= ((x 1- μ) x+ (x 2- μ) x+ … + (xn- μ) j,)/n, where, mu represents the average value, xi represents the ith sample point, and n represents the number of sample points.

Standard deviation (standard device): σ= v (σ (j)), where σ represents variance.

Maximum value (Maximum): max=max (x 1, x2, …, xn), where xi represents the i-th sample point.

Minimum (Minimum): min=min (x 1, x2, …, xn), where xi represents the i-th sample point.

Slope (Slope): slope= (xn-x 1)/(n-1), where xi represents the i-th sample point and n represents the number of sample points.

Optionally, a frequency domain feature extraction algorithm is used to extract relevant features from the spectral distribution of the signal, including but not limited to:

mean frequency (MeanFrequency): mf=Σ (fi×pi)/Σ (Pi), where fi represents the i-th frequency point and Pi represents the power spectral density of the corresponding frequency point.

Energy band ratio (BandPowerRatio): bp=Σ (Pi)/Σ (Pj), where Pi and Pj represent power spectral densities in different frequency band ranges, the energy distributions of the different frequency bands can be compared.

Variance frequency (VarianceFrequency): vf=Σ ((fi-mf) Pi)/Σ (Pi), where fi represents the i-th frequency point, pi represents the power spectral density of the corresponding frequency point, and mf represents the mean frequency.

Peak of spectrum (peakamplite): pk=max (ASD), where ASD represents the amplitude spectral density, taking the maximum value of the spectrum.

Frequency characteristic percentage (frequency featurescore): ffp = Σ (Pi)/Σ (ASD), where Pi represents the power spectral density of the corresponding frequency bin and ASD represents the amplitude spectral density.

Optionally, the correlation features between the electroencephalogram sequences include, but are not limited to:

time domain correlation: and calculating the time domain correlation among different electroencephalogram sequences, including a cross-correlation function, an autocorrelation function, a correlation coefficient and the like, so as to measure the linear or nonlinear relation among the electroencephalogram signals.

Frequency domain correlation: the relation between the electroencephalogram sequences is described by calculating the frequency domain correlation in different frequency bands. For example, correlation (coherence) or phase synchronization index (phase synchronization index) can be used to measure correlation over different frequency bands.

Mutual information: mutual information (mutual information) is used as an index to evaluate the correlation between brain electrical sequences. Mutual information can measure the degree of uncertainty reduction between two signals, reflecting their correlation.

Correlation coefficient: including correlation methods (e.g., pearson correlation coefficients, spearman correlation coefficients), coherence methods, granger causality, and the like.

And classifying and summarizing the three types of features, and constructing a feature word set according to the standards of physical significance, statistical significance and the like of the features. Specific: firstly, the extracted time domain features are classified and summarized, wherein the average amplitude reflects signal intensity, which can be summarized as a feature word of 'signal intensity', the variance reflects signal stability, is classified as 'stability', the waveform length reflects waveform span, is classified as 'span', the kurtosis reflects the degree of waveform kurtosis, is classified as 'kurtosis', and the kurtosis also reflects the condition of waveform kurtosis. And secondly, classifying and summarizing the frequency domain features, namely, classifying the energy spectrum into a ' distribution ', classifying the power spectrum into a ' strength ', classifying the power spectrum into a ' time-frequency feature, and classifying the wavelet transform into a ' time-frequency feature '. Again, the correlation features are categorized, i.e. pearson correlation coefficients reflect a linear correlation between the two signals, which is categorized as a "linear correlation". Finally, the three types of feature words are unified and summarized, necessary feature words are added, a final feature word set such as 'signal intensity', 'stability', 'span', 'kurtosis', 'distribution', 'time-frequency feature', 'linear correlation' and the like is constructed, and therefore a feature word set containing various features of different time window sequences is constructed, and input is provided for subsequent vectorization representation and network classification learning.

Step S105, determining a feature word weight sequence and a feature word vector matrix according to the feature word set.

Optionally, determining a feature word vector matrix according to the feature word set includes: after the feature word set is constructed, it is further vectorized by word vector techniques for processing and classification by the machine learning model. Firstly, constructing a feature word dictionary matrix with the size of m multiplied by n, wherein the row number m of the matrix is the number of feature words in a feature word set, and the column number n is the vector dimension of each feature word, and generally takes 100-300 dimensions. Each vector in the matrix is used to represent a word vector for the corresponding feature word. After the initialization of the dictionary matrix is completed, a word vector training algorithm such as CBOW or Skip-Gram is used for the feature word set, and the word vector of each feature word, namely the vector corresponding to each feature word in the matrix, is learned through feature word context training. The training goal is to make the word vectors corresponding to the feature words appearing in similar contexts closer and the word vectors corresponding to the feature words appearing in different contexts farther apart. After multiple training iterations, a stable feature word set dictionary matrix V is obtained, each feature word is mapped to a word vector with fixed dimension, so that vectorization representation of the feature word set is completed, and the feature word set dictionary matrix V is converted into a feature vector matrix form acceptable by machine learning.

Optionally, determining a feature word weight sequence according to the feature word set includes: and determining a feature word weight sequence according to the expression of each feature word in the feature word set in different time windows. Specific: after the feature word set f= { F1, F2, & gt. fM } is built and vectorized, the weight wi of each feature word needs to be determined to represent the contribution of the feature word to the decoding of the user intention due to the fact that the importance of different feature words fi to reflect the user intention is inconsistent.

The specific method is that for each feature word fi, the expression value sequences of the feature word fi on N time windows are counted:

fi: [ni,1, ni,2, ..., ni,N]；

wherein ni, j can be represented by the statistic value of the feature corresponding to fi in the j window, such as mean, variance, etc.;

then normalize the above-mentioned expression value sequence, map to [0,1] range fi: [ N;

then, the average value of the normalized sequence is calculated as the weight wi, wi= (1/N) ×Σ (n≡i, j);

the above calculation is repeated to obtain a weight sequence of all feature words, w= [ W1, W2, ], wM.

And S106, inputting the feature word weight sequence and the feature word vector matrix into a preset cascade neural network for classification, and obtaining a classification result.

The preset cascade neural network is a cascade network module and comprises an input layer, a three-level convolutional neural network and an output layer. The classification result refers to user intent semantics.

Optionally, the step S106 includes:

and outputting the classification result according to the output layer.

It can be understood that the feature word weight sequence W and the feature word vector matrix V are input into a three-level network for classification, and a first-level, a second-level and a third-level classification result is obtained.

In this embodiment, the entire cascade network module includes an input layer, a three-level convolutional neural network, and an output layer;

the input layer receives the characteristic word weight sequence W= [ W1, W2, ], wm and characteristic word vector matrix V= [ V1, V2, ], vm ] T, wherein T represents the transposition operation of the matrix V, m is the number of characteristic words, vi epsilon Rn is the word vector of the ith characteristic word, and n is the word vector dimension.

And 2, three-level convolutional neural networks, namely, convolutional neural networks with three identical structures, wherein each network consists of a convolutional layer, an activation function layer and a pooling layer.

The three-stage convolutional neural network is specifically described as follows:

(1) A first level network, comprising:

a convolution layer, wherein a one-dimensional convolution with the convolution kernel size of h multiplied by n is used for carrying out convolution operation on the feature word vector matrix V to obtain a feature graph Y1= [ Y1, Y1,2, ], Y1, k ], wherein k is the number of first-stage convolution kernels, and Y1, j is the response output of the jth convolution kernel. The convolution operation can be expressed as y1, j=σ (w 1, j v+b1, j) σ is the activation function, w1, j and b1, j being the convolution kernel weight and offset, respectively;

and an activation function layer, wherein nonlinear activation functions such as ReLU and the like are adopted to obtain a first-stage network output activation characteristic A1.

-pooling layer, performing operations such as maximum pooling on the activation feature A1, and reducing the dimension to obtain a pooled feature P1.

(2) The second level network, carry on the characteristic extraction on the basis of the output of the first level network, include:

the convolution layer is used for carrying out feature extraction by using a convolution kernel on the basis of the output of the first-stage network to obtain a feature map Y2;

an activation layer, which performs nonlinear activation processing such as ReLU on the feature map Y2 to obtain an activation feature A2;

-a pooling layer for performing operations such as maximum pooling on the activation feature A2 to obtain a pooled feature P2;

in order to enhance the dependency modeling and semantic analysis capability of the network on the time series electroencephalogram signals, a transition layer and dense blocks are added into the second-level network. The transition layer performs feature mapping, fusion and dimension reduction, dense blocks construct dense connection of the convolution layer, each layer is connected with all the previous layers, and the capability of capturing the time dependence of the electroencephalogram signals by the network is enhanced. The output of the transition layer is used as the input of the next stage network to realize the connection of the multi-stage network. Specific: and processing the pooled feature P2 by using a plurality of convolution layers of the dense block to construct a dense connection structure, wherein the dense block expression is H= [ P2, f (P2), f ([ P2, f (P2) ]), and the dense block output H is subjected to feature downsampling and downsampling by using the Transition layer, wherein the Transition layer expression is Y_ { Transition } = Transition (H) = proportioning (convolution (H)), wherein the Transition represents overoperation, Y represents user intention data, proportioning represents pooling operation, and proportioning represents convolution operation, and the Transition layer output is used as the input of a third-level network.

(3) And the third level network is the same as the above, and a feature map Y3, an activation feature A3 and a pooling feature P3 are obtained.

3 output layer, comprising:

-the first-level network outputs a first-level classification result O1, coarse-grained classification;

-the second level network outputs a second level classification result O2, medium granularity classification;

-the third level network outputs a third level classification result O3, fine-grained classification;

finally, the cascade connection of the three-level classification network realizes the feature learning from thick to thin, and the high-level network adjusts and infers on the basis of the low-level network to finally obtain the accurate identification of the continuous intention.

Illustratively, assume that we want to implement a smart home system, controlling various appliances through voice commands. In order to accurately understand instruction semantics, a three-level cascade network structure is adopted, wherein a first-level network identifies basic control actions such as opening and closing, a second-level network identifies target household appliances such as air conditioners and televisions based on first-level output, and a third-level network identifies parameters such as temperature 26 degrees and volume increment based on second-level output. And finally integrating three-level network results, and completely analyzing instruction semantics, such as opening (action) +air conditioner (target) +temperature 26 degrees (parameters). It can be difficult to identify action, target, and parameter multi-granularity semantics simultaneously if only a single network. The cascade network analyzes the semantics step by step, thereby accurately controlling the state of the target household appliance.

In the embodiment, the three-level deep network structure is adopted, so that the characteristics of different semantic levels can be extracted, the recognition accuracy of complex dynamic intentions is remarkably improved, the cascade network is utilized for dynamic increment learning, the characteristic expression is continuously optimized, the classification result is corrected, the stability of intended decoding is greatly improved, continuous real-time intended decoding is supported, and the complex man-machine interaction requirement is better met.

It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic of each process, and should not limit the implementation process of the embodiment of the present application in any way.

In order to execute the method for continuous dynamic intent decoding corresponding to the method embodiment, corresponding functions and technical effects are realized. Referring to fig. 2, fig. 2 shows a block diagram of a system for continuous dynamic intent decoding according to an embodiment of the present application. For convenience of explanation, only a portion related to the present embodiment is shown, and the electroencephalogram-based intent decoding system provided in the embodiment of the present application includes:

the acquisition module 201 is used for acquiring continuous electroencephalogram signals of a user in real time by utilizing the electrode array;

The interpolation module 202 is configured to perform interpolation processing on the continuous electroencephalogram signals to obtain target electroencephalogram signals;

the sliding window module 203 is configured to perform sliding window processing on the target electroencephalogram signals based on a preset time window, so as to obtain an electroencephalogram sequence matrix of each preset time window;

the extracting module 204 is configured to extract a plurality of features from the electroencephalogram feature matrix, and construct a feature word set;

a determining module 205, configured to determine a feature word weight sequence and a feature word vector matrix according to the feature word set;

and the classification module 206 is configured to input the feature word weight sequence and the feature word vector matrix to a preset cascade neural network for classification, so as to obtain a classification result.

In some embodiments, the acquisition module 201 is specifically configured to:

In some embodiments, the interpolation module 202 is specifically configured to:

In some embodiments, the sliding window module 203 is specifically configured to:

In some embodiments, the sliding window module 203 is further specifically configured to:

In some embodiments, the extracting module 204 is specifically configured to:

In some embodiments, the preset cascade neural network comprises an input layer, a three-level convolutional neural network, and an output layer; the classification module 206 is specifically configured to:

and outputting the classification result according to the output layer.

The system for continuous dynamic intent decoding described above may implement the method for continuous dynamic intent decoding of the method embodiments described above. The options in the method embodiments described above are also applicable to this embodiment and will not be described in detail here. The rest of the embodiments of the present application may refer to the content of the method embodiments described above, and in this embodiment, no further description is given.

Fig. 3 is a schematic structural diagram of a computer device according to an embodiment of the present application. As shown in fig. 3, the computer device 3 of this embodiment includes: at least one processor 30 (only one is shown in fig. 3), a memory 31 and a computer program 32 stored in the memory 31 and executable on the at least one processor 30, the processor 30 implementing the steps in any of the method embodiments described above when executing the computer program 32.

The computer device 3 may be a smart phone, a tablet computer, a desktop computer, a cloud server, or other computing devices. The computer device may include, but is not limited to, a processor 30, a memory 31. It will be appreciated by those skilled in the art that fig. 3 is merely an example of the computer device 3 and is not meant to be limiting as the computer device 3, and may include more or fewer components than shown, or may combine certain components, or different components, such as may also include input-output devices, network access devices, etc.

The processor 30 may be a central processing unit (Central Processing Unit, CPU), the processor 30 may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 31 may in some embodiments be an internal storage unit of the computer device 3, such as a hard disk or a memory of the computer device 3. The memory 31 may in other embodiments also be an external storage device of the computer device 3, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like, which are provided on the computer device 3. Further, the memory 31 may also include both an internal storage unit and an external storage device of the computer device 3. The memory 31 is used for storing an operating system, application programs, boot loader (BootLoader), data, other programs etc., such as program codes of the computer program etc. The memory 31 may also be used for temporarily storing data that has been output or is to be output.

In addition, the embodiment of the present application further provides a computer readable storage medium, where a computer program is stored, where the computer program is executed by a processor to implement the steps in any of the above-mentioned method embodiments.

The present embodiments provide a computer program product which, when run on a computer device, causes the computer device to perform the steps of the method embodiments described above.

In several embodiments provided herein, it will be understood that each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.

The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device to perform all or part of the steps of the method described in the various embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The foregoing embodiments have been provided for the purpose of illustrating the objects, technical solutions and advantages of the present application in further detail, and it should be understood that the foregoing embodiments are merely examples of the present application and are not intended to limit the scope of the present application. It should be noted that any modifications, equivalent substitutions, improvements, etc. made by those skilled in the art, which are within the spirit and principles of the present application, are intended to be included within the scope of the present application.

Claims

1. A method for continuous dynamic intent decoding, comprising:

extracting a plurality of features from the electroencephalogram sequence matrix, and constructing a feature word set;

inputting the characteristic word weight sequence and the characteristic word vector matrix into a preset cascade neural network for classification to obtain a classification result;

the preset cascade neural network comprises an input layer, a three-level convolutional neural network and an output layer;

outputting the classification result according to the output layer;

performing interpolation processing on the continuous electroencephalogram signals to obtain target electroencephalogram signals, wherein the interpolation processing comprises the following steps:

detecting a missing segment in the continuous electroencephalogram signal, and performing linear interpolation on the missing segment to obtain a linear interpolation result;

2. The method for continuous dynamic intent decoding as claimed in claim 1, wherein continuous electroencephalogram signals of a user are acquired in real time using an electrode array including:

3. The method for continuous dynamic intent decoding as claimed in claim 1, wherein the sliding window processing is performed on the target electroencephalogram signal based on a preset time window to obtain an electroencephalogram sequence matrix, including:

4. A method for continuous dynamic intent decoding as claimed in claim 3, wherein adaptively segmenting the target brain electrical signal results in a sequence of adaptive signal segments including:

5. The method for continuous dynamic intent decoding as claimed in claim 1, wherein extracting a plurality of features from the electroencephalogram sequence matrix, constructing feature word sets, includes:

6. A system for continuous dynamic intent decoding, comprising:

the acquisition module is used for acquiring continuous brain electrical signals of a user in real time by utilizing the electrode array;

the extraction module is used for extracting a plurality of features from the electroencephalogram sequence matrix and constructing a feature word set;

the classification module is used for inputting the characteristic word weight sequence and the characteristic word vector matrix into a preset cascade neural network for classification to obtain a classification result;

the preset cascade neural network comprises an input layer, a three-level convolutional neural network and an output layer; the classification module is specifically configured to:

outputting the classification result according to the output layer;

the interpolation module is specifically configured to:

7. A computer device comprising a processor and a memory for storing a computer program which, when executed by the processor, implements the method for continuous dynamic intent decoding as claimed in any of claims 1 to 5.

8. A computer readable storage medium, characterized in that it stores a computer program which, when executed by a processor, implements the method for continuous dynamic intent decoding as claimed in any of claims 1 to 5.