CN115844402A - Attention prediction method, attention prediction device, electronic equipment and computer readable storage medium - Google Patents
Attention prediction method, attention prediction device, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN115844402A CN115844402A CN202211666836.6A CN202211666836A CN115844402A CN 115844402 A CN115844402 A CN 115844402A CN 202211666836 A CN202211666836 A CN 202211666836A CN 115844402 A CN115844402 A CN 115844402A
- Authority
- CN
- China
- Prior art keywords
- information
- state
- electroencephalogram
- attention
- target object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The embodiment of the application provides an attention prediction method, an attention prediction device, electronic equipment and a computer readable storage medium, and relates to the technical field of computers. The method comprises the following steps: the electroencephalogram information and the state information of the target object are subjected to prediction processing through a preset predictor by obtaining the electroencephalogram information of the target object when the target object operates the controlled device and the state information of the controlled device when the controlled device is operated by the target object, so that the attention prediction state of the target object is obtained. The electroencephalogram information can reflect the activity condition of the brain during the operation and control of the controlled equipment by the target object; the state information can represent the control state of the controlled equipment during the period that the target object controls the controlled equipment; therefore, the electroencephalogram information and the state information can reflect the attention state of the target object when the target object controls the controlled equipment to a certain extent. Therefore, the attention state of the target object is predicted based on the electroencephalogram information and the state information, and the prediction accuracy of the attention state can be improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to an attention prediction method, an attention prediction apparatus, an electronic device, and a computer-readable storage medium.
Background
In an equipment control scene, for example, in a scene of controlling an unmanned aerial vehicle, a processing device, a vehicle, and the like, the attention state (such as attention-focused state or attention-distracted state) of a controller generally affects the accident rate of the control process, so that the attention state of an operator is detected and identified to identify the attention state with attention-distracted state and remind the operator in time, thereby avoiding some accidents.
In the related art, the attention state of the operator is generally recognized by facial features and the like of the operator; the facial features are whether the operator yawns or not. However, the above facial features may be a reaction to an external stimulus, and the state of attention may not be accurately reflected, and thus, the state of attention is recognized by the above method, and the recognition accuracy is low.
Disclosure of Invention
The object of the present application is to solve at least one of the above-mentioned technical drawbacks, in particular the low accuracy of identifying the state of attention.
According to an aspect of the present application, there is provided an attention prediction method, including: responding to target trigger operation, and acquiring information to be detected; the information to be detected comprises electroencephalogram information of a target object and state information of the controlled device when the controlled device is controlled by the target object and acquired by electroencephalogram acquisition equipment;
predicting the information to be detected through a preset predictor to obtain the attention prediction state of the target object; and the preset predictor executes a preset prediction algorithm to perform prediction processing on the information to be tested to obtain the attention prediction state.
Optionally, after the information to be detected is obtained, the method further includes:
filtering and denoising a time domain electroencephalogram signal acquired through a preset frequency in the electroencephalogram information to obtain a denoised electroencephalogram signal;
and carrying out fast Fourier transform on the de-noised electroencephalogram signal to obtain a frequency domain electroencephalogram signal of a preset frequency band.
Optionally, the method further includes:
extracting the characteristics of the frequency domain electroencephalogram signals to obtain electroencephalogram characteristics;
the predicting the information to be detected through a preset predictor to obtain the attention predicting state of the target object comprises the following steps:
predicting the electroencephalogram characteristics and the state characteristics of the state information through a preset predictor to obtain the attention prediction state of the target object;
wherein the electroencephalogram characteristic and the state characteristic are visible time sequences; and predicting the time sequence through the preset predictor to obtain the invisible attention prediction state.
Optionally, the performing feature extraction on the frequency domain electroencephalogram signal to obtain electroencephalogram features includes:
vectorizing the frequency domain electroencephalogram signal to obtain an electroencephalogram vector;
and extracting the vector characteristics of the electroencephalogram vector through a characteristic extraction model, and obtaining the electroencephalogram characteristics according to the vector characteristics.
Optionally, the electroencephalogram vector includes electroencephalogram vector elements corresponding to at least two sampling channels respectively;
the extracting the vector characteristics of the electroencephalogram vector through the characteristic extraction model comprises the following steps:
determining an average vector of the brain electrical vector elements;
and extracting the vector characteristics of the average vector through the characteristic extraction model.
Optionally, the obtaining the electroencephalogram feature according to the vector feature includes:
and performing dimensionality reduction processing on the vector features to obtain the electroencephalogram features.
Optionally, before the information to be detected is subjected to prediction processing by using a preset predictor, the method further includes:
obtaining a training sample;
inputting the sample electroencephalogram information and the sample control state information of the training samples into an initial model to obtain a prediction result corresponding to each training sample; the prediction result comprises a sample attention prediction state;
determining a training loss value according to the sample attention prediction state and the attention reference state;
and repeatedly training the initial model based on the training loss value until the preset predictor meeting the training end condition is obtained.
Optionally, after obtaining the predicted attention state of the target object, the method further includes:
generating state prompt information under the condition that the attention prediction state does not meet a preset state condition; the state cue information indicates an attention state of the adjustment target object.
Optionally, the control status information at least includes at least one of the following:
position information of the controlled device;
speed information of the controlled device;
height information of the controlled device.
According to another aspect of the present application, there is provided an attention prediction apparatus including:
the information acquisition module is used for responding to target trigger operation and acquiring information to be detected; the information to be detected comprises electroencephalogram information of a target object and state information of the controlled device when the controlled device is controlled by the target object and acquired by electroencephalogram acquisition equipment;
the prediction module is used for carrying out prediction processing on the information to be detected through a preset predictor to obtain the attention prediction state of the target object; and the preset predictor executes a preset prediction algorithm to perform prediction processing on the information to be tested to obtain the attention prediction state.
According to another aspect of the present application, there is provided an electronic device including: memory, processor and computer program stored on the memory, characterized in that the processor executes the computer program to implement the steps of the attention prediction method of any of the first aspect of the present application.
For example, in a third aspect of the present application, there is provided a computing device comprising: the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the attention prediction method as shown in the first aspect of the application.
According to yet another aspect of the present application, a computer-readable storage medium is provided, which when executed by a processor, performs the steps of the attention prediction method of any one of the first aspect of the present application.
For example, in a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program, when executed by a processor, implements the attention prediction method shown in the first aspect of the present application.
According to an aspect of the application, a computer program product or computer program is provided, comprising computer instructions, the computer instructions being stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the method provided in the various alternative implementations of the first aspect described above.
The beneficial effect that technical scheme that this application provided brought is:
in the embodiment of the application, when a target object controls a controlled device, electroencephalogram information of the target object and state information of the controlled device when the controlled device is controlled by the target object are obtained, and prediction processing is performed on the electroencephalogram information and the state information through a preset predictor, so that the attention prediction state of the target object is obtained. The electroencephalogram information can reflect the activity condition of the brain when a target object controls the controlled device; the state information can represent the control state of the controlled equipment during the control of the controlled equipment by the target object; therefore, the electroencephalogram information and the state information can reflect the attention state of the target object when the target object controls the controlled device to a certain extent. In this way, the attention state of the target object is predicted based on the electroencephalogram information and the state information, and the prediction accuracy of the attention state can be improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments of the present application will be briefly described below.
Fig. 1 is a schematic system architecture diagram of an attention prediction method according to an embodiment of the present disclosure;
fig. 2 is a flowchart illustrating an attention prediction method according to an embodiment of the present disclosure;
fig. 3 is a schematic diagram illustrating an application of an attention prediction method according to an embodiment of the present application;
FIG. 4 is a second flowchart illustrating an attention prediction method according to an embodiment of the present disclosure;
fig. 5 is a third schematic flowchart of an attention prediction method according to an embodiment of the present application;
fig. 6 is a fourth schematic flowchart of an attention prediction method according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of an attention prediction apparatus according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device for attention prediction according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the embodiments set forth below in connection with the drawings are exemplary descriptions for explaining technical solutions of the embodiments of the present application, and do not limit the technical solutions of the embodiments of the present application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that the terms "comprises" and/or "comprising," when used in this specification in connection with embodiments of the present application, specify the presence of stated features, information, data, steps, operations, elements, and/or components, but do not preclude the presence or addition of other features, information, data, steps, operations, elements, components, and/or groups thereof, as embodied in the art. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein indicates at least one of the items defined by the term, e.g., "a and/or B" may be implemented as "a", or as "B", or as "a and B".
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
At least part of the content of the attention prediction method provided by the embodiment of the application relates to the fields of machine learning and the like in the field of artificial intelligence, and also relates to various fields of Cloud technology, such as Cloud computing in Cloud technology, cloud service and related data computing processing in the field of big data.
Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the implementation method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.
The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.
Machine Learning (ML) is a multi-domain cross subject, and relates to multiple subjects such as probability theory, statistics, approximation theory, convex analysis and algorithm complexity theory. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and is applied to all fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and formal education learning.
To further illustrate the technical solutions provided by the embodiments of the present application, the following detailed description is made with reference to the accompanying drawings and the detailed description. Although the embodiments of the present application provide method steps as shown in the following embodiments or figures, more or fewer steps may be included in the method based on conventional or non-inventive efforts. In steps where no necessary causal relationship exists logically, the order of execution of these steps is not limited to the order of execution provided by the embodiments of the present application.
Reference is first made to fig. 1, which is a system architecture diagram of an attention prediction method according to an embodiment of the present application. The system may include a server 101 and a cluster of terminals, wherein the server 101 may be considered as a background server for the attention prediction process.
The terminal cluster may include: the system comprises a terminal 102, a terminal 103, a terminal 104, and a terminal … …, wherein the terminal can be installed with a client supporting attention prediction processing. There may be a communication connection between the terminals, for example, a communication connection between terminal 102 and terminal 103, and a communication connection between terminal 103 and terminal 104.
Meanwhile, the server 101 may provide a service for the terminal cluster through a communication connection function, and any terminal in the terminal cluster may have a communication connection with the server 101, for example, a communication connection exists between the terminal 102 and the server 101, and a communication connection exists between the terminal 103 and the server 101, where the communication connection is not limited to a connection manner, and may be directly or indirectly connected through a wired communication manner, may also be directly or indirectly connected through a wireless communication manner, and may also be through other manners.
The communicatively coupled network may be a wide area network or a local area network, or a combination thereof. The application is not limited thereto.
In the embodiment of the application, when a target object controls a controlled device, electroencephalogram information of the target object and state information of the controlled device when the controlled device is controlled by the target object are obtained, and prediction processing is performed on the electroencephalogram information and the state information through a preset predictor, so that the attention prediction state of the target object is obtained. The electroencephalogram information can reflect the activity condition of the brain during the operation and control of the controlled equipment by the target object; the state information can represent the control state of the controlled equipment during the control of the controlled equipment by the target object; therefore, the electroencephalogram information and the state information can reflect the attention state of the target object when the target object controls the controlled device to a certain extent. In this way, the attention state of the target object is predicted based on the electroencephalogram information and the state information, and the prediction accuracy of the attention state can be improved.
The method provided by the embodiment of the present application can be executed by a computer device, which includes but is not limited to a terminal (also including the user terminal described above) or a server (also including the server 101 described above). The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network service, cloud communication, middleware service, a domain name service, a security service, a CDN, a big data and artificial intelligence platform, and the like. The terminal may be, but is not limited to, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and the like. The terminal and the server may be directly or indirectly connected through wired or wireless communication, and the application is not limited herein.
The embodiments of the present application are not intended to be limiting. The functions that can be implemented by each device in the application scenario shown in fig. 1 will be described in the following method embodiments, and will not be described in detail herein.
The embodiment of the present application provides a possible implementation manner, and the scheme may be executed by any electronic device, and optionally, any electronic device may be a server device with attention prediction capability, or may also be a device or a chip integrated on these devices. As shown in fig. 2, which is a schematic flow chart of an attention prediction method provided in an embodiment of the present application, the method includes the following steps:
step S201: and responding to the target trigger operation to acquire the information to be detected. The information to be detected comprises electroencephalogram information of a target object and state information of the controlled device when the controlled device is controlled by the target object and acquired by electroencephalogram acquisition equipment.
Optionally, the attention prediction method in the embodiment of the present application may be applied to predicting information to be detected, so as to identify an application scenario of the attention state of the target object.
The information to be detected comprises electroencephalogram information when a target object controls a controlled device and state information when the controlled device is controlled by the target object.
In particular, the target object may be any object that can manipulate the controlled device, for example, in an actual implementation scenario, the target object may be a driver of a vehicle, such as a car driver, an airplane driver, a ship driver, and the like; correspondingly, the controlled equipment can be vehicles such as automobiles, airplanes, ships and the like. The target object may also be an equipment operator, such as an unmanned aerial vehicle operator, a machining equipment operator, a detection equipment operator, a laboratory equipment operator, and the like; correspondingly, the controlled equipment can be unmanned aerial vehicles, processing equipment, detection equipment, experimental equipment and the like. The embodiments of the present application are not intended to be exhaustive or to limit the present application.
That is to say, in the foregoing scenario, the electroencephalogram information of the controlled device when being controlled by the target object and the state information of the controlled device when being controlled by the target object may be acquired, so as to predict the attention state of the target object based on the electroencephalogram information and the state information.
The electroencephalogram information can include electroencephalogram signals collected through electroencephalogram collecting equipment at a preset sampling frequency.
Brain wave (EEG) signals record brain activity in terms of electrophysiological indices; the brain wave signal is formed by summing up postsynaptic potentials generated by a large number of neurons synchronously when the brain is active; it records the electrical wave changes during brain activity, which is a general reflection of the electrophysiological activity of brain neurons on the surface of the cerebral cortex or scalp. That is, the brain wave signals may reflect the activity of the brain. For convenience of description, in the embodiments of the present application, the brain wave signal may be referred to as an electroencephalogram signal.
In an actual implementation scene, electrodes can be placed according to an international 10-20 system electrode placement method, and electroencephalogram signals are acquired through 14 sampling channels at a sampling frequency of 6 Hz.
The state information is the state information of the controlled equipment when the controlled equipment is controlled by the target object; the status information may include control parameters, operating parameters, experimental parameters, and the like of the controlled device. For example, the state information may include control parameters such as a position parameter, a speed parameter, an altitude parameter (altitude parameter of an airplane), and the like of an automobile, an airplane, a ship, and the like; the state information may further include operating parameters such as current parameters, voltage parameters, and rotational speed parameters of the processing device and the detection device, and experimental data parameters of the experimental device. The state information in the embodiment of the present application is used to characterize the control state of the controlled device, and based on the difference of the controlled device, the state information may include corresponding device parameters, which are merely examples, and are not listed one by one and not limited herein.
In the embodiment of the application, the information to be tested can be acquired in response to the target triggering operation, so that the attention state of the target object can be predicted based on the information to be tested. For example, in an actual scene, a target trigger operation of operating the controlled device may be started after a target object is received, that is, information to be measured may be acquired, and real-time prediction may be performed. And in a preset time period during the target object controls the controlled equipment, for example, 10 to 12 am every day, the target trigger operation is received, so that the information to be detected can be acquired, and real-time prediction is performed. And after the target object finishes controlling the controlled equipment, the target object receives target trigger operation, and can acquire the information to be tested acquired during the control period to predict.
Step S202: predicting the information to be detected through a preset predictor to obtain the attention prediction state of the target object; and the preset predictor executes a preset prediction algorithm to perform prediction processing on the information to be tested to obtain the attention prediction state.
In particular, since the electroencephalogram information may reflect an activity condition of a brain of a target object, and the state information may represent a control state of a controlled device during operation by the target object, embodiments of the present application may predict an attention state of the target object based on the electroencephalogram information and the state information.
In specific implementation, the electroencephalogram information and the state information can be subjected to prediction processing through a preset predictor, so that the attention prediction state of the target object is obtained.
Wherein the attention prediction state is a predicted attention state of the target object, such as attention focusing or attention non-focusing; furthermore, in some embodiments, the attention may also be divided into a plurality of levels, for example, the state of attention may be divided into an attention 1 level, an attention 2 level, an attention 3 level, and the like in terms of the degree of attention, with a higher level indicating a higher degree of attention.
In an actual implementation scenario, the attention prediction state obtained by the preset predictor may be characterized by an attention score, for example, the preset predictor may output the attention score, and determine whether the attention prediction state is attention-focused or attention-inattentive by comparing the attention score with a preset score threshold. As an example, for example, the preset score threshold is 80 points, and when the attention score is greater than 80 points, the attention state may be determined to be attention-focused; conversely, when the attention score is less than or equal to 80 minutes, the state of attention may be determined to be inattentive.
The prediction algorithm executed by the preset predictor can be obtained by training sample electroencephalogram information based on the sample object and sample control state information of the sample controlled equipment. For example, in an actual implementation scenario, the preset predictor may be a trained attention prediction model; optionally, in this embodiment of the application, the attention prediction model may be obtained based on a Hidden Markov Model (HMM). The HMM is a dynamic bayesian network generation model with a simple structure, and can effectively detect non-stationary changes in the electroencephalogram signal.
The attention prediction model of the embodiment of the application may be composed of two HMMs, an attention-focused HMM and an attention-inattentive HMM; each HMM is trained by a corresponding electroencephalogram signal sequence. For example, an attention-focused HMM is trained on an EEG signal sequence with attention in a focused state.
In the embodiment of the application, when a target object controls a controlled device, electroencephalogram information of the target object and state information of the controlled device when the controlled device is controlled by the target object are obtained, and prediction processing is performed on the electroencephalogram information and the state information through a preset predictor, so that the attention prediction state of the target object is obtained. The electroencephalogram information can reflect the activity condition of the brain when a target object controls the controlled device; the state information can represent the control state of the controlled equipment during the control of the controlled equipment by the target object; therefore, the electroencephalogram information and the state information can reflect the attention state of the target object when the target object controls the controlled device to a certain extent. In this way, the attention state of the target object is predicted based on the electroencephalogram information and the state information, and the prediction accuracy of the attention state can be improved.
In an embodiment of the application, after the information to be measured is obtained, the method further includes:
filtering and denoising a time domain electroencephalogram signal acquired through a preset frequency in the electroencephalogram information to obtain a denoised electroencephalogram signal;
and carrying out fast Fourier transform on the de-noised electroencephalogram signal to obtain a frequency domain electroencephalogram signal of a preset frequency band.
In the embodiment of the application, the electroencephalogram information can include a time domain electroencephalogram signal acquired through a preset frequency. In the time-domain electroencephalogram signal, a certain noise signal often exists due to the acquired equipment factor or the acquired time factor. For example, the noise signal may be an electromagnetic interference signal introduced by the acquisition device itself; by way of example, the embodiments of the present application may filter the time domain brain electrical signal through a band pass filter with a bandwidth from 2Hz to 42Hz to remove noise. In addition, in the initial control stage of the target object, the controlled device is unstable, so that errors exist in state information and a prediction attention state of the time domain electroencephalogram signal, and therefore the time domain electroencephalogram signal and the state information in the initial control stage can be regarded as noise signals, and therefore the time domain electroencephalogram signal and the state information in the initial control stage can be deleted.
And carrying out the filtering and denoising treatment on the time domain electroencephalogram signal to obtain a denoised electroencephalogram signal. Further, fast Fourier Transform (FFT) may be performed on the denoised electroencephalogram signal to obtain a frequency domain electroencephalogram signal of a preset frequency band; the frequency domain EEG signals of the preset frequency band comprise delta signals, theta signals, alpha signals and beta signals.
In one embodiment of the present application, the method further comprises:
extracting the characteristics of the frequency domain electroencephalogram signals to obtain electroencephalogram characteristics;
the predicting the information to be detected through a preset predictor to obtain the attention predicting state of the target object comprises the following steps:
predicting the electroencephalogram characteristics and the state characteristics of the state information through a preset predictor to obtain the attention prediction state of the target object;
wherein the electroencephalogram characteristic and the state characteristic are visible time sequences; and predicting the time sequence through the preset predictor to obtain the invisible attention prediction state.
After the frequency domain electroencephalogram signal is obtained, the embodiment of the application can perform feature extraction on the frequency domain electroencephalogram signal to obtain electroencephalogram features, and then the information to be detected is subjected to prediction processing through a preset predictor to obtain the attention prediction state of the target object.
In an embodiment of the application, the frequency domain electroencephalogram signal can be subjected to feature extraction through the following steps to obtain electroencephalogram features:
vectorizing the frequency domain electroencephalogram signal to obtain an electroencephalogram vector;
and extracting the vector characteristics of the electroencephalogram vector through a characteristic extraction model, and obtaining the electroencephalogram characteristics according to the vector characteristics.
Specifically, the frequency domain electroencephalogram signal is vectorized and expressed as a three-dimensional vector et, i.e. an electroencephalogram vector in the embodiment of the present application. Wherein et is a matrix sequence of time t; t represents the acquisition time of the information to be measured (optionally, T =1,2 … … T). The sequence elements of the matrix sequence et can be represented as
Wherein i represents a frequency band, the frequency band respectively corresponds to the frequency bands of a delta signal, a theta signal, an alpha signal and a beta signal, and correspondingly, i can take the values of 1,2, 3 and 4; for example, if i is 1, the frequency band corresponding to the delta signal is indicated, if i is 2, the frequency band corresponding to the theta signal is indicated, if i is 3, the frequency band corresponding to the alpha signal is indicated, and if i is 4, the frequency band corresponding to the beta signal is indicated. j denotes a sampling channel of the digital video signal,in this embodiment, the sampling channels may include 14 channels, and therefore j may take the value of 1,2, 3 … …. It is understood that the EEG vector et of the frequency domain EEG signal may include 56 sequence elements->
Obtaining the brain electrical vector
And then, extracting the vector characteristics of the electroencephalogram vector through a characteristic extraction model, and obtaining the electroencephalogram characteristics according to the vector characteristics.
Optionally, in an embodiment of the present application, the electroencephalogram vector includes electroencephalogram vector elements corresponding to at least two sampling channels, respectively;
the extracting the vector characteristics of the electroencephalogram vector through the characteristic extraction model comprises the following steps:
determining an average vector of the brain electrical vector elements;
and extracting the vector characteristics of the average vector through the characteristic extraction model.
The obtaining of the electroencephalogram feature according to the vector feature comprises:
and performing dimensionality reduction processing on the vector features to obtain the electroencephalogram features.
With reference to fig. 3, in the embodiment of the present application, electroencephalogram signals can be collected through 14 sampling channels, that is, electroencephalogram vectors include electroencephalogram vector elements corresponding to the 14 sampling channels, respectively
Wherein j represents a sampling channel, that is, j can take the values of 1,2, 3 … …. i represents frequency bands corresponding to the frequency bands of the delta signal, the theta signal, the alpha signal and the beta signal, and can take values of 1,2, 3 and 4.
For each frequency band, the average of the electroencephalogram vector elements can be determined respectivelyMean vector
I.e. is>
Then, the vector feature of the average vector can be extracted by the feature extraction model. Optionally, the feature extraction model may be an Adaptive Auto Regression (AAR) model, where the AAR model estimates a model coefficient by a least square method. In the embodiment of the present application, when the vector features are extracted through the AAR model, for each average vector, 10 feature elements may be extracted, for example, for £ or £ r>
The characteristic element is->
It will be appreciated that a pair->
By performing feature extraction, a feature matrix of 4 × 10, that is, the vector features of the embodiment of the present application can be obtained.
Since the high-dimensional features easily increase the complexity of classification and reduce the calculation speed, in the embodiment of the present application, after the vector features are obtained, the vector features may be subjected to dimensionality reduction, and specifically, linear and uncorrelated Principal components may be extracted from the vector features by a Principal Component Analysis (PCA), so as to obtain an electroencephalogram feature f with lower dimensionality and capable of reflecting subtle changes of electroencephalogram waves t . Optionally, in this embodiment of the application, the electroencephalogram feature f t Can comprise 10 characteristic elements, namely an electroencephalogram characteristic tableIs shown as f 1 t 、f 2 t 、f 3 t ……f 10 t The electroencephalogram features are time series.
In an embodiment of the present application, before the predicting processing is performed on the information to be measured by using a preset predictor, the method further includes a step of training an initial model to obtain the preset predictor, which specifically includes:
obtaining a training sample;
inputting the sample electroencephalogram information and the sample control state information of the training samples into an initial model to obtain a prediction result corresponding to each training sample; the prediction result comprises a sample attention prediction state;
determining a training loss value according to the sample attention prediction state and the attention reference state;
and repeatedly training the initial model based on the training loss value until the preset predictor meeting the training end condition is obtained.
Specifically, the training sample comprises sample data, and the sample data comprises sample electroencephalogram information of the sample object and sample control state information of the sample controlled equipment.
The attention reference state may be understood as the attention state of the pre-labeled sample object. For example, in a scenario where an operator is manipulating an aircraft, during an unstable flight of the aircraft, there is a large fluctuation in the state information of the aircraft, and the attentional state may be marked as inattentive; during a stable flight, the attentional state may be marked as attentional focus, and so on. As another example, attention status may also be marked based on status information; optionally, in the above scenario where the operator manipulates the aircraft, the flying height a within a preset time period may be calculated t And the flying speed v t The attention state may be marked by comparing the variance with a preset threshold, e.g. when the variance is greater than the preset threshold, the attention state may be marked as attentive. In another example, the brain wave signal can be marked by the fluctuation of the brain wave signal, and when the brain wave signal has larger wavesOn the move, the attentiveness state may be marked as inattentive.
With reference to fig. 4, during training, inputting the sample electroencephalogram information and the sample control state information of the training samples into an initial model to obtain a prediction result corresponding to each training sample; the prediction result comprises a sample attention prediction state;
determining a training loss value according to the sample attention prediction state and the attention reference state;
and repeatedly training the initial model based on the training loss value until the preset predictor meeting the training end condition is obtained.
After the preset predictor is obtained, the attention state of the target object can be predicted in a subsequent actual prediction scene based on the prediction information.
In an embodiment of the application, after obtaining the predicted attentiveness state of the target object, the method further includes:
generating state prompt information under the condition that the attention prediction state does not meet a preset state condition; the state cue information indicates an attention state of the adjustment target object.
In an actual scenario, when the attention prediction state is inattentive or the attention score is lower than a preset score threshold, state prompt information may be generated. As an example, as shown in fig. 5, when the attention prediction state does not satisfy the preset state condition, a prompt command signal, such as a visual signal or an audible signal, may be generated, so that the target object may be prompted visually and audibly to adjust the attention state of the target object.
In some optional embodiments, the status prompt message may be generated by an alarm module; the alarm module comprises a command converter and a stimulus generator, wherein the command converter generates state prompt information under the condition that the attention prediction state does not meet the preset state condition, and the stimulus generator provides visual signals or auditory signals for the target object according to the state prompt information; visual signals, for example LED light signals with flashing lights, and audible signals, for example beeps generated by a loudspeaker.
In the following, with reference to fig. 6, a practical application scenario of the present application is described by taking an operation of an aircraft by an operator as an example:
firstly, collecting data to be detected, namely electroencephalogram information of an operator, and state information of an aircraft, such as flight position, flight height, flight speed and the like; carrying out preprocessing such as filtering denoising and fast Fourier transform on data to be detected, and then carrying out attention prediction on the data to be detected; prompting the operator to pay attention to the flight state under the condition of predicting that the attention of the operator is not focused, for example, prompting the operator to perform flight route detection, flight altitude detection, flight speed detection and the like so as to determine the current position, the current flight altitude, the current flight speed and the like; and adjusts the flying position, flying height, flying speed, etc. in real time. In addition, in the case where it is predicted that the attention of the operator is focused, the operator may be prompted to check the connection state of the aircraft or the like to confirm that the connection state of the aircraft is good.
In the embodiment of the application, when a target object controls a controlled device, electroencephalogram information of the target object and state information of the controlled device when the controlled device is controlled by the target object are obtained, and prediction processing is performed on the electroencephalogram information and the state information through a preset predictor, so that the attention prediction state of the target object is obtained. The electroencephalogram information can reflect the activity condition of the brain during the operation and control of the controlled equipment by the target object; the state information can represent the control state of the controlled equipment during the control of the controlled equipment by the target object; therefore, the electroencephalogram information and the state information can reflect the attention state of the target object when the target object controls the controlled device to a certain extent. In this way, the attention state of the target object is predicted based on the electroencephalogram information and the state information, and the prediction accuracy of the attention state can be improved.
An embodiment of the present application provides an attention prediction apparatus, and as shown in fig. 7, the attention prediction apparatus 70 may include: an information acquisition module 701, a prediction module 702, wherein,
an information obtaining module 701, configured to obtain information to be detected in response to a target trigger operation; the information to be detected comprises electroencephalogram information of a target object and state information of the controlled device when the controlled device is controlled by the target object acquired by electroencephalogram acquisition equipment;
the prediction module 702 is configured to perform prediction processing on the information to be detected through a preset predictor to obtain an attention prediction state of the target object; and the preset predictor executes a preset prediction algorithm to perform prediction processing on the information to be tested to obtain the attention prediction state.
In one embodiment of the present application, the apparatus further includes a denoising module, configured to, after the acquiring of the information to be measured,
filtering and denoising a time domain electroencephalogram signal acquired through a preset frequency in the electroencephalogram information to obtain a denoised electroencephalogram signal;
and carrying out fast Fourier transform on the de-noised electroencephalogram signal to obtain a frequency domain electroencephalogram signal of a preset frequency band.
In an embodiment of the application, the device further includes a feature extraction module, configured to perform feature extraction on the frequency domain electroencephalogram signal to obtain electroencephalogram features;
the prediction module is used for performing prediction processing on the electroencephalogram characteristics and the state characteristics of the state information through a preset predictor to obtain the attention prediction state of the target object;
wherein the electroencephalogram characteristic and the state characteristic are visible time sequences; and predicting the time sequence through the preset predictor to obtain the invisible attention prediction state.
In an embodiment of the application, the feature extraction module is configured to perform vectorization processing on the frequency-domain electroencephalogram signal to obtain an electroencephalogram vector;
and extracting the vector characteristics of the electroencephalogram vector through a characteristic extraction model, and obtaining the electroencephalogram characteristics according to the vector characteristics.
In one embodiment of the present application, the electroencephalogram vector includes electroencephalogram vector elements corresponding to at least two sampling channels, respectively;
the feature extraction module is used for determining an average vector of the electroencephalogram vector elements;
and extracting the vector characteristics of the average vector through the characteristic extraction model.
In an embodiment of the present application, the feature extraction module is configured to perform dimension reduction processing on the vector features to obtain the electroencephalogram features.
In one embodiment of the present application, the apparatus further includes a training module, configured to, before the information to be tested is subjected to the prediction processing by the preset predictor,
obtaining a training sample;
inputting the sample electroencephalogram information and the sample control state information of the training samples into an initial model to obtain a prediction result corresponding to each training sample; the prediction result comprises a sample attention prediction state;
determining a training loss value according to the sample attention prediction state and the attention reference state;
and repeatedly training the initial model based on the training loss value until the preset predictor meeting the training end condition is obtained.
In one embodiment of the present application, the apparatus further comprises a prompt module for, after obtaining the predicted attention state of the target subject,
generating state prompt information under the condition that the attention prediction state does not meet a preset state condition; the state cue information indicates an attention state of the adjustment target object.
In one embodiment of the present application, the control state information includes at least one of:
position information of the controlled device;
speed information of the controlled device;
height information of the controlled device.
The apparatus of the embodiment of the present application may execute the method provided by the embodiment of the present application, and the implementation principle is similar, the actions executed by the modules in the apparatus of the embodiments of the present application correspond to the steps in the method of the embodiments of the present application, and for the detailed functional description of the modules of the apparatus, reference may be specifically made to the description in the corresponding method shown in the foregoing, and details are not repeated here.
In the embodiment of the application, when a target object controls a controlled device, electroencephalogram information of the target object and state information of the controlled device when the controlled device is controlled by the target object are obtained, and prediction processing is performed on the electroencephalogram information and the state information through a preset predictor, so that the attention prediction state of the target object is obtained. The electroencephalogram information can reflect the activity condition of the brain during the operation and control of the controlled equipment by the target object; the state information can represent the control state of the controlled equipment during the control of the controlled equipment by the target object; therefore, the electroencephalogram information and the state information can reflect the attention state of the target object when the target object controls the controlled device to a certain extent. In this way, the attention state of the target object is predicted based on the electroencephalogram information and the state information, and the prediction accuracy of the attention state can be improved.
An embodiment of the present application provides an electronic device, which includes: a memory and a processor; at least one program stored in the memory for execution by the processor, which when executed by the processor, implements: in the embodiment of the application, when a target object controls a controlled device, electroencephalogram information of the target object and state information of the controlled device when the controlled device is controlled by the target object are obtained, and prediction processing is performed on the electroencephalogram information and the state information through a preset predictor, so that the attention prediction state of the target object is obtained. The electroencephalogram information can reflect the activity condition of the brain during the operation and control of the controlled equipment by the target object; the state information can represent the control state of the controlled equipment during the control of the controlled equipment by the target object; therefore, the electroencephalogram information and the state information can reflect the attention state of the target object when the target object controls the controlled device to a certain extent. In this way, the attention state of the target object is predicted based on the electroencephalogram information and the state information, and the prediction accuracy of the attention state can be improved.
In an alternative embodiment, an electronic device is provided, as shown in FIG. 8, the electronic device 8000 shown in FIG. 8 including: a processor 8001 and memory 8003. Processor 8001 is coupled to memory 8003, such as via bus 8002. Optionally, the electronic device 8000 may further include a transceiver 8004, and the transceiver 8004 may be used for data interaction between the electronic device and other electronic devices, such as transmission of data and/or reception of data. In addition, the transceiver 8004 is not limited to one in practical applications, and the structure of the electronic device 8000 does not limit the embodiment of the present application.
The Memory 8003 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical Disc storage, optical Disc storage (including Compact Disc, laser Disc, optical Disc, digital versatile Disc, blu-ray Disc, etc.), a magnetic Disc storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to these.
The memory 8003 is used for storing application program codes (computer programs) for executing the scheme of the present application, and is controlled by the processor 8001 to execute. Processor 8001 is used to execute application program code stored in memory 8003 to implement what is shown in the foregoing method embodiments.
Among them, electronic devices include but are not limited to: mobile phones, notebook computers, multimedia players, desktop computers, and the like.
The present application provides a computer-readable storage medium, on which a computer program is stored, which, when running on a computer, enables the computer to execute the corresponding content in the foregoing method embodiments.
In the embodiment of the application, when a target object controls a controlled device, electroencephalogram information of the target object and state information of the controlled device when the controlled device is controlled by the target object are obtained, and prediction processing is performed on the electroencephalogram information and the state information through a preset predictor, so that the attention prediction state of the target object is obtained. The electroencephalogram information can reflect the activity condition of the brain during the operation and control of the controlled equipment by the target object; the state information can represent the control state of the controlled equipment during the control of the controlled equipment by the target object; therefore, the electroencephalogram information and the state information can reflect the attention state of the target object when the target object controls the controlled device to a certain extent. In this way, the attention state of the target object is predicted based on the electroencephalogram information and the state information, and the prediction accuracy of the attention state can be improved.
The terms "first," "second," "third," "fourth," "1," "2," and the like in the description and in the claims of the present application and in the above-described drawings (if any) are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in other sequences than described or illustrated herein.
It should be understood that, although each operation step is indicated by an arrow in the flowchart of the embodiment of the present application, the implementation order of the steps is not limited to the order indicated by the arrow. In some implementation scenarios of the embodiments of the present application, the implementation steps in the flowcharts may be performed in other sequences as needed, unless explicitly stated otherwise herein. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on an actual implementation scenario. Some or all of these sub-steps or stages may be performed at the same time, or each of these sub-steps or stages may be performed at different times, respectively. In a scenario where execution times are different, an execution sequence of the sub-steps or the phases may be flexibly configured according to requirements, which is not limited in the embodiment of the present application.
The foregoing is only an optional implementation manner of a part of implementation scenarios in this application, and it should be noted that, for those skilled in the art, other similar implementation means based on the technical idea of this application are also within the protection scope of the embodiments of this application without departing from the technical idea of this application.
Claims (12)
1. An attention prediction method, comprising:
responding to target trigger operation, and acquiring information to be detected; the information to be detected comprises electroencephalogram information of a target object and state information of the controlled device when the controlled device is controlled by the target object acquired by electroencephalogram acquisition equipment;
predicting the information to be detected through a preset predictor to obtain the attention prediction state of the target object; and the preset predictor executes a preset prediction algorithm to predict the information to be detected so as to obtain the attention prediction state.
2. The attention prediction method of claim 1, wherein after obtaining the information to be measured, the method further comprises:
filtering and denoising a time domain electroencephalogram signal acquired through a preset frequency in the electroencephalogram information to obtain a denoised electroencephalogram signal;
and carrying out fast Fourier transform on the de-noised electroencephalogram signal to obtain a frequency domain electroencephalogram signal of a preset frequency band.
3. The attention prediction method of claim 2, further comprising:
extracting the characteristics of the frequency domain electroencephalogram signals to obtain electroencephalogram characteristics;
the predicting the information to be detected through a preset predictor to obtain the attention predicting state of the target object comprises the following steps:
predicting the electroencephalogram characteristics and the state characteristics of the state information through a preset predictor to obtain the attention prediction state of the target object;
wherein the electroencephalogram characteristic and the state characteristic are visible time sequences; and predicting the time sequence through the preset predictor to obtain the invisible attention prediction state.
4. The attention prediction method of claim 3, wherein the extracting the features of the frequency domain electroencephalogram signal to obtain the electroencephalogram features comprises:
vectorizing the frequency domain electroencephalogram signal to obtain an electroencephalogram vector;
and extracting the vector characteristics of the electroencephalogram vector through a characteristic extraction model, and obtaining the electroencephalogram characteristics according to the vector characteristics.
5. The attention prediction method of claim 4, wherein the electroencephalogram vector includes electroencephalogram vector elements corresponding to at least two sampling channels, respectively;
the extracting the vector characteristics of the electroencephalogram vector through the characteristic extraction model comprises the following steps:
determining an average vector of the brain electrical vector elements;
and extracting the vector characteristics of the average vector through the characteristic extraction model.
6. The method of claim 4, wherein said deriving the brain electrical features from the vector features comprises:
and performing dimensionality reduction processing on the vector features to obtain the electroencephalogram features.
7. The attention prediction method according to claim 1, wherein before the prediction processing of the information to be measured by the preset predictor, the method further comprises:
obtaining a training sample;
inputting the sample electroencephalogram information and the sample control state information of the training samples into an initial model to obtain a prediction result corresponding to each training sample; the prediction result comprises a sample attention prediction state;
determining a training loss value according to the sample attention prediction state and the attention reference state;
and repeatedly training the initial model based on the training loss value until the preset predictor meeting the training end condition is obtained.
8. The attention prediction method of claim 1, wherein after obtaining the predicted attention state of the target subject, the method further comprises:
generating state prompt information under the condition that the attention prediction state does not meet a preset state condition; the state cue information indicates an attention state of the adjustment target object.
9. The attention prediction method of claim 1, wherein the control state information comprises at least one of:
position information of the controlled device;
speed information of the controlled device;
height information of the controlled device.
10. An attention prediction apparatus, comprising:
the information acquisition module is used for responding to target trigger operation and acquiring information to be detected; the information to be detected comprises electroencephalogram information of a target object and state information of the controlled device when the controlled device is controlled by the target object and acquired by electroencephalogram acquisition equipment;
the prediction module is used for carrying out prediction processing on the information to be detected through a preset predictor to obtain the attention prediction state of the target object; and the preset predictor executes a preset prediction algorithm to predict the information to be detected so as to obtain the attention prediction state.
11. An electronic device comprising a memory, a processor and a computer program stored on the memory, characterized in that the processor executes the computer program to implement the steps of the attention prediction method of any one of claims 1-9.
12. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the attention prediction method as claimed in any one of the claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211666836.6A CN115844402A (en) | 2022-12-23 | 2022-12-23 | Attention prediction method, attention prediction device, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211666836.6A CN115844402A (en) | 2022-12-23 | 2022-12-23 | Attention prediction method, attention prediction device, electronic equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115844402A true CN115844402A (en) | 2023-03-28 |
Family
ID=85654422
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211666836.6A Pending CN115844402A (en) | 2022-12-23 | 2022-12-23 | Attention prediction method, attention prediction device, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115844402A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116719418A (en) * | 2023-08-09 | 2023-09-08 | 湖南马栏山视频先进技术研究院有限公司 | Method and device for checking gaze point prediction model |
-
2022
- 2022-12-23 CN CN202211666836.6A patent/CN115844402A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116719418A (en) * | 2023-08-09 | 2023-09-08 | 湖南马栏山视频先进技术研究院有限公司 | Method and device for checking gaze point prediction model |
| CN116719418B (en) * | 2023-08-09 | 2023-10-27 | 湖南马栏山视频先进技术研究院有限公司 | Method and device for checking gaze point prediction model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112785016B (en) | New energy automobile maintenance and fault monitoring and diagnosis method based on machine learning | |
| Chai et al. | Driver fatigue classification with independent component by entropy rate bound minimization analysis in an EEG-based system | |
| US10931772B2 (en) | Method and apparatus for pushing information | |
| CN110706700B (en) | In-vehicle disturbance prevention alarm method and device, server and storage medium | |
| CN109243490A (en) | Driver's Emotion identification method and terminal device | |
| CN106108894A (en) | A kind of emotion electroencephalogramrecognition recognition method improving Emotion identification model time robustness | |
| Khare et al. | Classification of emotions from EEG signals using time‐order representation based on the S‐transform and convolutional neural network | |
| CN113693613A (en) | Electroencephalogram signal classification method and device, computer equipment and storage medium | |
| DE102019109148A1 (en) | WAKE-ON-VOICE KEY PHRASE SEGMENTATION | |
| CN115844402A (en) | Attention prediction method, attention prediction device, electronic equipment and computer readable storage medium | |
| CN114492513A (en) | Electroencephalogram emotion recognition method for adaptation to immunity domain based on attention mechanism in cross-user scene | |
| Yang et al. | EEG-based emotion recognition using temporal convolutional network | |
| CN116129510A (en) | Method and device for evaluating pilot perception of different coded information based on eye movement data | |
| CN114343640B (en) | Attention assessment method and electronic equipment | |
| CN107334481B (en) | Driving distraction detection method and system | |
| Dehzangi et al. | EEG based driver inattention identification via feature profiling and dimensionality reduction | |
| CN114129163A (en) | Electroencephalogram signal-based emotion analysis method and system for multi-view deep learning | |
| Xie et al. | An SVM parameter learning algorithm scalable on large data size for driver fatigue detection | |
| CN114693945A (en) | Vehicle engine abnormality detection method, vehicle engine abnormality detection device, electronic apparatus, and storage medium | |
| CN112735436A (en) | Voiceprint recognition method and voiceprint recognition system | |
| Suresh et al. | Analysis and Implementation of Deep Convolutional Neural Network Models for Intelligent Driver Drowsiness Detection System | |
| CN116705013B (en) | Voice wake-up word detection method and device, storage medium and electronic equipment | |
| CN117195082B (en) | Driving behavior prediction method and device | |
| CN116012776B (en) | Method and device for monitoring number of people, electronic equipment and readable storage medium | |
| CN117204856A (en) | Method and device for detecting emotion state of driver based on brain electrical data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |