CN111345784B - Wearable protective equipment based on individual intelligent bone conduction auditory feedback technology - Google Patents
Wearable protective equipment based on individual intelligent bone conduction auditory feedback technology Download PDFInfo
- Publication number
- CN111345784B CN111345784B CN202010384240.1A CN202010384240A CN111345784B CN 111345784 B CN111345784 B CN 111345784B CN 202010384240 A CN202010384240 A CN 202010384240A CN 111345784 B CN111345784 B CN 111345784B
- Authority
- CN
- China
- Prior art keywords
- electroencephalogram
- signals
- audio
- bone conduction
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
- A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
- A61B5/6802—Sensor mounted on worn items
- A61B5/6803—Head-worn items, e.g. helmets, masks, headphones or goggles
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/40—Detecting, measuring or recording for evaluating the nervous system
- A61B5/4076—Diagnosing or monitoring particular conditions of the nervous system
- A61B5/4088—Diagnosing of monitoring cognitive diseases, e.g. Alzheimer, prion diseases or dementia
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/48—Other medical applications
- A61B5/4806—Sleep evaluation
- A61B5/4815—Sleep quality
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
- A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
- A61B5/6843—Monitoring or controlling sensor contact pressure
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
- A61F11/00—Methods or devices for treatment of the ears or hearing sense; Non-electric hearing aids; Methods or devices for enabling ear patients to achieve auditory perception through physiological senses other than hearing sense; Protective devices for the ears, carried on the body or in the hand
- A61F11/04—Methods or devices for enabling ear patients to achieve auditory perception through physiological senses other than hearing sense, e.g. through the touch sense
- A61F11/045—Methods or devices for enabling ear patients to achieve auditory perception through physiological senses other than hearing sense, e.g. through the touch sense using mechanical stimulation of nerves
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
- A61M21/00—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
- A61M21/02—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis for inducing sleep or relaxation, e.g. by direct nerve stimulation, hypnosis, analgesia
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
- A61M21/00—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
- A61M2021/0005—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis by the use of a particular sense, or stimulus
- A61M2021/0027—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis by the use of a particular sense, or stimulus by the hearing sense
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Heart & Thoracic Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Medical Informatics (AREA)
- Pathology (AREA)
- Psychology (AREA)
- Neurology (AREA)
- Psychiatry (AREA)
- Hospice & Palliative Care (AREA)
- Acoustics & Sound (AREA)
- Child & Adolescent Psychology (AREA)
- Developmental Disabilities (AREA)
- Physiology (AREA)
- Anesthesiology (AREA)
- Pain & Pain Management (AREA)
- Neurosurgery (AREA)
- Hematology (AREA)
- Otolaryngology (AREA)
- Vascular Medicine (AREA)
- Educational Technology (AREA)
- Social Psychology (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
Wearable protective equipment based on individual soldier's intelligence bone conduction auditory feedback technique includes: the flexible shell is provided with a control module, a stereo bone conduction sound transmission vibrator and an electroencephalogram sensor; the control module controls the stereo bone conduction sound transmission vibrator to send an audio stimulation signal, and the electroencephalogram sensor acquires an electroencephalogram signal of a forehead area; the control module extracts cognitive competence characteristics and emotional state characteristics of an individual soldier based on an electroencephalogram signal acquired when a human brain is in a working state, matches an optimum audio stimulation signal corresponding to the cognitive competence characteristics and emotional state characteristics from an audio database based on the corresponding relation between the cognitive competence characteristics and the emotional state characteristics in the electroencephalogram database and the audio signals in the audio database, and plays the optimum audio stimulation signal to the individual soldier; the sleep quality characteristics of the individual soldiers are extracted based on the electroencephalogram signals collected when the brains of the individuals are in the sleep state, the most suitable audio stimulation signals corresponding to the sleep quality characteristics in the electroencephalogram database are matched from the audio database based on the corresponding relation between the sleep quality characteristics in the electroencephalogram database and the audio signals in the audio database, and the most suitable audio stimulation signals are played to the individual soldiers.
Description
Technical Field
The invention relates to the technical field of wearable equipment, in particular to wearable protective equipment capable of improving cognitive ability and emotional state of deep sea closed space operators based on an individual intelligent bone conduction auditory feedback technology.
Background
The deep sea closed space cabin has very bad environmental conditions, and the environmental factors influencing the survival ability of operators not only comprise factors such as illumination, color, temperature and humidity, noise, vibration, harmful gas, microorganisms, nuclear radiation and the like, but also comprise a plurality of aspects such as space layout on a boat, sanitary facilities, dietary nutrition, work and rest on duty, physical exercise and the like. Through investigation and research, the environmental factors influencing the operation ability of the operators are complex and various, the composite hazard effect of the environmental factors is more complex, and the physical and mental health, the operation ability and the emotional state of the operators are seriously influenced.
How to maintain, enhance and guarantee the cognitive ability of the operating personnel in the long-distance navigation process is the key for guaranteeing that the operating task is effectively completed. Therefore, researching maintenance of cognitive ability of operators, objective evaluation of cognitive impairment and non-drug intervention methods, establishing an enhancement/protection non-drug intervention platform for the operational ability of the operators in the deep-sea closed space environment and researching and developing novel wearable intelligent protection and war force enhancement equipment become important problems to be solved urgently in deep and far-sea work of our army.
Aiming at the problems of cognitive decline and low emotion induced by deep sea closed space environment and high stress state, no special and efficient treatment means for non-drug cognitive intervention exists at present.
(1) The cabin in the deep sea closed space has complex and various environmental factors, obvious superposition effect and difficult operation task, and is easy to cause high stress state of operators.
In the cabin environment of the deep sea closed space, a large number of stressors exist, including nucleation biological factors, physiological factors, psychology and social factors and the like, which all have important influence on the physiology and psychology of operators, so that officers and soldiers are in a high stress state for a long time, run at full load or even overload to cause physical and psychological stress, which is manifested by fatigue, immunity decline, sleep-wake rhythm disorder, cognitive ability decline, anxiety and negative emotions, and even cause decision making and operation errors to cause serious consequences. Therefore, the mechanism of the influence of the deep-sea enclosed space on the cognitive ability of the operators must be deeply researched, measures for maintaining and enhancing the cognitive ability and relieving the mental stress are made in a targeted manner, and the operation ability of the operators in the deep-sea enclosed space is improved.
(2) The cognitive ability of the operating personnel is reduced due to the deep-sea closed space environment, and the operating efficiency of the operating personnel is seriously influenced.
The deep sea closed space has high requirements on the aspects of accurate operation, quick response and the like of operators, and long-time and high-intensity mental work during the execution of special tasks makes the importance of cognitive functions in the working capacity prominent. In the deep sea enclosed space "man-machine-environment" unit, man is the weakest link, while in the physical and mental (cognitive) fatigue of the working personnel, cognitive ability is the weakest link. The physiological and psychological influences of various adverse factors in the deep-sea closed space environment are necessarily reflected on the performance of cognitive operation, and the decline of cognitive ability can cause serious consequences. Foreign research finds that the overall performance of the cognitive work in 24 hours due to lack of sleep in continuous operation is reduced by 30 percent, the reduction amplitude in 42 hours is as high as 60 percent, more than 70 percent of serious flight accidents in flight tasks are caused by human factors, and 80 percent of large equipment accidents in various countries are related to the reduction of the cognitive ability and the emotional deterioration of people.
(3) The cognitive enhancement in the deep sea closed space operation is still lack of scientific methods, platforms and enhancement technologies, and a cognitive enhancement method system is urgently built.
In view of the narrow and closed environment of the deep sea closed space, the high concealment and the high cruising ability, the lonely marching can be carried out or the long-time latency is realized, and higher requirements are provided for the early warning and the protection of the cognitive ability damage of the operating personnel. At present, China still lacks deep research on evaluation indexes, diagnosis standards and protective measures of cognitive operation capacity, and a complete prevention, diagnosis and treatment system and a reliable cognitive enhancement technology are not established. In order to improve the operation performance and the operation capability of deep sea closed space operators, a multi-level cognitive enhancement method system is urgently established. At present, on the basis of clarifying a neural mechanism of operation capacity damage of a deep sea airtight space, real-time and personalized monitoring of cognitive capacity and emotional state is urgently needed to be completed based on brain science and brain-computer interface technology, and a personalized and precise intelligent auditory feedback intervention method and a training paradigm are provided based on noninvasive sound intervention technology without side effects and electroencephalogram and sound signal-based intelligent matching technology, so that the prevention and treatment of the operation cognitive capacity damage of the deep sea airtight space and the enhancement of cognitive efficiency are positively promoted.
Disclosure of Invention
Aiming at the problems and the defects in the prior art, the invention provides novel wearable protective equipment based on an individual intelligent bone conduction auditory feedback technology.
The invention solves the technical problems through the following technical scheme:
the invention provides wearable protective equipment based on an individual intelligent bone conduction auditory feedback technology, which is characterized by comprising a flexible shell, wherein the flexible shell is used for being attached to the forehead of an individual, and a control module, at least one stereo bone conduction sound transmission vibrator and at least one electroencephalogram sensor are arranged on the flexible shell;
the control module is also used for controlling the stereo bone conduction sound transmission vibrator to send an audio stimulation signal, and the electroencephalogram sensor is used for collecting an electroencephalogram signal of a forehead area and transmitting the collected electroencephalogram signal to the control module;
the control module is also used for extracting cognitive ability characteristics and emotional state characteristics in the cognitive function of the individual soldier based on the electroencephalogram signals collected when the brain of the individual soldier is in a working state, matching the most suitable audio stimulation signals corresponding to the cognitive ability characteristics and the emotional state characteristics in the electroencephalogram database based on the analyzed cognitive ability characteristics and emotional state characteristics and the corresponding relations of the psychological, semantic and emotional characteristics of the audio signals in the audio database from the audio database, and playing the most suitable audio stimulation signals to the individual soldier through a stereo bone conduction sound transmission vibrator;
the control module is also used for extracting sleep quality characteristics in the cognitive function of the individual soldier based on the electroencephalogram signals collected when the brain of the individual soldier is in a sleep state, matching the most suitable audio stimulation signals corresponding to the sleep quality characteristics in the electroencephalogram database based on the analyzed sleep quality characteristics and the corresponding relation between the sleep quality characteristics in the electroencephalogram database and the psychological, semantic and emotional characteristics of the audio signals in the audio database, and playing the most suitable audio stimulation signals to the individual soldier through the stereo bone conduction sound transmission vibrator so as to achieve the purpose of improving and enhancing the cognitive ability of the individual soldier in the deep sea closed environment.
Preferably, a first skin pressure sensor is arranged on the left side of the flexible shell, and a second skin pressure sensor is arranged on the right side of the flexible shell;
the first skin pressure sensor is used for detecting the pressure between the left side of the flexible shell and the skin and transmitting a first pressure value to the control module, and the second skin pressure sensor is used for detecting the pressure between the right side of the flexible shell and the skin and transmitting a second pressure value to the control module;
the control module is used for judging whether the first pressure value is lower than a set threshold or whether the second pressure value is lower than the set threshold, and sending out warning information that the equipment is in poor contact with the skin when any pressure value is lower than the set threshold, otherwise sending out information that the equipment is in good contact with the skin.
Preferably, aiming at the specific requirement of intelligent matching of individual brain activity signals and sound signals, the electroencephalogram database and the audio database corresponding to the electroencephalogram database are established through a large number of cognitive experiments, the cognitive ability related evaluation features are extracted from the electroencephalogram signals, the psychology, the semantics, the emotion and other related features are extracted from the sound signals, and a typical controllable sparse correlation analysis technology is provided to realize the intelligent quantitative matching of the electroencephalogram database and the sound signals:
giving an electroencephalogram signalXObtaining itpDimensional sparse modeling coefficient vectora=(a 1,a 2,…a p ) Providing a sparse performance optimization index SP to control the sparse degree, constraining most components to 0, and specifyingSP(a) =1 to realize automatic extraction of EEG signalsXIs the most critical component of (a) a,SP(a) The calculation method of (a) is expressed as:
similarly, for all audio data in the audio databaseY n （n=1,2,…N) Extracting itqDimensional sparse modeling coefficient vectorb n =(b n1,b n2,…b nq )；
Establishing association between electroencephalogram and audio data by expanding typical correlation analysis CCA method, and defining covariance matrix of electroencephalogram signals and audio signals asC ab （p×q) Then, the optimized form of the three cases is:
in the formula (I), the compound is shown in the specification,λweights for sparse performance optimization terms;is a constant number of times, and is,the larger the correlation of the components allowed by the model is, the higher the correlation of the components is, and the electroencephalogram database and the audio database are associated together on the premise of ensuring the integrity of the components by optimizing the objective function.
Preferably, the control module is further configured to pre-process the acquired electroencephalogram signal to remove noise and artifact interference, adaptively find a center position of each frequency band according to a harmonic rule of tone quality of the electroencephalogram signal, calculate a CQT spectrum according to the center position of each frequency band, extract a characteristic parameter of the frequency spectrum of each frequency band to construct a specific tone quality, and analyze a cognitive ability characteristic, an emotional state characteristic, and a sleep quality characteristic according to the specific tone quality.
Preferably, the control module is used for analyzing the acquired electroencephalogram signals by using a cognitive ability quantitative detection method based on EEG sparse modeling to obtain cognitive ability characteristics.
Preferably, the control module is configured to analyze the acquired electroencephalogram signal by using an emotion quantitative detection method for audio event point-related evoked potentials AERPs to obtain an emotional state characteristic.
Preferably, the control module is configured to analyze the acquired electroencephalogram signal by using a sleep quality quantitative detection method of hilbert-yellow transform to obtain a sleep quality characteristic.
Preferably, a first stereo bone conduction sound transmission vibrator and a first electroencephalogram sensor for a left sound channel are arranged on the left side of the flexible shell, a second stereo bone conduction sound transmission vibrator and a second electroencephalogram sensor for a right sound channel are arranged on the right side of the flexible shell, and a third electroencephalogram sensor is arranged in the middle of the flexible shell;
the control module is also used for controlling the first stereo bone conduction sound transmission vibrator and the second stereo bone conduction sound transmission vibrator to send audio stimulation signals, and the first brain electrical sensor, the second brain electrical sensor and the third brain electrical sensor are used for collecting brain electrical signals of a forehead area and transmitting the collected brain electrical signals to the control module;
the control module is also used for playing the optimum audio stimulation signals corresponding to the optimum audio stimulation signals from the audio database to the individual soldier through the first stereo bone conduction sound transmission vibrator and the second stereo bone conduction sound transmission vibrator.
Preferably, the first stereo bone conduction sound transmission vibrator, the first brain electrical sensor and the first skin pressure sensor are bonded on the left side of the flexible shell through a first bonding layer, the second stereo bone conduction sound transmission vibrator, the second brain electrical sensor and the second skin pressure sensor are bonded on the right side of the flexible shell through a second bonding layer, and the third brain electrical sensor is bonded on the middle position of the flexible shell through a third bonding layer.
On the basis of the common knowledge in the field, the above preferred conditions can be combined randomly to obtain the preferred embodiments of the invention.
The positive progress effects of the invention are as follows:
the invention develops an individual intelligent bone conduction auditory feedback principle prototype aiming at the problems of cognitive decline, low emotion and the like of personnel in a long-distance voyage medium and deep sea closed environment based on a bone conduction sound transmission technology and an artificial intelligence technology, verifies the recognition accuracy and the intervention effectiveness, reduces the monitoring and screening thresholds of the cognitive decline and the low emotion, achieves demonstration and popularization in the deep sea closed space operation personnel crowd environment, and even achieves demonstration and popularization in the deep sea closed space operation personnel crowd environment during epidemic situations.
According to the invention, the wearable equipment for individual intelligent bone conduction auditory feedback is constructed, the effective rate of intervention is up to more than 80%, the real-time monitoring of cognitive ability, emotional state and sleep quality is realized, and the early recognition accuracy rate of cognitive ability decline and emotional decline is up to more than 90%.
Drawings
Fig. 1 is a wearing schematic view of wearable protective equipment according to a preferred embodiment of the invention.
Fig. 2 is a schematic structural diagram of a wearable protective device according to a preferred embodiment of the invention.
Fig. 3 is a block diagram of the auditory feedback technique and training paradigm based on brain activity and voice intelligent matching technique according to the preferred embodiment of the present invention.
FIG. 4 is a diagram of the process of analyzing the frequency band of the brain electrical signal based on CQT transformation according to the preferred embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
As shown in fig. 1 and 2, this embodiment provides a wearable protective equipment based on individual intelligent bone conduction auditory feedback technology, and it includes being used for pasting flexible shell 1 on individual's forehead, be equipped with control module 2 on the flexible shell 1, the left side of flexible shell 1 is equipped with first stereo bone conduction sound transducer 3, first brain electric sensor 4 and the first skin pressure sensor 5 to the left channel, the right side of flexible shell 1 is equipped with the second stereo bone conduction sound transducer 6, the second brain electric sensor 7 and the second skin pressure sensor 8 to the right channel, the intermediate position department of flexible shell 1 is equipped with third brain electric sensor 9.
First stereo bone conduction transaudient oscillator 3, first brain electric sensor 4 and first skin pressure sensor 5 bond in the left side of flexible enclosure 1 through first bond line 10, second stereo bone conduction transaudient oscillator 6, second brain electric sensor 7 and second skin pressure sensor 8 bond in the right side of flexible enclosure 1 through second bond line 12, third brain electric sensor 9 bonds in the intermediate position department of flexible enclosure 1 through third bond line 11.
First skin pressure sensor 5 is used for detecting the pressure size between flexible housing 1 left side and the skin to transmit first pressure value to control module 2, second skin pressure sensor 8 is used for detecting the pressure size between flexible housing 1 right side and the skin, and transmits the second pressure value to control module 2.
The control module 2 is used for judging whether the first pressure value is lower than a set threshold or whether the second pressure value is lower than the set threshold, when any pressure value is lower than the set threshold (the first pressure value is lower than the set threshold, the second pressure value is lower than the set threshold, or the first pressure value is lower than the set threshold and the second pressure value is lower than the set threshold), the equipment is not well contacted with the skin of the individual soldier, warning information that the equipment is not well contacted with the skin is sent out to remind a user of adjustment, and otherwise, information that the equipment is well contacted with the skin is sent out.
As shown in fig. 3, the control module 2 is further configured to control the first stereo bone conduction sound transmission vibrator 3 and the second stereo bone conduction sound transmission vibrator 6 to send audio stimulation signals, and the first electroencephalogram sensor 4, the second electroencephalogram sensor 7, and the third electroencephalogram sensor 9 are configured to acquire electroencephalogram signals of a forehead area and transmit the acquired electroencephalogram signals to the control module 2.
The control module 2 is also used for extracting cognitive competence characteristics and emotional state characteristics in the cognitive function of the individual soldier based on the electroencephalogram signals collected when the brain of the individual soldier is in a working state, matching the most suitable audio stimulation signals corresponding to the cognitive competence characteristics and the emotional state characteristics in the electroencephalogram database based on the analyzed cognitive competence characteristics and emotional state characteristics and the corresponding relations of the psychological, semantic and emotional characteristics of the audio signals in the audio database from the audio database, and playing the most suitable audio stimulation signals to the individual soldier through the first stereo bone conduction sound transmission vibrator 3 and the second stereo bone conduction sound transmission vibrator 6 so as to achieve the purpose of improving and enhancing the cognitive competence of the individual soldier in the deep sea closed environment.
The control module 2 is also used for extracting sleep quality characteristics in the cognitive function of the individual soldier based on the electroencephalogram signals collected when the brain of the individual soldier is in a sleep state, matching the most suitable audio stimulation signals corresponding to the sleep quality characteristics in the electroencephalogram database based on the analyzed sleep quality characteristics and the corresponding relation between the sleep quality characteristics in the electroencephalogram database and the psychological, semantic and emotional characteristics of the audio signals in the audio database, and playing the most suitable audio stimulation signals to the individual soldier through the first stereo bone conduction sound transmission vibrator 3 and the second stereo bone conduction sound transmission vibrator 6 so as to achieve the purpose of improving the cognitive ability of the individual soldier in the deep sea closed environment.
As shown in fig. 4, the control module 2 preprocesses the acquired electroencephalogram signal to remove noise and artifact interference, adaptively finds the center position of each frequency band according to the harmonic rule of the timbre (time) of the electroencephalogram signal, calculates the CQT spectrum according to the center position of each frequency band, extracts the frequency spectrum characteristic parameters of each frequency band to construct specific timbre, and can perform refined modeling for individuals after the individual characteristics of the electroencephalogram signal time can be quickly found through the CQT.
And analyzing the cognitive ability characteristic, the emotional state characteristic and the sleep quality characteristic according to the specific tone quality. Specifically, the acquired electroencephalogram signals are analyzed by a cognitive ability quantitative detection method based on EEG sparse modeling to obtain cognitive ability characteristics, the acquired electroencephalogram signals are analyzed by an emotion quantitative detection method of audio event point-related evoked potential AERP to obtain emotion state characteristics, and the acquired electroencephalogram signals are analyzed by a sleep quality quantitative detection method of Hilbert-Huang Transform (HHT) to obtain sleep quality characteristics.
Performing personalized feature extraction on the electroencephalogram signals according to cognitive functions by a sparse canonical correlation analysis method, wherein the personalized features comprise cognitive ability features, emotional state features and sleep quality features, performing correlation analysis on the extracted personalized features and the audio stimulation signals, finding out the audio stimulation signals most related to the current electroencephalogram signals as the most suitable audio stimulation signals, storing the electroencephalogram signals corresponding to the extracted personalized features into an electroencephalogram database, storing the matched most suitable audio stimulation signals into an audio database, and establishing the corresponding matching relationship between the electroencephalogram signals in the electroencephalogram database and the most suitable audio stimulation signals in the audio database.
The brain electrical signal and sound signal intelligent matching technology based on sparse canonical correlation analysis comprises the following steps: the intelligent auditory feedback method based on the intelligent matching of the brain state and the sound of the person is mainly characterized in that a brain wave database and an audio database corresponding to the brain wave database are established through data collected by a large number of cognitive experiments, the characteristics of brain electric signals related to the cognitive ability are extracted for the brain electric signals, the characteristics of psychology, semantics, emotion and the like of the brain electric signals are extracted for the audio electric signals, and the brain electric signals and the audio electric signals are associated through sparse typical correlation analysis. When analyzing the correlation between different types of data, a common method is Canonical Correlation Analysis (CCA), which can better solve the problem of correlation Analysis between multivariate different modality information.
In intelligent auditory feedback, sound signals of at least three matching situations need to be found to realize diversified auditory conditioning, including: strong matching, high correlation between sound and electroencephalogram, resonance conditioning and cognitive ability improvement; the sound is irrelevant to the brain electricity, so that relaxed conditioning is realized, the mental stress and the emotional deterioration are relieved, and the sleep quality is improved; and negative matching is carried out, and sound is inversely related to electroencephalogram, so that antiresonance conditioning is realized, and negative emotion and cognitive disorder tendency are corrected. Aiming at the specific requirement of intelligent matching of the brain activity signal and the sound signal of the person, firstly, establishing an electroencephalogram database and an audio database corresponding to the electroencephalogram database through a large number of cognitive experiments; extracting cognitive ability related evaluation characteristics from the electroencephalogram signals, and extracting psychology, semantics, emotion and other related characteristics from the sound signals. Moreover, a typical Sparse correlation analysis (SPCCA) technology is proposed to achieve intelligent quantitative matching of the two.
Given a segment of electroencephalogram signalXFirstly, an improved sparse decomposition method with controllable sparsity is provided to obtain the methodpDimensional sparse modeling coefficient vectora=(a 1,a 2,…a p ). Providing a sparse performance optimization index SP (sparse Performance) to control the sparse degree, constraining most components to 0, and specifyingSP(a) =1 to implement automatic extractionXOne of the most critical components.SP(a) The calculation of (c) can be expressed as:
similarly, for all audio data in the sound bankY n （n=1,2,…N) Extracting itqDimensional sparse modeling coefficient vectorb n =(b n1,b n2,…b nq )。
Then, establishing correlation between electroencephalogram and audio data by expanding a typical correlation analysis CCA method, and defining a covariance matrix of the electroencephalogram signal and the audio signal asC ab （p×q) Then, the optimized form of the three cases is:
in the formula (I), the compound is shown in the specification,λweights for sparse performance optimization terms;is a constant number of times, and is,the larger the correlation of the component components allowed by the model is, the higher the correlation of the component components is, and by optimizing the objective function, the electroencephalogram database and the audio database can be intelligently associated on the premise of ensuring the integrity of the component components.
When a new electroencephalogram signal is faced, the most needed audio data are intelligently matched according to three targets of auditory feedback: in the task execution, aiming at the requirements of keeping alert and improving cognitive ability, strong matching retrieval is implemented, the audio most relevant to brain waves is found, and resonance conditioning is implemented; meanwhile, aiming at possible emotional deterioration and cognitive errors, negative matching retrieval is implemented, an audio frequency opposite to brain waves is found, and anti-resonance conditioning is implemented; while at rest, irrelevant audio is searched by adopting weak matching, and relaxed conditioning is implemented; through the personalized and diversified auditory conditioning realized by intelligent matching, the cognitive state of an individual soldier can be greatly improved or enhanced, and the operational capacity is improved.
According to the invention, through research, analysis and simulation of the cognitive experiment research of the deep-sea closed space environment, the neural mechanism and the cognitive rule which can reflect the cognitive decline and the emotional depression of the operating personnel under the high stress state of the deep-sea closed space environment are explored, and on the basis, the key technology and equipment research of real-time evaluation of the cognitive ability of the operating personnel in the deep-sea closed space and personalized non-drug intervention is developed, so that theoretical and technical support is provided for the maintenance and improvement of the brain operation ability of the operating personnel.
In the invention, the sound therapy influences the regulation and control of the nervous system of the human body and the release of chemical transmitters and the like through special sound signals, so that the cognitive ability of the human body can be improved, and the mental depression can be relieved. In view of the characteristics of no wound, safety, high efficiency, convenience, no side effect and the like, the acoustic feedback is particularly suitable for carrying out cognitive ability improvement training and emotion regulation on operators.
The bone conduction of sound is that the inner ear is passed to with the vibration of sound wave through skull and jawbone etc. can realize clear sound reduction in noisy environment, and can not influence other people because the sound wave spreads in the air, and specially adapted narrow and small space has characteristics such as secret, low-loss, portable and safety. Therefore, the sound therapy adopting bone conduction is particularly suitable for the cabin environment of the deep sea closed space, and the research on the personalized sound therapy has great significance for ensuring the operation capability of operators.
The invention provides a model machine which combines a bone conduction sound transmission technology and a sound therapy, researches an individualized cognition ability real-time monitoring technology and an intelligent matching technology of brain activity signals and sounds, and develops an individual intelligent bone conduction sound feedback intervention technology and principle. The technology has the characteristics of no wound, portability, safety, strong pertinence, no interference to others and the like. The research uses brain science as a breakthrough, combines the brain science with an acoustic technology, develops innovative brain protection and cognition enhancement technology, creates novel wearable intelligent protective equipment, effectively reduces the negative influence on the mind and body of an operator caused by long-term deep sea closed environment operation, improves the cognitive ability of the operator, improves the emotional state of the operator, practically improves the operation ability of the operator, and has important theoretical significance and practical application value.
While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that these are by way of example only, and that the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the spirit and scope of the invention, and these changes and modifications are within the scope of the invention.
Claims (8)
1. A wearable protective device based on an individual intelligent bone conduction auditory feedback technology is characterized by comprising a flexible shell which is used for being attached to the forehead of an individual, wherein a control module, at least one stereo bone conduction sound transmission vibrator and at least one electroencephalogram sensor are arranged on the flexible shell;
the control module is also used for controlling the stereo bone conduction sound transmission vibrator to send an audio stimulation signal, and the electroencephalogram sensor is used for collecting an electroencephalogram signal of a forehead area and transmitting the collected electroencephalogram signal to the control module;
the control module is also used for extracting cognitive competence characteristics and emotional state characteristics in the cognitive function of the individual soldier based on the electroencephalogram signals collected when the brain of the individual soldier is in a working state, and based on the analyzed cognitive competence characteristics and emotional state characteristics and the one-to-one correspondence relationship between the cognitive competence characteristics and the emotional state characteristics in the electroencephalogram database and the audio signals containing psychological, semantic and emotional characteristics in the audio database, namely the two characteristics of the cognitive competence characteristics and the emotional state characteristics in the electroencephalogram database are correspondingly corresponding to the audio signals in the audio database, and the most suitable audio stimulation signals corresponding to the collected electroencephalogram signals are matched from the audio database and are played to the individual soldier through a stereo bone conduction sound transducer;
the control module is also used for extracting sleep quality characteristics in the cognitive function of the individual soldier based on the electroencephalogram signals collected when the brain of the individual soldier is in a sleep state, matching the most suitable audio stimulation signals corresponding to the collected electroencephalogram signals from the audio database based on the analyzed sleep quality characteristics and the corresponding relation between the sleep quality characteristics in the electroencephalogram database and the psychological, semantic and emotional characteristics of the audio signals in the audio database, and playing the most suitable audio stimulation signals to the individual soldier through the stereo bone conduction sound transmission vibrator so as to achieve the purpose of improving and enhancing the cognitive ability of the individual soldier in the deep sea closed environment;
aiming at the specific requirements of intelligent matching of individual brain activity signals and sound signals, an electroencephalogram database and an audio database corresponding to the electroencephalogram database are established through a large number of cognitive experiments, cognitive function related evaluation features are extracted from the electroencephalogram signals, psychological, semantic and emotion related features are extracted from the sound signals, and a typical controllable sparse correlation analysis technology is provided to realize intelligent quantitative matching of the electroencephalogram database and the sound signals:
giving an electroencephalogram signalXObtaining itpDimensional sparse modeling coefficient vectora=(a 1,a 2,…a p ) Providing a sparse performance optimization index SP to control the sparse degree, constraining most components to 0, and specifyingSP(a) =1 to realize automatic extraction of EEG signalsXIs the most critical component of (a) a,SP(a) The calculation method of (a) is expressed as:
similarly, for all audio data in the audio databaseY n （n=1,2,…N) Extracting itqDimensional sparse modeling coefficient vectorb n =(b n1,b n2,…b nq )；
Establishing association between electroencephalogram and audio data by expanding typical correlation analysis CCA method, and defining covariance matrix of electroencephalogram signals and audio signals asC ab （p×q) Then, the optimized form of the three cases is:
in the formula (I), the compound is shown in the specification,λweights for sparse performance optimization terms;is a constant number of times, and is,the larger the correlation of the components allowed by the model is, the higher the correlation of the components is, and the electroencephalogram database and the audio database are associated together on the premise of ensuring the integrity of the components by optimizing the objective function.
2. The wearable protective equipment based on individual intelligent bone conduction auditory feedback technology as claimed in claim 1, wherein a first skin pressure sensor is arranged on the left side of the flexible casing, and a second skin pressure sensor is arranged on the right side of the flexible casing;
the first skin pressure sensor is used for detecting the pressure between the left side of the flexible shell and the skin and transmitting a first pressure value to the control module, and the second skin pressure sensor is used for detecting the pressure between the right side of the flexible shell and the skin and transmitting a second pressure value to the control module;
the control module is used for judging whether the first pressure value is lower than a set threshold or whether the second pressure value is lower than the set threshold, and sending out warning information that the equipment is in poor contact with the skin when any pressure value is lower than the set threshold, otherwise sending out information that the equipment is in good contact with the skin.
3. The wearable protective equipment based on the individual intelligent bone conduction auditory feedback technology as claimed in claim 1, wherein the control module is further configured to preprocess the acquired electroencephalogram signals to remove noise and artifact interference, adaptively find the center position of each frequency band according to the harmonic law of the tone quality of the electroencephalogram signals, calculate a CQT spectrum according to the center position of each frequency band, extract the characteristic parameters of the frequency spectrum of each frequency band to construct specific tone quality, analyze cognitive competence characteristics, emotional state characteristics and sleep quality characteristics according to the specific tone quality, the CQT represents a filter bank in which the center frequency is exponentially distributed, the filter bandwidths are different, but the ratio of the center frequency to the bandwidth is constant Q.
4. The wearable protective equipment based on individual soldier's intelligent bone conduction auditory feedback technology of claim 1, wherein the control module is used for analyzing the acquired electroencephalogram signals by utilizing a cognitive ability quantitative detection method based on EEG sparse modeling to obtain cognitive ability characteristics.
5. The wearable protective equipment based on individual intelligent bone conduction auditory feedback technology of claim 1, wherein the control module is used for analyzing the acquired electroencephalogram signals by utilizing an emotion quantitative detection method of audio event point-related evoked potentials (AERPs) to obtain emotional state characteristics.
6. The wearable protective equipment based on individual soldier's intelligent bone conduction auditory feedback technology of claim 1, wherein the control module is used for analyzing the acquired electroencephalogram signals by using a sleep quality quantitative detection method of Hilbert-Huang transform to obtain sleep quality characteristics.
7. The wearable protective equipment based on the individual intelligent bone conduction auditory feedback technology as claimed in claim 1, wherein a first stereo bone conduction sound transducer and a first brain electrical sensor for a left channel are arranged on the left side of the flexible shell, a second stereo bone conduction sound transducer and a second brain electrical sensor for a right channel are arranged on the right side of the flexible shell, and a third brain electrical sensor is arranged in the middle of the flexible shell;
the control module is also used for controlling the first stereo bone conduction sound transmission vibrator and the second stereo bone conduction sound transmission vibrator to send audio stimulation signals, and the first brain electrical sensor, the second brain electrical sensor and the third brain electrical sensor are used for collecting brain electrical signals of a forehead area and transmitting the collected brain electrical signals to the control module;
the control module is also used for playing the optimum audio stimulation signals corresponding to the optimum audio stimulation signals from the audio database to the individual soldier through the first stereo bone conduction sound transmission vibrator and the second stereo bone conduction sound transmission vibrator.
8. The wearable protective equipment based on individual intelligent bone conduction auditory feedback technology of claim 7, wherein the first stereo bone conduction sound transducer, the first brain electrical sensor and the first skin pressure sensor are bonded on the left side of the flexible shell through a first adhesive layer, the second stereo bone conduction sound transducer, the second brain electrical sensor and the second skin pressure sensor are bonded on the right side of the flexible shell through a second adhesive layer, and the third brain electrical sensor is bonded on the middle position of the flexible shell through a third adhesive layer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010384240.1A CN111345784B (en) | 2020-05-09 | 2020-05-09 | Wearable protective equipment based on individual intelligent bone conduction auditory feedback technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010384240.1A CN111345784B (en) | 2020-05-09 | 2020-05-09 | Wearable protective equipment based on individual intelligent bone conduction auditory feedback technology |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111345784A CN111345784A (en) | 2020-06-30 |
| CN111345784B true CN111345784B (en) | 2020-08-14 |
Family
ID=71189887
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010384240.1A Active CN111345784B (en) | 2020-05-09 | 2020-05-09 | Wearable protective equipment based on individual intelligent bone conduction auditory feedback technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111345784B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111930221A (en) * | 2020-07-14 | 2020-11-13 | 苏州宣佑科技有限公司 | Method for judging and switching low-power-consumption mode of intelligent equipment |
| CN112263768A (en) * | 2020-10-21 | 2021-01-26 | 上海市精神卫生中心（上海市心理咨询培训中心） | Anxiety treatment method and system based on auditory rhythm |
| CN112675010A (en) * | 2020-12-03 | 2021-04-20 | 未来穿戴技术有限公司 | Vibration control circuit, massage apparatus, and control method of massage apparatus |
| CN113368365A (en) * | 2021-05-21 | 2021-09-10 | 苏州声动医疗科技有限公司 | Sound vibration regulation and control equipment and method for brain function monitoring, headrest and head-mounted equipment |
| CN113729711B (en) * | 2021-09-30 | 2023-10-13 | 深圳航天科技创新研究院 | Electroencephalogram signal analysis method, device, equipment and storage medium |
| CN113986010B (en) * | 2021-10-27 | 2024-04-16 | 京东方科技集团股份有限公司 | Individual control method and related equipment |
| GB202215652D0 (en) * | 2022-10-21 | 2022-12-07 | Univ Newcastle | A method of closed-loop modulation of audio data for neural oscillation suppression or enhancement |
| CN116172559B (en) * | 2023-02-22 | 2023-11-24 | 中国人民解放军海军特色医学中心 | Psychological stress assessment method and system based on multiple physiological parameters |
Family Cites Families (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN203342164U (en) * | 2013-03-19 | 2013-12-18 | 北京思源佳创技术开发有限公司 | Intelligent feeding-back type relaxation training system |
| CN105794223B (en) * | 2013-10-02 | 2019-08-09 | 声音睡眠有限责任公司 | Multi-functional headwear |
| CN107427716B (en) * | 2015-01-05 | 2020-05-22 | 骷髅头有限公司 | Method and system for optimizing and training human performance |
| EP3515552A4 (en) * | 2016-09-19 | 2019-09-25 | NYX Technologies Ltd | Multifunctional closed loop neuro feedback stimulating device and methods thereof |
| CN106345034A (en) * | 2016-11-09 | 2017-01-25 | 武汉智普天创科技有限公司 | Device based on brain electricity acquisition terminal for cognitive emotion regulation |
| KR20190099006A (en) * | 2016-12-16 | 2019-08-23 | 엘화 엘엘씨 | Systems and Methods for Enhancing Learning Using Neuromodulation |
| CA3024741A1 (en) * | 2017-03-02 | 2018-09-07 | Sana Health, Inc. | Methods and systems for modulating stimuli to the brain with biosensors |
| CN107126615A (en) * | 2017-04-20 | 2017-09-05 | 重庆邮电大学 | Music induced hypnotic method and system based on EEG signals |
| CN108153414B (en) * | 2017-12-19 | 2021-06-15 | 同济大学 | Method, system and device for identifying steady-state visual evoked potential signals |
| CN209932734U (en) * | 2018-11-28 | 2020-01-14 | 华南理工大学 | Head wearable equipment |
| CN109620257B (en) * | 2018-11-28 | 2020-12-22 | 华南理工大学 | Mental state intervention and regulation system based on biofeedback and working method thereof |
| CN109688500B (en) * | 2018-12-25 | 2019-12-31 | 湖北工业大学 | Intelligent sleep earphone based on user sleep characteristic model and use method |
| CN110464344A (en) * | 2019-08-16 | 2019-11-19 | 兰州大学 | The method for collecting the device of eeg signal acquisition and music and its playing music |
-
2020
- 2020-05-09 CN CN202010384240.1A patent/CN111345784B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111345784A (en) | 2020-06-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111345784B (en) | Wearable protective equipment based on individual intelligent bone conduction auditory feedback technology | |
| US11595749B2 (en) | Systems and methods for dynamic noise reduction | |
| GB2608690A (en) | Personalized mental state adjustment system and method based on brainwave music | |
| CN108882851B (en) | Frequency analysis feedback system and method | |
| AU2014232300A1 (en) | Speaker and light source responsive to states | |
| CN105167785A (en) | Fatigue monitoring and early warning system and method based on digital helmet | |
| Sengupta et al. | A multimodal system for assessing alertness levels due to cognitive loading | |
| Dong et al. | A WPCA-based method for detecting fatigue driving from EEG-based internet of vehicles system | |
| US20230328417A1 (en) | Secure identification methods and systems | |
| Zeng et al. | Classifying driving fatigue by using EEG signals | |
| Ali et al. | A single-channel wireless EEG headset enabled neural activities analysis for mental healthcare applications | |
| Zhu et al. | Design and evaluation of the mental relaxation VR scenes using forehead EEG features | |
| Purnamasari et al. | Mobile EEG based drowsiness detection using K-nearest neighbor | |
| EP3804355B1 (en) | Hearing protection and noise recording system and method | |
| CN206007247U (en) | A kind of bone conduction feedback device based on the anti-fatigue monitoring of brain wave | |
| CN213030626U (en) | Wearable protective equipment | |
| Saia et al. | Influencing brain waves by evoked potentials as biometric approach: taking stock of the last six years of research | |
| CN115944307B (en) | Epileptic brain electrical signal monitoring system and method based on space-time converter | |
| Chowdhury et al. | Stress detection through single channel real-time EEG signal from temporal lobe using machine learning model | |
| Al-Turabi et al. | Brain computer interface for wheelchair control in smart environment | |
| CN108852313A (en) | Non-interference Intellisense method and system based on radar | |
| CN114903247A (en) | Intelligent safety helmet system based on EEG-NIRS and MHGAT | |
| Wang et al. | What you wear know how you feel: An emotion inference system with multi-modal wearable devices | |
| CN206400473U (en) | A kind of wearable social sensory perceptual system | |
| Karam et al. | Neural network for recognition of brain wave signals |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |