CN116616794A - Underwater operator fatigue adjustment method and system based on electroencephalogram signals - Google Patents
Underwater operator fatigue adjustment method and system based on electroencephalogram signals Download PDFInfo
- Publication number
- CN116616794A CN116616794A CN202310567274.8A CN202310567274A CN116616794A CN 116616794 A CN116616794 A CN 116616794A CN 202310567274 A CN202310567274 A CN 202310567274A CN 116616794 A CN116616794 A CN 116616794A
- Authority
- CN
- China
- Prior art keywords
- fatigue
- acquisition
- underwater
- electroencephalogram
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 14
- 210000004556 brain Anatomy 0.000 claims abstract description 27
- 238000007781 pre-processing Methods 0.000 claims abstract description 19
- 238000013441 quality evaluation Methods 0.000 claims description 21
- 238000000605 extraction Methods 0.000 claims description 13
- 238000001914 filtration Methods 0.000 claims description 12
- 238000011156 evaluation Methods 0.000 claims description 8
- 230000000284 resting effect Effects 0.000 claims description 7
- 230000004424 eye movement Effects 0.000 claims description 6
- 230000003183 myoelectrical effect Effects 0.000 claims description 6
- 238000005192 partition Methods 0.000 claims description 3
- 230000003925 brain function Effects 0.000 claims description 2
- 230000000638 stimulation Effects 0.000 abstract 1
- 238000001514 detection method Methods 0.000 description 6
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 230000004630 mental health Effects 0.000 description 1
- 238000001303 quality assessment method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
- A61B5/372—Analysis of electroencephalograms
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
- A61M21/00—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
- A61M21/00—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
- A61M2021/0005—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis by the use of a particular sense, or stimulus
- A61M2021/0027—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis by the use of a particular sense, or stimulus by the hearing sense
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2123/00—Data types
- G06F2123/02—Data types in the time domain, e.g. time-series data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
Abstract
The invention discloses an electroencephalogram signal-based fatigue adjustment method and system for underwater operators: s1, acquiring brain electrical signals of N acquisition periods corresponding to each acquisition point in a human brain vision area and an auditory area of an underwater operator; s2, evaluating signal quality of the brain electrical signals of each acquisition period corresponding to the ith acquisition point, and reserving if the quality is qualified; s3, judging whether a collection period M reserved corresponding to the ith collection point meets the condition or not, if not, entering S4, and if so, entering S5; s4, acquiring brain electrical signals of which the ith acquisition point corresponds to 2 (L-M) acquisition periods, and repeating the steps S2-S3; s5, judging whether i reaches (P+Q), if not, entering S6, and if so, entering S7; s6, i=i+1, repeating S2-S3; s7, preprocessing the brain electrical signals of M acquisition periods of each acquisition point, extracting features, calculating similar values, accumulating the similar values, judging whether the sum of the similar values reaches a set value, and if not, playing the corresponding stimulation audio of the fatigue state.
Description
Technical Field
The invention relates to the technical field of fatigue detection, in particular to an underwater operator fatigue adjusting method and system based on an electroencephalogram signal.
Background
An electroencephalogram (EEG) signal is used as a source of exploring information for the brain of a human body, and can reflect the fatigue state of the human body, so that the fatigue state of the human body is generally detected by the electroencephalogram signal in the prior art. The electroencephalogram signal is taken as a bioelectric signal, and can be generally obtained through the head-mounted electroencephalogram device, so that the electroencephalogram signal can be used as a brain-computer interface to perform information interaction with the head-mounted electroencephalogram device.
The environment of the deep sea closed cabin is poor, and underwater operators are in the environment for a long time, so that physical and mental health and operation capability of the underwater operators are easily affected, and fatigue states are caused, and in order to improve the operation capability of the underwater operators, fatigue detection and fatigue adjustment of the underwater operators are required. Based on the fatigue detection and fatigue adjustment technical scheme for the underwater operators is designed. Because the brain electrical signals are irregular and unstable and weaker than the electrocardiosignals, the electromyographic signals and the like, the brain electrical signals have the signal quality problem. Therefore, the signal quality of the electroencephalogram signals determines the accuracy of fatigue detection and fatigue adjustment of the underwater operator.
Disclosure of Invention
Aiming at the problems and the defects existing in the prior art, the invention provides an underwater operator fatigue adjusting method and system based on an electroencephalogram signal.
The invention solves the technical problems by the following technical proposal:
the invention provides an electroencephalogram signal-based fatigue adjustment method for underwater operators, which is characterized by comprising the following steps of:
s1, acquiring brain electrical signals of N acquisition periods corresponding to each acquisition point in P acquisition points of a human brain vision area and Q acquisition points of an auditory area of an underwater operator according to brain functional areas, wherein P, Q and N are positive integers;
s2, evaluating signal quality of the electroencephalogram signal of each acquisition period corresponding to the ith acquisition point, wherein the evaluation result is that the signal quality of the electroencephalogram signal of the acquisition period corresponding to the ith acquisition point is qualified, the electroencephalogram signal of the acquisition period corresponding to the ith acquisition point is reserved, otherwise, the electroencephalogram signal of the acquisition period corresponding to the ith acquisition point is discarded, i is a positive integer, i is not less than 1 and not more than (P+Q), and the initial value of i is 1;
s3, judging whether the number M of the acquisition periods of the reserved electroencephalogram signals corresponding to the ith acquisition point meets the condition L.ltoreq.M.ltoreq.N, if not, entering a step S4, if yes, entering a step S5, wherein L and M are positive integers;
s4, acquiring the brain electrical signals of 2 (L-M) acquisition periods corresponding to the ith acquisition point again, and repeatedly executing the steps S2-S3;
s5, judging whether i reaches (P+Q), if not, entering a step S6, and if so, entering a step S7;
s6, i=i+1, and repeatedly executing the steps S2-S3;
s7, preprocessing the electroencephalogram signals of each acquisition point of the underwater operator in M acquisition periods;
s8, carrying out feature extraction on the preprocessed electroencephalogram signals of each acquisition point of the underwater operator to obtain electroencephalogram signal features of each acquisition point, wherein the electroencephalogram signal features comprise electroencephalogram time domain features, electroencephalogram frequency domain features and electroencephalogram time-frequency features;
s9, calculating similarity values of each sub-characteristic value in the electroencephalogram signal characteristic of each acquisition point of the underwater worker and the corresponding reference sub-characteristic value in the fatigue-free set;
s10, counting whether the accumulated sum of the similar values reaches a set threshold value, if so, entering a step S11, and if not, entering a step S12;
s11, determining that the underwater staff is in a fatigue-free state;
s12, determining that the underwater worker is in a fatigue state, and playing stimulus audio corresponding to the fatigue state to adjust the fatigue state of the underwater worker.
Preferably, in step S12, after determining that the underwater operator is in a fatigue state, calculating a similarity value of each sub-feature value in the electroencephalogram signal feature of each acquisition point of the underwater operator and a corresponding reference sub-feature value in each degree fatigue subset in the fatigue set;
and counting the accumulated sums of the similar values corresponding to the fatigue subsets of each degree, selecting the degree grade fatigue corresponding to the maximum value of the accumulated sums from the accumulated sums as the fatigue grade of the underwater operator, and playing the stimulus audio corresponding to the fatigue grade to adjust the fatigue state of the underwater operator, wherein the fatigue grade comprises a mild fatigue grade, a moderate fatigue grade, a severe fatigue grade and a severe fatigue grade.
Preferably, when the fatigue level of the underwater operator is a mild fatigue level, only the stimulus audio corresponding to the mild fatigue level is played, when the fatigue level of the underwater operator is a moderate fatigue level, the stimulus audio corresponding to the moderate fatigue level is played, the reminding information that the fatigue level reaches the moderate fatigue level and takes care of rest is sent, when the fatigue level of the underwater operator is a severe fatigue level, the stimulus audio corresponding to the severe fatigue level is played, the warning information that the fatigue level reaches the severe fatigue and has to take care of is sent, and when the fatigue level of the underwater operator is a severe fatigue level, the stimulus audio corresponding to the severe fatigue level is played, the warning information that the fatigue level reaches the severe fatigue and has to take care of is sent to related personnel at the same time.
Preferably, in step S7, band-pass filtering and notch filtering are performed on the electroencephalogram signals of M acquisition periods of each acquisition point of the underwater operator, the filtered electroencephalogram signal data of M acquisition periods of each acquisition point is subtracted by the average value of the electroencephalogram signal data of the corresponding acquisition point when the underwater operator is in a resting state, and artifact removal is performed on the electroencephalogram signals of the subtracted average value of M acquisition periods of each acquisition point to remove eye movement artifacts and myoelectric noise in the electroencephalogram signals.
Preferably, in step S2, signal quality evaluation is performed on the electroencephalogram signal of each acquisition cycle corresponding to the ith acquisition point through signal-to-noise ratio.
The invention also provides an electroencephalogram signal-based fatigue adjusting system for underwater operators, which is characterized by comprising a first acquisition module, a quality evaluation module, a first judgment module, a second acquisition module, a second judgment module, a endowing module, a preprocessing module, a feature extraction module, a calculation module, a statistics judgment module, a determination module and a determination playing module;
the first acquisition module is used for acquiring brain electrical signals of N acquisition periods corresponding to each acquisition point in P acquisition points of a human brain vision area and Q acquisition points of an auditory area of an underwater operator according to a brain function partition, wherein P, Q and N are positive integers;
the quality evaluation module is used for evaluating the signal quality of the electroencephalogram signal of each acquisition period corresponding to the ith acquisition point, if the evaluation result is that the signal quality of the electroencephalogram signal of the acquisition period corresponding to the ith acquisition point is qualified, the electroencephalogram signal of the acquisition period corresponding to the ith acquisition point is reserved, otherwise, the electroencephalogram signal of the acquisition period corresponding to the ith acquisition point is discarded, i is a positive integer, i is not more than 1 and not more than (P+Q), and the initial value of i is 1;
the first judging module is used for judging whether the number M of the acquisition periods of the reserved EEG signals corresponding to the ith acquisition point meets the condition L which is less than or equal to M and less than or equal to N, if not, the second acquisition module is called, and if yes, the second judging module is called, and both L and M are positive integers;
the second acquisition module is used for acquiring 2 x (L-M) acquisition cycle electroencephalograms corresponding to the ith acquisition point, and repeatedly executing the quality evaluation module and the first judgment module;
the second judging module is used for judging whether the i reaches (P+Q), if not, the giving module is called, and if yes, the preprocessing module is called;
the giving module is used for giving i=i+1, and repeatedly executing the quality evaluation module;
the preprocessing module is used for preprocessing the electroencephalogram signals of M acquisition periods of each acquisition point of the underwater operator;
the characteristic extraction module is used for carrying out characteristic extraction on the preprocessed electroencephalogram signals of each acquisition point of the underwater operator in M acquisition periods to obtain electroencephalogram signal characteristics of each acquisition point, wherein the electroencephalogram signal characteristics comprise electroencephalogram time domain characteristics, electroencephalogram frequency domain characteristics and electroencephalogram time-frequency characteristics;
the calculating module is used for calculating the similarity value of each sub-characteristic value in the electroencephalogram signal characteristic of each acquisition point of the underwater worker and the corresponding reference sub-characteristic value in the fatigue-free set;
the statistics judging module is used for counting whether the accumulated sum of the similar values reaches a set threshold value, if yes, the determining module is called, and if no, the determining and playing module is called;
the determining module is used for determining that the underwater staff is in a fatigue-free state;
the determining and playing module is used for determining that the underwater worker is in a fatigue state and playing the stimulus audio corresponding to the fatigue state so as to adjust the fatigue state of the underwater worker.
Preferably, the determining and playing module is configured to calculate, after determining that the underwater operator is in a fatigue state, a similarity value between each sub-feature value in the electroencephalogram signal feature of each acquisition point of the underwater operator and a corresponding reference sub-feature value in each degree level fatigue subset in the fatigue set, count an accumulated sum of the similarity values corresponding to each degree level fatigue subset, select a degree level fatigue corresponding to a maximum value of the accumulated sum from the accumulated sums as a fatigue level of the underwater operator, and play a stimulus audio corresponding to the fatigue level to adjust the fatigue state of the underwater operator, where the fatigue level includes a mild fatigue level, a moderate fatigue level, a severe fatigue level and a severe fatigue level.
Preferably, the determining and playing module is configured to play only the stimulus audio corresponding to the mild fatigue level when the fatigue level of the underwater operator is the mild fatigue level, play the stimulus audio corresponding to the moderate fatigue level and send out the reminding information that the fatigue level has reached the moderate fatigue level and the rest is noticed when the fatigue level of the underwater operator is the severe fatigue level, play the stimulus audio corresponding to the severe fatigue level and send out the warning information that the fatigue level has reached the severe fatigue and the rest is necessary, and play the stimulus audio corresponding to the severe fatigue level and send out the warning information that the fatigue level has reached the severe fatigue and the rest is necessary when the fatigue level of the underwater operator is the severe fatigue level and send out the information to the related personnel at the same time.
Preferably, the preprocessing module is configured to perform band-pass filtering and notch filtering on the electroencephalogram signals of M acquisition periods of each acquisition point of the underwater operator, respectively subtract the electroencephalogram signal data average value of the corresponding acquisition point when the underwater operator is in a resting state from the filtered electroencephalogram signal data of the M acquisition periods of each acquisition point, and perform artifact removal on the electroencephalogram signals after the average value reduction of the M acquisition periods of each acquisition point to remove the eye movement artifact and myoelectric noise in the electroencephalogram signals.
Preferably, the quality evaluation module is configured to evaluate signal quality of the electroencephalogram signal of each acquisition cycle corresponding to the ith acquisition point through signal-to-noise ratio.
On the basis of conforming to the common knowledge in the field, the above preferred conditions can be arbitrarily combined to obtain the preferred examples of the invention.
The invention has the positive progress effects that:
according to the invention, firstly, signal quality evaluation is carried out on the acquired original electroencephalogram signals, only the electroencephalogram signals with qualified signal quality are reserved, the number of acquisition periods corresponding to the reserved electroencephalogram signals with qualified signal quality meets the number requirement, then preprocessing operation, feature extraction operation and similarity value calculation operation can be carried out on the electroencephalogram signals with M acquisition periods of each acquisition point, the underwater staff is determined to be in a fatigue-free state when the sum of the similarity values reaches a set value, and then the underwater staff is not required to be subjected to fatigue adjustment operation, otherwise, the underwater staff is determined to be in a fatigue state, and then the underwater staff is required to be subjected to fatigue adjustment operation, and stimulus audio corresponding to the fatigue state is played to adjust the fatigue state of the underwater staff. The electroencephalogram signals involved in fatigue detection and fatigue regulation are qualified electroencephalogram signals, so that the accuracy of fatigue detection and fatigue regulation of underwater operators is higher.
Drawings
Fig. 1 is a flowchart of an electroencephalogram signal-based fatigue adjusting method for underwater operators according to a preferred embodiment of the present invention.
Fig. 2 is a block diagram of a fatigue adjusting method for underwater operators based on brain electrical signals according to a preferred embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
As shown in fig. 1, the embodiment provides an electroencephalogram signal-based fatigue adjustment method for underwater operators, which comprises the following steps:
step 101, acquiring brain electrical signals of N (e.g. 20) acquisition periods corresponding to each acquisition point in P acquisition points of a human brain vision area and Q acquisition points of an auditory area of a certain underwater operator according to a brain functional partition, wherein P, Q and N are positive integers.
Step 102, evaluating signal quality of the electroencephalogram signal of each acquisition period corresponding to the ith acquisition point through signal-to-noise ratio, wherein the evaluation result is that the signal quality of the electroencephalogram signal of the acquisition period corresponding to the ith acquisition point is qualified, the electroencephalogram signal of the acquisition period corresponding to the ith acquisition point is reserved, otherwise, the electroencephalogram signal of the acquisition period corresponding to the ith acquisition point is discarded, i is a positive integer, i is not more than 1 and not more than (P+Q), and the initial value of i is 1.
For example: and evaluating the signal quality of the electroencephalogram signal of each acquisition period in 20 acquisition periods corresponding to the 1 st acquisition point through a signal-to-noise ratio, wherein the evaluation result is that the signal quality of the electroencephalogram signal of the 1 st acquisition period corresponding to the 1 st acquisition point is qualified, the electroencephalogram signal of the 1 st acquisition period corresponding to the 1 st acquisition point is reserved, the evaluation result is that the signal quality of the electroencephalogram signal of the 2 nd acquisition period corresponding to the 1 st acquisition point is unqualified, and the electroencephalogram signal of the 2 nd acquisition period corresponding to the 1 st acquisition point is discarded. Based on the operation, the electroencephalogram signals with 20 acquisition periods corresponding to the 1 st acquisition point are subjected to signal quality evaluation one by one, the electroencephalogram signals with acquisition periods with quality not meeting the requirements are removed, and the electroencephalogram signals with the acquisition periods with quality meeting the requirements are reserved.
Step 103, judging whether the number M of the acquisition periods of the reserved electroencephalogram signals corresponding to the ith acquisition point meets the condition L.ltoreq.M.ltoreq.N, if not, entering step 104, and if yes, entering step 105, wherein L and M are positive integers.
Step 104, acquiring the electroencephalogram signals of 2 (L-M) acquisition periods corresponding to the ith acquisition point again, and repeatedly executing the steps 102-103.
Step 105, judging whether i reaches (p+q), if not, proceeding to step 106, if yes, proceeding to step 107.
Step 106, i=i+1, repeating steps 102-103.
For example: l=15, N=20, judge the number M of acquisition cycle of the electroencephalogram signal that the 1 st acquisition point corresponds to and keep still is not less than 15 and not more than 20 of condition, if M=17, meet the condition, continue to carry on the signal quality assessment to the electroencephalogram signal of each acquisition cycle in 20 acquisition cycles that the 2 nd acquisition point corresponds to; if m=13, the condition is not satisfied, acquiring 2 x (15-13) electroencephalograms of the acquisition periods corresponding to the 1 st acquisition point, namely, 4 electroencephalograms of the acquisition periods, performing signal quality evaluation on the 4 electroencephalograms of the acquisition periods, evaluating that the signal quality of the electroencephalograms of the acquisition periods is qualified, judging that the number m=13+3=16 of the acquisition periods of the reserved electroencephalograms of the 1 st acquisition point satisfies the condition 15.ltoreq.m.ltoreq.20, and continuously performing signal quality evaluation on the electroencephalograms of each acquisition period in the 20 acquisition periods corresponding to the 2 nd acquisition point. Similarly, each acquisition point can correspondingly hold the electroencephalogram signals of acquisition cycles meeting the number requirements.
And 107, preprocessing the electroencephalogram signals of each acquisition point of the underwater operator in M acquisition periods. The specific pretreatment operation is as follows: band-pass filtering and notch filtering are carried out on the electroencephalogram signals of the M acquisition periods of each acquisition point of the underwater worker, the electroencephalogram signal data average value of the corresponding acquisition point when the underwater worker is in a resting state is subtracted from the filtered electroencephalogram signal data of the M acquisition periods of each acquisition point, and artifact removal is carried out on the electroencephalogram signals of the M acquisition periods of each acquisition point to remove eye movement artifacts and myoelectric noise in the electroencephalogram signals, so that the preprocessed electroencephalogram signals of the M acquisition periods of each acquisition point are obtained.
And step 108, performing feature extraction on the preprocessed electroencephalogram signals of each acquisition point of the underwater operator to obtain electroencephalogram signal features of each acquisition point, wherein the electroencephalogram signal features comprise electroencephalogram time domain features, electroencephalogram frequency domain features and electroencephalogram time-frequency features.
And 109, calculating the ratio of each sub-characteristic value in the electroencephalogram signal characteristic of each acquisition point of the underwater worker to the corresponding reference sub-characteristic value in the fatigue-free set as a similar value.
Step 110, counting whether the accumulated sum of the similar values reaches a set threshold, if yes, proceeding to step 111, otherwise proceeding to step 112.
Step 111, determining that the underwater worker is in a fatigue-free state, and at this time, performing fatigue adjustment operation on the underwater worker is not required.
Step 112, determining that the underwater worker is in a fatigue state, and at the moment, performing fatigue adjustment operation on the underwater worker, and playing stimulus audio corresponding to the fatigue state to adjust the fatigue state of the underwater worker.
In step 112, the function of applying different stimulus audios to adjust the fatigue state of the underwater worker based on different fatigue levels of the underwater worker may also be implemented, specifically: after determining that the underwater operator is in a fatigue state, calculating the ratio of each sub-characteristic value in the electroencephalogram signal characteristics of each acquisition point of the underwater operator to the corresponding reference sub-characteristic value in each degree level fatigue sub-set in the fatigue set as a similar value; and counting the accumulated sums of the similar values corresponding to the fatigue subsets of each degree, selecting the degree grade fatigue corresponding to the maximum value of the accumulated sums from the accumulated sums as the fatigue grade of the underwater operator, and playing the stimulus audio corresponding to the fatigue grade to adjust the fatigue state of the underwater operator, wherein the fatigue grade comprises a mild fatigue grade, a moderate fatigue grade, a severe fatigue grade and a severe fatigue grade.
The more optimized technical scheme is as follows:
1. when the fatigue level of the underwater operator is a light fatigue level, only playing the stimulus audio corresponding to the light fatigue level;
2. when the fatigue level of the underwater operator is a moderate fatigue level, playing a stimulus audio corresponding to the moderate fatigue level and sending out reminding information that the fatigue level reaches the moderate fatigue level and paying attention to rest;
3. when the fatigue grade of the underwater operator is a severe fatigue grade, playing the stimulus audio corresponding to the severe fatigue grade and sending out warning information that the fatigue grade reaches the severe fatigue and has to rest;
4. when the fatigue level of the underwater operator is a serious fatigue level, the stimulus audio corresponding to the serious fatigue level is played, the warning information that the fatigue level reaches the serious fatigue and has to be at rest is sent, and the information is sent to related personnel at the same time.
As shown in fig. 2, the embodiment further provides an electroencephalogram signal-based fatigue adjustment system for underwater operators, which comprises a first acquisition module 1, a quality evaluation module 2, a first judgment module 3, a second acquisition module 4, a second judgment module 5, a giving module 6, a preprocessing module 7, a feature extraction module 8, a calculation module 9, a statistics judgment module 10, a determination module 11 and a determination playing module 12.
The first acquisition module 1 is used for acquiring brain electrical signals of N acquisition periods corresponding to each acquisition point in P acquisition points of a human brain vision area and Q acquisition points of an auditory area of a certain underwater operator according to brain functional areas, and P, Q and N are positive integers.
The quality evaluation module 2 is configured to perform signal quality evaluation on the electroencephalogram signal of each acquisition period corresponding to the ith acquisition point through a signal-to-noise ratio, where the evaluation result is that the signal quality of the electroencephalogram signal of the acquisition period corresponding to the ith acquisition point is qualified, then the electroencephalogram signal of the acquisition period corresponding to the ith acquisition point is reserved, otherwise, the electroencephalogram signal of the acquisition period corresponding to the ith acquisition point is discarded, i is a positive integer, i is less than or equal to 1 and less than or equal to (p+q), and the i initial value is 1.
The first judging module 3 is configured to judge whether the number M of the collection periods of the retained electroencephalogram signal corresponding to the ith collection point satisfies a condition l.ltoreq.m.ltoreq.n, if yes, call the second collecting module 4, and if yes, call the second judging modules 5,L and M both to be positive integers.
The second acquisition module 4 is configured to acquire 2 x (L-M) electroencephalogram signals of an acquisition period corresponding to the ith acquisition point, and repeatedly execute the quality evaluation module 2 and the first judgment module 3.
The second judging module 5 is configured to judge whether i reaches (p+q), and if not, call the assigning module 6, and if yes, call the preprocessing module 7.
The assigning module 6 is configured to assign i=i+1, and repeatedly execute the quality evaluation module 2.
The preprocessing module 7 is configured to perform preprocessing operation on the electroencephalogram signals of M acquisition periods of each acquisition point of the underwater operator, specifically, perform band-pass filtering and notch filtering on the electroencephalogram signals of M acquisition periods of each acquisition point of the underwater operator, respectively subtract the average value of the electroencephalogram signals of the corresponding acquisition point when the underwater operator is in a resting state from the filtered electroencephalogram signal data of the M acquisition periods of each acquisition point, and perform artifact removal on the electroencephalogram signals of the subtracted average value of the M acquisition periods of each acquisition point to remove the eye movement artifacts and myoelectric noise in the electroencephalogram signals.
The feature extraction module 8 is used for carrying out feature extraction on the preprocessed electroencephalogram signals of each acquisition point of the underwater operator in M acquisition periods to obtain electroencephalogram signal features of each acquisition point, wherein the electroencephalogram signal features comprise electroencephalogram time domain features, electroencephalogram frequency domain features and electroencephalogram time-frequency features.
The calculating module 9 is used for calculating the similarity value of each sub-characteristic value in the electroencephalogram signal characteristic of each acquisition point of the underwater worker and the corresponding reference sub-characteristic value in the fatigue-free set.
The statistics determining module 10 is configured to determine whether the sum of the similarity values reaches a set threshold, and if yes, call the determining module 11, and if no, call the determining playing module 12.
The determination module 11 is used for determining that the underwater worker is in a fatigue-free state.
The determining and playing module 12 is configured to determine that the underwater worker is in a fatigue state, and play a stimulus audio corresponding to the fatigue state to adjust the fatigue state of the underwater worker.
Specifically, the determining and playing module 12 is configured to calculate, after determining that the underwater operator is in a fatigue state, a similarity value between each sub-feature value in the electroencephalogram signal feature of each acquisition point of the underwater operator and a corresponding reference sub-feature value in each level fatigue subset in the fatigue set, count an accumulated sum of the similarity values corresponding to each level fatigue subset, select a level fatigue corresponding to a maximum value of the accumulated sum from the accumulated sums as a fatigue level of the underwater operator, and play a stimulus audio corresponding to the fatigue level to adjust the fatigue state of the underwater operator, where the fatigue level includes a mild fatigue level, a moderate fatigue level, a severe fatigue level, and a severe fatigue level.
When the fatigue level of the underwater operator is a light fatigue level, only the stimulus audio corresponding to the light fatigue level is played; when the fatigue level of the underwater operator is a moderate fatigue level, playing a stimulus audio corresponding to the moderate fatigue level and sending out reminding information that the fatigue level reaches the moderate fatigue level and paying attention to rest; when the fatigue grade of the underwater operator is a severe fatigue grade, playing the stimulus audio corresponding to the severe fatigue grade and sending out warning information that the fatigue grade reaches the severe fatigue and has to rest; when the fatigue level of the underwater operator is a serious fatigue level, the stimulus audio corresponding to the serious fatigue level is played, the warning information that the fatigue level reaches the serious fatigue and has to be at rest is sent, and the information is sent to related personnel at the same time.
While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that these are by way of example only, and the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the principles and spirit of the invention, but such changes and modifications fall within the scope of the invention.
Claims (10)
1. An electroencephalogram signal-based fatigue adjustment method for underwater operators is characterized by comprising the following steps of:
s1, acquiring brain electrical signals of N acquisition periods corresponding to each acquisition point in P acquisition points of a human brain vision area and Q acquisition points of an auditory area of an underwater operator according to brain functional areas, wherein P, Q and N are positive integers;
s2, evaluating signal quality of the electroencephalogram signal of each acquisition period corresponding to the ith acquisition point, wherein the evaluation result is that the signal quality of the electroencephalogram signal of the acquisition period corresponding to the ith acquisition point is qualified, the electroencephalogram signal of the acquisition period corresponding to the ith acquisition point is reserved, otherwise, the electroencephalogram signal of the acquisition period corresponding to the ith acquisition point is discarded, i is a positive integer, i is not less than 1 and not more than (P+Q), and the initial value of i is 1;
s3, judging whether the number M of the acquisition periods of the reserved electroencephalogram signals corresponding to the ith acquisition point meets the condition L.ltoreq.M.ltoreq.N, if not, entering a step S4, if yes, entering a step S5, wherein L and M are positive integers;
s4, acquiring the brain electrical signals of 2 (L-M) acquisition periods corresponding to the ith acquisition point again, and repeatedly executing the steps S2-S3;
s5, judging whether i reaches (P+Q), if not, entering a step S6, and if so, entering a step S7;
s6, i=i+1, and repeatedly executing the steps S2-S3;
s7, preprocessing the electroencephalogram signals of each acquisition point of the underwater operator in M acquisition periods;
s8, carrying out feature extraction on the preprocessed electroencephalogram signals of each acquisition point of the underwater operator to obtain electroencephalogram signal features of each acquisition point, wherein the electroencephalogram signal features comprise electroencephalogram time domain features, electroencephalogram frequency domain features and electroencephalogram time-frequency features;
s9, calculating similarity values of each sub-characteristic value in the electroencephalogram signal characteristic of each acquisition point of the underwater worker and the corresponding reference sub-characteristic value in the fatigue-free set;
s10, counting whether the accumulated sum of the similar values reaches a set threshold value, if so, entering a step S11, and if not, entering a step S12;
s11, determining that the underwater staff is in a fatigue-free state;
s12, determining that the underwater worker is in a fatigue state, and playing stimulus audio corresponding to the fatigue state to adjust the fatigue state of the underwater worker.
2. The method for fatigue adjustment of an underwater worker based on an electroencephalogram signal according to claim 1, wherein in step S12, after determining that the underwater worker is in a fatigue state, calculating a similarity value of each sub-feature value in the electroencephalogram signal feature of each acquisition point of the underwater worker and a corresponding reference sub-feature value in each degree-level fatigue sub-set in a fatigue set;
and counting the accumulated sums of the similar values corresponding to the fatigue subsets of each degree, selecting the degree grade fatigue corresponding to the maximum value of the accumulated sums from the accumulated sums as the fatigue grade of the underwater operator, and playing the stimulus audio corresponding to the fatigue grade to adjust the fatigue state of the underwater operator, wherein the fatigue grade comprises a mild fatigue grade, a moderate fatigue grade, a severe fatigue grade and a severe fatigue grade.
3. The electroencephalogram signal-based fatigue adjustment method for underwater operators according to claim 2, wherein when the fatigue level of the underwater operators is a mild fatigue level, only the stimulus audio corresponding to the mild fatigue level is played, when the fatigue level of the underwater operators is a moderate fatigue level, the stimulus audio corresponding to the moderate fatigue level is played and the reminding information of paying attention to rest is sent, when the fatigue level of the underwater operators is a severe fatigue level, the stimulus audio corresponding to the severe fatigue level is played and the warning information of severe fatigue and having to rest is sent, and when the fatigue level of the underwater operators is a severe fatigue level, the stimulus audio corresponding to the severe fatigue level is played and the warning information of having to rest of severe fatigue is sent to related operators at the same time.
4. The method for fatigue adjustment of underwater operation personnel based on electroencephalogram signals according to claim 1, wherein in step S7, band-pass filtering and notch filtering are performed on the electroencephalogram signals of each acquisition point of the underwater operation personnel, the filtered electroencephalogram signal data of each acquisition point in the M acquisition periods is subtracted by the electroencephalogram signal data average value of the corresponding acquisition point when the underwater operation personnel is in a resting state, and artifact removal is performed on the electroencephalogram signals of each acquisition point in which the average value of the M acquisition periods is subtracted to remove eye movement artifacts and myoelectric noise in the electroencephalogram signals.
5. The method for fatigue adjustment of underwater worker based on electroencephalogram signals according to claim 1, wherein in step S2, signal quality evaluation is performed on the electroencephalogram signal of each acquisition cycle corresponding to the ith acquisition point by signal-to-noise ratio.
6. The underwater operator fatigue adjusting system based on the electroencephalogram signals is characterized by comprising a first acquisition module, a quality evaluation module, a first judgment module, a second acquisition module, a second judgment module, a endowing module, a preprocessing module, a feature extraction module, a calculation module, a statistics judgment module, a determination module and a determination playing module;
the first acquisition module is used for acquiring brain electrical signals of N acquisition periods corresponding to each acquisition point in P acquisition points of a human brain vision area and Q acquisition points of an auditory area of an underwater operator according to a brain function partition, wherein P, Q and N are positive integers;
the quality evaluation module is used for evaluating the signal quality of the electroencephalogram signal of each acquisition period corresponding to the ith acquisition point, if the evaluation result is that the signal quality of the electroencephalogram signal of the acquisition period corresponding to the ith acquisition point is qualified, the electroencephalogram signal of the acquisition period corresponding to the ith acquisition point is reserved, otherwise, the electroencephalogram signal of the acquisition period corresponding to the ith acquisition point is discarded, i is a positive integer, i is not more than 1 and not more than (P+Q), and the initial value of i is 1;
the first judging module is used for judging whether the number M of the acquisition periods of the reserved EEG signals corresponding to the ith acquisition point meets the condition L which is less than or equal to M and less than or equal to N, if not, the second acquisition module is called, and if yes, the second judging module is called, and both L and M are positive integers;
the second acquisition module is used for acquiring 2 x (L-M) acquisition cycle electroencephalograms corresponding to the ith acquisition point, and repeatedly executing the quality evaluation module and the first judgment module;
the second judging module is used for judging whether the i reaches (P+Q), if not, the giving module is called, and if yes, the preprocessing module is called;
the giving module is used for giving i=i+1, and repeatedly executing the quality evaluation module;
the preprocessing module is used for preprocessing the electroencephalogram signals of M acquisition periods of each acquisition point of the underwater operator;
the characteristic extraction module is used for carrying out characteristic extraction on the preprocessed electroencephalogram signals of each acquisition point of the underwater operator in M acquisition periods to obtain electroencephalogram signal characteristics of each acquisition point, wherein the electroencephalogram signal characteristics comprise electroencephalogram time domain characteristics, electroencephalogram frequency domain characteristics and electroencephalogram time-frequency characteristics;
the calculating module is used for calculating the similarity value of each sub-characteristic value in the electroencephalogram signal characteristic of each acquisition point of the underwater worker and the corresponding reference sub-characteristic value in the fatigue-free set;
the statistics judging module is used for counting whether the accumulated sum of the similar values reaches a set threshold value, if yes, the determining module is called, and if no, the determining and playing module is called;
the determining module is used for determining that the underwater staff is in a fatigue-free state;
the determining and playing module is used for determining that the underwater worker is in a fatigue state and playing the stimulus audio corresponding to the fatigue state so as to adjust the fatigue state of the underwater worker.
7. The electroencephalogram signal-based fatigue adjustment system for underwater operators according to claim 6, wherein the determining and playing module is configured to calculate, after determining that the underwater operator is in a fatigue state, similarity values of each sub-feature value in the electroencephalogram signal feature of each acquisition point of the underwater operator and corresponding reference sub-feature values in each degree-level fatigue subset in the fatigue set, count accumulated sums of the similarity values corresponding to each degree-level fatigue subset, select a degree-level fatigue corresponding to a maximum value of the accumulated sums from the accumulated sums as a fatigue level of the underwater operator, and play stimulus audio corresponding to the fatigue level to adjust the fatigue state of the underwater operator, where the fatigue level includes a mild fatigue level, a moderate fatigue level, a severe fatigue level, and a severe fatigue level.
8. The electroencephalogram signal-based fatigue regulation system for underwater operators according to claim 7, wherein the determining and playing module is used for playing only the stimulus audio corresponding to the mild fatigue level when the fatigue level of the underwater operators is the mild fatigue level, playing the stimulus audio corresponding to the moderate fatigue level when the fatigue level of the underwater operators is the moderate fatigue level and sending the reminding information that the fatigue level has reached the moderate fatigue level and the attention is resting, playing the stimulus audio corresponding to the severe fatigue level and sending the warning information that the fatigue level has reached the severe fatigue and the attention is necessary to rest when the fatigue level of the underwater operators is the severe fatigue level, and playing the stimulus audio corresponding to the severe fatigue level and sending the warning information that the fatigue level has reached the severe fatigue and the attention is necessary to rest when the fatigue level of the underwater operators is the severe fatigue level and simultaneously sending the information to the relevant operators.
9. The electroencephalogram signal-based underwater operator fatigue adjustment system according to claim 6, wherein the preprocessing module is configured to perform band-pass filtering and notch filtering on the electroencephalogram signals of M acquisition periods of each acquisition point of the underwater operator, respectively subtract an electroencephalogram signal data average value of an acquisition point corresponding to the underwater operator when the underwater operator is in a resting state from the filtered electroencephalogram signal data of the M acquisition periods of each acquisition point, and perform artifact removal on the electroencephalogram signals after the subtraction of the average values of the M acquisition periods of each acquisition point to remove eye movement artifacts and myoelectric noise in the electroencephalogram signals.
10. The electroencephalogram signal-based fatigue adjustment system for underwater operators as claimed in claim 6, wherein said quality evaluation module is configured to evaluate the signal quality of the electroencephalogram signal of each acquisition cycle corresponding to the ith acquisition point by means of signal-to-noise ratio.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310567274.8A CN116616794B (en) | 2023-05-18 | 2023-05-18 | Underwater operator fatigue adjustment method and system based on electroencephalogram signals |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310567274.8A CN116616794B (en) | 2023-05-18 | 2023-05-18 | Underwater operator fatigue adjustment method and system based on electroencephalogram signals |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116616794A true CN116616794A (en) | 2023-08-22 |
| CN116616794B CN116616794B (en) | 2024-02-20 |
Family
ID=87637576
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310567274.8A Active CN116616794B (en) | 2023-05-18 | 2023-05-18 | Underwater operator fatigue adjustment method and system based on electroencephalogram signals |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116616794B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116746927A (en) * | 2023-05-18 | 2023-09-15 | 中国人民解放军海军特色医学中心 | Method and system for adjusting states of underwater operators during underwater operation of closed cabin |
| CN117643470A (en) * | 2024-01-30 | 2024-03-05 | 武汉大学 | Fatigue driving detection method and device based on electroencephalogram interpretation |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101690659A (en) * | 2009-09-29 | 2010-04-07 | 华东理工大学 | Brain wave analysis method |
| CN103815896A (en) * | 2014-01-22 | 2014-05-28 | 东软熙康健康科技有限公司 | Mental fatigue monitoring method, device and system and mobile processing terminal |
| CN105286890A (en) * | 2015-09-22 | 2016-02-03 | 江西科技学院 | Driver sleepy state monitoring method based on electroencephalogram signal |
| CN106327802A (en) * | 2016-08-17 | 2017-01-11 | 浪潮电子信息产业股份有限公司 | Fatigue driving identification device, system and method |
| CN109394248A (en) * | 2018-12-22 | 2019-03-01 | 江西科技学院 | Driving fatigue detection method and system |
| CN111634288A (en) * | 2020-04-30 | 2020-09-08 | 长城汽车股份有限公司 | Fatigue driving monitoring method and system and intelligent recognition system |
| CN112451833A (en) * | 2020-11-10 | 2021-03-09 | 东北电力大学 | System and method for detecting and relieving driving fatigue based on personalized music recommender |
| CN114177417A (en) * | 2022-01-18 | 2022-03-15 | 深圳市威浩康医疗器械有限公司 | Anesthesia target control intelligent infusion pump controller based on electroencephalogram parameter feedback |
-
2023
- 2023-05-18 CN CN202310567274.8A patent/CN116616794B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101690659A (en) * | 2009-09-29 | 2010-04-07 | 华东理工大学 | Brain wave analysis method |
| CN103815896A (en) * | 2014-01-22 | 2014-05-28 | 东软熙康健康科技有限公司 | Mental fatigue monitoring method, device and system and mobile processing terminal |
| CN105286890A (en) * | 2015-09-22 | 2016-02-03 | 江西科技学院 | Driver sleepy state monitoring method based on electroencephalogram signal |
| CN106327802A (en) * | 2016-08-17 | 2017-01-11 | 浪潮电子信息产业股份有限公司 | Fatigue driving identification device, system and method |
| CN109394248A (en) * | 2018-12-22 | 2019-03-01 | 江西科技学院 | Driving fatigue detection method and system |
| CN111634288A (en) * | 2020-04-30 | 2020-09-08 | 长城汽车股份有限公司 | Fatigue driving monitoring method and system and intelligent recognition system |
| CN112451833A (en) * | 2020-11-10 | 2021-03-09 | 东北电力大学 | System and method for detecting and relieving driving fatigue based on personalized music recommender |
| CN114177417A (en) * | 2022-01-18 | 2022-03-15 | 深圳市威浩康医疗器械有限公司 | Anesthesia target control intelligent infusion pump controller based on electroencephalogram parameter feedback |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116746927A (en) * | 2023-05-18 | 2023-09-15 | 中国人民解放军海军特色医学中心 | Method and system for adjusting states of underwater operators during underwater operation of closed cabin |
| CN116746927B (en) * | 2023-05-18 | 2024-04-23 | 中国人民解放军海军特色医学中心 | Method and system for adjusting states of underwater operators during underwater operation of closed cabin |
| CN117643470A (en) * | 2024-01-30 | 2024-03-05 | 武汉大学 | Fatigue driving detection method and device based on electroencephalogram interpretation |
| CN117643470B (en) * | 2024-01-30 | 2024-04-26 | 武汉大学 | Fatigue driving detection method and device based on electroencephalogram interpretation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116616794B (en) | 2024-02-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116616794B (en) | Underwater operator fatigue adjustment method and system based on electroencephalogram signals | |
| Islam et al. | A wavelet-based artifact reduction from scalp EEG for epileptic seizure detection | |
| Gómez-Herrero et al. | Automatic removal of ocular artifacts in the EEG without an EOG reference channel | |
| US9044155B2 (en) | Stimulus artifact removal for neuronal recordings | |
| CN111012341B (en) | Artifact removal and electroencephalogram signal quality evaluation method based on wearable electroencephalogram equipment | |
| Jammes et al. | Automatic EOG analysis: A first step toward automatic drowsiness scoring during wake-sleep transitions | |
| US20140358025A1 (en) | System and apparatus for seizure detection from EEG signals | |
| EP3359022A1 (en) | Method and system for pre-processing of an eeg signal for cognitive load measurement | |
| US9477640B2 (en) | Neural signal processing and/or interface methods, architectures, apparatuses, and devices | |
| Parvez et al. | EEG signal classification using frequency band analysis towards epileptic seizure prediction | |
| Song et al. | A novel technique for selecting EMG-contaminated EEG channels in self-paced brain–computer Interface task onset | |
| Hung et al. | VLSI implementation for epileptic seizure prediction system based on wavelet and chaos theory | |
| JP2023018131A (en) | Medical system and method for detecting change in electrophysiological induction electric potential | |
| Turnip et al. | Adaptive principal component analysis based recursive least squares for artifact removal of EEG signals | |
| CN111067513B (en) | Sleep quality detection key brain area judgment method based on characteristic weight self-learning | |
| CN107480635B (en) | Glance signal identification method and system based on bimodal classification model fusion | |
| Uthayakumar et al. | Multifractal-wavelet based denoising in the classification of healthy and epileptic EEG signals | |
| CN108553084B (en) | Sleep staging event identification method, device and equipment | |
| David et al. | Decoupling action potential bias from cortical local field potentials | |
| Goovaerts et al. | Advanced EEG Processing for the Detection of Drowsiness in Drivers. | |
| CN110507299A (en) | Heart rate signal detection device and method | |
| CN113208622A (en) | Electroencephalogram EEG signal denoising method and system based on deep neural network technology | |
| CN113869107A (en) | Signal denoising method, signal denoising device, electronic device and storage medium | |
| CN113113109A (en) | Aerobic exercise management method and system based on wearable device sign data analysis | |
| CN116746927B (en) | Method and system for adjusting states of underwater operators during underwater operation of closed cabin |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |