KR20160081003A - Device and method for measuring cognitive fatigue based on tactile stimuli - Google Patents

Abstract

The present invention relates to a device and a method for measuring cognitive fatigue based on tactile stimuli. The method for measuring cognitive fatigue based on tactile stimuli according to an embodiment of the present invention includes a fatigue measuring step of performing a fatigue measuring task for measuring electroencephalogram and electrooculogram signals based on tactile stimuli, and a fatigue measuring and fatigue inducing step of performing a fatigue measuring task of measuring electroencephalogram and electrooculogram signals based on tactile stimuli while providing visual and auditory stimuli with the pattern of predetermined difficulty and performing a fatigue inducing task of inducing fatigue. So, a fatigue inducing factor can be evaluated.

Description

TECHNICAL FIELD [0001] The present invention relates to a tactile stimulation-based cognitive fatigue measurement apparatus and method,

The present invention relates to an apparatus and method for measuring cognitive fatigue based on a tactile stimulus, which measures changes in tactile evoked potentials and behavioral responses to evaluate user's immersion and fatigue inducing factors in real time, The present invention relates to an apparatus and method for measuring cognitive fatigue based on a tactile stimulus that can be used to identify stressors.

The complex work environment of the modern information society and the potential stress factors due to high industrialization are increasing. And, interest in health such as well-being and smart health care is increasing.

Stress is the cause of premature aging of the brain, and sustained stress can cause severe degeneration of the hippocampal neurons and can lead to temporary cognitive decline. In addition, the social environment requiring persistent stress and complex information processing may temporarily impair cognitive mechanisms associated with attention, or adversely affect autonomic homeostasis, leading to cognitive fatigue and secondary chronic diseases.

Therefore, the development of cognitive fatigue measurement and evaluation technology has increased the need to identify specific cognitive fatigue factors.

In addition, studies are being conducted to apply to various fields such as drowsiness driving and prevention of various industrial disasters through derivation of optimum GUI and information design in a specific field.

Conventionally, Patent Literature 1 is a technique for measuring cognitive fatigue based on visual stimulation. However, when using visual stimuli that are repeatedly flickered to induce event-related dislocations, the possibility of causing additional fatigue can not be excluded, and there is a problem that it takes a long time to analyze various components of the dislocation.

KR10-1285821 B

Hart, S. G. NASA-Task Load Index (NASA-TLX), 20 Years Later. Proceedings of the 50th Annual Meeting of the Human Factors and Ergonomics Society, 2006. Santa Monica: HFES. 904-908. Hart, S. G. & Staveland, L. E. 1998. Development of NASA-TLX (Task Load Index), Results of empirical and theoretical research. In: Hancock, P. A. & Meshkati, N. (eds.) Human Mental Wordload. Amsterdam: North Holland Press. Jaeggi, S. M., Buschkuehl, M., Jonides, J. & Perrig, W. J. 2008. Improving fluid intelligence with training on working memory. Proceedings of the National Academy of Sciences of the United States of America (PNAS), 105, 6829-6833 Roger W. Cholewiak, J. Christopher Brill & Anja Schwab 2004. Vibrotactile localization on the abdomen, Effects of place and space. Perception & Psychophysics, 66, 970-987

According to one embodiment of the present invention, there is provided a cognitive fatigue measurement method and apparatus capable of evaluating fatigue inducing factors through cognitive fatigue measurement based on tactile stimulation and providing appropriate feedback do.

We propose a method to measure cognitive fatigue by using an independent and simple tactile stimulus in the mechanism of visual perception mainly related to task commitment.

In addition, the present invention provides a device and method for measuring cognitive fatigue capable of shortening the conventional contrast evaluation time and real-time cognitive fatigue evaluation by evaluating only one significant component change without analyzing various components of the event evoked potential.

To provide an apparatus and method for measuring cognitive fatigue which can contribute to establishment of low-fatigue content and sensible UX architecture by deriving optimum design parameters by evaluating fatigue inducing factors in real time while maintaining the user's immersion level as much as possible do.

The tactile stimulation-based cognitive fatigue measurement method according to an embodiment of the present invention includes a fatigue measurement step in which a fatigue measurement task for measuring an EEG and a safety signal is performed based on tactile stimulation, And a fatigue measuring step and fatigue inducing step in which a fatigue measuring task for measuring an EEG signal and a safety degree signal is performed based on tactile stimulation in a state where a fatigue inducing task causing fatigue is proceeding.

The fatigue measurement task can be performed by wearing the tactile stimulus output device to the subject and measuring the subject's brain conduction and safety signal while applying a tactile stimulus of predetermined intensity at a predetermined time interval.

The tactile stimulus of the predetermined intensity may be from 25 dB to 66 dB.

The fatigue-inducing task may be a task that provides visual stimulation or auditory stimulation, or both, at a predetermined time interval, m times per cycle, and determines whether the mn th stimulus (n is a natural number of 1 or more) ≪ / RTI >

The degree of difficulty in the fatigue-inducing task may be set by at least one of a time interval, m value, and n value at which the visual stimulation and auditory stimulation are applied.

The fatigue measurement and fatigue induction step may include a first fatigue measurement and fatigue induction step in which a fatigue measurement task is performed while a first fatigue-inducing task is progressing at a first degree of difficulty and a second fatigue measurement and fatigue induction step, And a second fatigue measurement and fatigue inducing step in which a fatigue measurement task is performed as the fatigue-inducing task proceeds.

And a fatigue-inducing task training step in which a fatigue-inducing task for causing fatigue is previously performed by providing visual and auditory stimulation in a pattern of predetermined difficulty levels before the fatigue measurement step.

Attaching an electroencephalogram and a safety sensor before the fatigue measurement step and selectively paying attention to a second focus which changes in a time between 1 degree and 10 degrees with the eyes fixed with respect to the first focus of the display unit Safety may also include a training phase.

According to another aspect of the present invention, there is provided an apparatus for measuring tactile stimulation based on cognitive fatigue comprising: a visual and auditory stimulation output unit for outputting a visual stimulus and an auditory stimulus in a predetermined difficulty pattern; a tactile stimulation output unit for outputting a tactile stimulus; And an EEG and EOG measuring unit for measuring a degree of safety, an input unit for inputting a response of the subject to the visual stimulation and auditory stimulation, a fatigue measuring task for measuring an EEG and a safety signal based on tactile stimulation, A controller for controlling a fatigue measurement and a fatigue-inducing task for measuring an EEG signal and a safety signal based on a tactile stimulus in a fatigue-induced state in which fatigue is induced by providing a predetermined pattern of difficulty; The data subject to which the perceived fatigue is measured according to the results of the first and second fatigue-induced tasks measured by the EEG measuring unit It includes parts and analyzed.

The apparatus may further include a support for supporting a subject's head to minimize noise generated by movement of the subject with respect to the tactile stimulus.

The haptic stimulation output may be in the form of a belt comprising at least one vibrating haptic sensor at the waist of the subject.

The data processing and analysis unit amplifies the EEG signal data measured by the EOG and EEG measuring unit by applying a band pass filter of 0.1 to 35 Hz and applies a band pass filter of 0.05 to 35 Hz to the safety signal Can be amplified and recorded.

The data processing and analyzing unit separates signals of 1000 ms after stimulus presentation at -200 ms from the point of presentation of the stimulus, and then detects eye flicker exceeding ± 80 μV and horizontal eye movement exceeding ± 30 μV EEG can be eliminated.

The data processing and analyzing unit may correct the baseline by subtracting the P300 amplitude of the separated and removed intervals from the presentation of the stimulus at the interval of -200 ms before the presentation of the stimulus, from the potential value of each interval.

The data processing and analyzing unit may calculate the P300 average potential value of each section after calibrating the baseline or set the P300 amplitude to a point having the highest amplitude in the static potential section of 400 to 600 ms generated after tactile stimulus presentation .

According to one embodiment of the present invention, a cognitive fatigue measurement method and apparatus capable of evaluating fatigue inducing factors through cognitive fatigue measurement based on tactile stimulation and providing appropriate feedback can be provided.

In addition, it is possible to provide a method of measuring cognitive fatigue by using an independent and simple tactile stimulus to the mechanism of the poison, which is mainly related to task commitment.

Also, it is possible to provide a device and method for measuring the cognitive fatigue capable of shortening the conventional contrast evaluation time and real-time cognitive fatigue evaluation by evaluating only one significant component change without analyzing various components of the event-induced potential.

Also, it is possible to provide emotional user feedback by evaluating the fatigue inducing factors in real time while deriving ergonomic design parameters while maintaining the user's immersion level as much as possible, and to be able to contribute to establishment of low-fatigue content and emotional UX architecture An apparatus and method for measuring fatigue can be provided.

1 is a schematic view schematically showing an apparatus for measuring cognitive fatigue based on a tactile stimulus according to an embodiment of the present invention.
2 is a block diagram schematically illustrating the configuration of a tactile stimulation-based cognitive fatigue measuring apparatus according to an embodiment of the present invention.
FIGS. 3A and 3B are flowcharts schematically illustrating a tactile stimulation-based cognitive fatigue measurement method according to various embodiments of the present invention.
FIG. 4 is a schematic view illustrating positions of electrodes for measuring electroencephalogram in a tactile stimulation-based cognitive fatigue measuring apparatus according to an embodiment of the present invention.
5A and 5B are diagrams illustrating sequences of first and second fatigue measurements and fatigue-inducing tasks in accordance with one embodiment of the present invention.
FIG. 6 is a flowchart illustrating a tactile stimulation-based cognitive fatigue measurement method according to an embodiment of the present invention.
FIG. 7 is a graph showing reaction accuracy and reaction time according to the conditions of the fatigue-induced task measured according to the embodiment of FIG.
FIG. 8 is a graph comparing P300 average potential values at AFz, Fz, Cz, and Pz points of EEG measurement points measured in the reference state, primary fatigue induced state, and secondary fatigue induced state according to the embodiment of FIG. 6 .
FIG. 9 is a graph showing an average potential value between the respective conditions of the induced EEG waveform at the Cz point and the Pz point in the EEG measurement points of the reference state, the primary fatigue inducing state and the secondary fatigue induced state measured according to the embodiment of FIG. 6 And comparing the differences.
10 is a graph comparing subjective fatigue evaluation results of primary fatigue inducing conditions and secondary fatigue inducing conditions according to the embodiment of FIG.

Hereinafter, a tactile stimulation-based cognitive fatigue measuring apparatus and method according to an embodiment of the present invention will be described in detail with reference to FIGS. 1 to 10. FIG.

FIG. 1 is a schematic view showing a tactile stimulation-based cognitive fatigue measuring apparatus according to an embodiment of the present invention. FIG. 2 schematically shows the configuration of a tactile stimulation-based cognitive fatigue measuring apparatus according to an embodiment of the present invention. Fig.

1 and 2, a tactile stimulation based cognitive fatigue device according to an embodiment of the present invention includes a visual and auditory stimulation output unit 30, a tactile stimulation output unit 60, a safety degree (EOG) A control unit 70 for controlling outputs of the visual and auditory stimulation output unit 30 and the tactile stimulation output unit 60, an input unit 40 for inputting stimulation data by the subject, And a data processing and analyzing unit 80 for collecting, processing and analyzing EOG and EEG data. The data processing and analysis unit 80 may be connected to the input unit 40 and the control unit 70 as shown in FIG. 2, and may analyze the collected data to evaluate the perceived fatigue.

The data processing and analyzing unit 80 may be configured as a data collecting apparatus and an information processing apparatus such as a computer for input, control, and data analysis. However, the present invention is not limited thereto, Of course.

The visual and auditory stimulation output unit 30 is for outputting visual stimulation and auditory stimulation in a pattern of predetermined difficulty level, and the tactile stimulation output unit 60 is for outputting tactile stimulation.

The visual and auditory stimulation output unit 30 may be connected to the controller 70 to output a video or sound of a predetermined pattern. For example, a display device such as an LCD monitor or a speaker may be used.

The tactile stimulation output unit 60 may be configured as a tactile stimulus outputting device and a belt type including at least one vibrating haptic sensor at the waist of the subject 10. [ However, the present invention is not limited thereto, and may be attached to various parts of the body of the subject 10, and various devices capable of providing various tactile stimuli according to control signals supplied from the control unit 70, other than the haptic sensor, .

The input unit 40 may be, for example, a keyboard, an input device through which a user can input his or her own cognitive information. Accordingly, the subject's reaction to the visual stimulation and the auditory stimulation can be input to the input unit 40.

The control unit 70 is connected to the visual and auditory stimulation output unit 30 and the tactile stimulation output unit 60 and includes a reference state fatigue measurement task for measuring an EEG and a safety degree signal based on tactile stimulation, It can be configured to control the fatigue measurement and the fatigue inducing task to measure the EEG and safety signal based on tactile stimulation in a fatigue induced state in which auditory stimulation is provided in a pattern of predetermined difficulty level.

The data processing and analysis unit 80 may be connected to the control unit 70, the input unit 40, and the EOG and EEG measurement unit 20. The data processing and analysis unit 80 may measure the perceived fatigue according to the fatigue-induced task results and the first and second fatigue-induced task results measured by the EOG and EEG measuring units.

The data processing and analyzing unit 80 applies the EEG and EEG measuring unit EEG signal data to the data processing and analyzing unit 80 by applying a band pass filter of 0.1 to 35 Hz to measure the EEG and the safety signal And the safety signal can be amplified and recorded by applying a band pass filter of 0.05 to 35 Hz.

In addition, the data processing and analysis unit can compare the peak amplitude at the 400 to 600 ms static potential interval generated after tactile stimulus presentation by setting the amplitude to P300 amplitude.

In this specification, P300 denotes an event-related potential (ERP) indicating a positive potential after about 300 ms after stimulation.

The data processing and analyzing unit 80 separates the signal of 1000 ms after the stimulus presentation at -200 ms from the stimulus presentation point and minimizes the noise by ± 80 μV Of eye blinking and eye movements in the horizontal direction exceeding ± 30 μV.

Also, the data processing and analysis unit 80 can correct the divided and removed intervals by subtracting the average potential value of the time interval of -200 ms from the potential value of each interval from the stimulus presentation time point.

Then, after calibration, the P300 average potential value of each section can be calculated and compared.

Alternatively, after the baseline is corrected to the value of the time interval of -200 ms from the stimulus presentation time point, the point having the highest amplitude in the static potential interval of 400 to 600 ms generated by the tactile stimulus can be set to the amplitude of P300.

The apparatus may further include a support unit 50 for supporting the head of the subject 10 to minimize noise generated by movement of the subject to the tactile stimulation.

FIG. 3A is a flowchart schematically illustrating a tactile stimulation-based cognitive fatigue measurement method according to an embodiment of the present invention. Hereinafter, a specific tactile stimulation-based cognitive fatigue measurement method of the present invention will be described in detail with reference to FIGS. 3A and 3B.

Referring to FIG. 3A, a tactile stimulation-based cognitive fatigue measurement method according to an embodiment of the present invention includes a fatigue measurement step S30 in which a fatigue measurement task for measuring an EEG signal and a safety signal is performed based on tactile stimulation, And fatigue causing fatigue by providing visual and auditory stimuli in a pattern of predetermined difficulty. Fatigue measurement that measures the EEG and safety signal based on tactile stimulation in a state where the task is progressing. And steps S40 and S50.

That is, the cognitive fatigue can be measured by the first step in which only the fatigue measurement task is performed and the second step in which the fatigue measurement task and the fatigue-inducing task are simultaneously performed. In the first step, the fatigue measurement task is performed in the reference state in which there is no additional fatigue triggering task, and the fatigue measurement is performed in the fatigue inducing condition by measuring the perceived fatigue of the user and performing the fatigue measurement task simultaneously with the fatigue- To determine the change in perceived fatigue under fatigue-induced conditions.

In the fatigue measurement step (S30) and the fatigue measurement and fatigue induction step (S40, S50), the fatigue measurement task is performed by putting the tactile stimulus output device on the subject and applying a predetermined intensity tactile stimulus at a predetermined time interval, And by measuring the electroencephalogram and safety signal of the subject.

FIG. 4 is a schematic view illustrating positions of electrodes for measuring electroencephalogram in a tactile stimulation-based cognitive fatigue measuring apparatus according to an embodiment of the present invention.

Referring to FIG. 4, a brain electrode may be attached to a subject to measure an event-induced potential by tactile stimulation.

The EEG electrode part of the cognitive fatigue measuring device using tactile stimulation can be attached to the subject's scalp according to the international 10-20 electrode system.

Specifically, the EEG recording electrodes can be attached to the four points AFz, Fz, Cz, and Pz. A1 point can be designated as a ground electrode and A2 point can be designated as a reference electrode.

In addition, the VEOG can be attached 2.5 cm below the left eye pupil, attach the HEOG to a horizontal position 2 cm away from the tip of both eyes, and attach a safety ground electrode 2 cm away from the vertical position.

On the other hand, the size of the tactile stimulus can be set to be 25 dB to 66 dB so that all subjects can clearly recognize the tactile stimulus.

Specifically, the signal is divided into frequency and amplitude, and the amplitude of the signal at a given frequency is mainly defined by its acceleration. The acceleration can be expressed by the following equation with respect to the position x.

[Formula 1]

For sinusoidal signals, x = A sin (ωt) and a = - Aω2 sin (ωt).

Where A is the maximum amplitude of the signal and? Is the frequency.

For example, when the amplitude is A = 1 占 퐉 and the frequency is ω = 160 Hz, a _max = 1 × 10 -6 × 1602 = 0.0256 m / s 2.

In general, the amplitude of vibration is expressed in decibels relative to the reference value. Then, if G1 and G2 are two acceleration amplitudes, their decibel ratios are:

[Formula 2]

It is known by Cholewiak, Brill & Schwab (2004) that for an acceleration of A = 1 탆 and a frequency of ω = 160 Hz, the mean vibrotactile detection threshold is 25 dB. (I.e., the G2 value is 0.0256 m / s2).

In the experiment, the intensity of the tactile sensor can be set to a level corresponding to the acceleration G1 = 49.95 m / s < 2 >.

The decibel value is expressed as follows.

[Formula 3]

This value is at least twice the threshold value. Therefore, the vibration of 65.8 dB is perceived by all the subjects due to the conspicuousness. As a result, the subject may have a predetermined intensity of the tactile stimulus of 25 dB to 66 dB.

As an example, a tactile stimulus can be applied at a random interval of between 12 and 60 seconds, with a vibration amplitude of 128 ms, and a real vibration range at the abdomen of a radius of 6 mm. 20 vibrations presented at random intervals constitute one cycle and can be applied in total of 80 times (4 cycles).

Subjects were exposed to the same stimulation time at a maximum random interval of 60 seconds x

20 (frequency per block) = 20 minutes can be set as the execution time per cycle. The time difference between the subjects due to the random interval can be processed as a rest and can be performed in such a manner that the next block is automatically started after 20 minutes without providing any additional information to the subject.

The electrophysiological data measured in the first step of the fatigue measurement task alone or in the second step of the fatigue measurement task and the fatigue inducing task together are amplified 50,000 times by applying a digital bandpass filter of 0.1 to 35 Hz . Likewise, the safety data can also be amplified 500 times by applying a band-pass filter of 0.05 to 35 Hz.

In the case of P300, it can be determined that the amplitude is the highest point in the static potential interval (400 to 600 ms) generated after stimulus presentation.

For each experimental condition, the evoked potential corresponding to each tactile stimulus (80 per condition) can be detected.

The target onset time recorded at each task stage (reference, fatigue-induced task) and the signal of 1000 ms after stimulus presentation at -200 ms from the stimulus presentation point were separated and then ± 80 μV Excessive eye blinking and horizontal eye movements in excess of ± 30 μV can eliminate the EEG. This is to separate only brain waves caused by tactile stimulation.

The separation intervals of each EEG measurement point (AFz, Fz, Cz, Pz) including the blinking of the eye and the left and right eye movements not exceeding ± 80 μV and ± 30 μV were analyzed by principal component analysis to determine the event- Can be detected. Meanwhile, in the method for measuring perceived fatigue according to an embodiment of the present invention, the fatigue-inducing task in each of the fatigue measurement and fatigue inducing steps (S40 and S50) presents a tactile stimulus at a predetermined interval, n-back) is performed to measure the change in the tactile evoked potential to measure the real-time cognitive fatigue.

 Specifically, the fatigue-inducing task provides visual stimulation or auditory stimulation, or both, at a predetermined time interval, m times per cycle, and determines whether the mn th stimulus (where n is a natural number of 1 or more) ≪ / RTI >

(Or blocks) at a predetermined time interval, and to input a predetermined key by confirming whether or not the provided stimulus coincides with the stimulus generated after the nth stimulus .

More specifically, Figures 5A and 5B show a sequence of first and second fatigue measurements and fatigue-inducing tasks in accordance with one embodiment of the present invention.

The fatigue measurement and fatigue induction step may include a first fatigue measurement and fatigue induction step in which a fatigue measurement task is performed while a first fatigue-inducing task is progressing at a first degree of difficulty and a second fatigue measurement and fatigue induction step, And a second fatigue measurement and fatigue inducing step in which a fatigue measurement task is performed as the fatigue-inducing task proceeds.

However, the present invention is not limited to this, and it is needless to say that the fatigue measurement and the fatigue inducing step can be variously set to one or more times according to the degree of difficulty.

According to one embodiment of the present invention, visual stimulation and auditory stimulation can occur simultaneously at predetermined intervals (e.g., every 3 seconds). For example, the screen or sound reproduction time is 0.5 seconds, and the interval between the next screen or sound reproduction time is 2.5 seconds, so that visual stimulation and auditory stimulation can be given at intervals of 3 seconds in total.

Then, in the continuously changing visual stimulation sequence, the subject memorizes the currently presented stimulus (the position of the square) and the stimulus (the position of the square) presented before the nth, and determines whether or not they match or do not match Can be confirmed by inputting.

Likewise, in a continuously changing auditory stimulation sequence (e. G., Eight letters presented sequentially), remember the currently presented auditory stimulus (alphabet) and the stimulus (alphabet) presented before the nth, It is possible to confirm whether or not they match by inputting a predetermined button.

In the case of FIG. 5A, n is 1, and when the time or auditory stimulus corresponds to the time or auditory stimulus of the previous time, the 'S' key is input in the case of auditory stimulation and the 'A' As shown in FIG. The stimulation duration per stimulus is 0.5 seconds and the interval between stimulations is 2.5 seconds. The total stimulus duration per stimulus is 3 seconds, and 21 stimuli per one cycle (block) can be given (m = 21).

In the case of FIG. 5B, n is 2, and when the time or auditory stimulus corresponds to the time or auditory stimulus two times before, the 'S' key is input for auditory stimulation and the 'A' As shown in FIG. The duration of the stimulation per one cycle is 0.5 seconds and the interval between the stimulation is 2.5 seconds. The total time required for stimulation per one cycle is 3 seconds, and 22 stimuli (m = 22) per cycle (block) can be given.

On the other hand, the degree of difficulty in the fatigue-induced task may be set by at least one of a time interval at which the visual stimulation and auditory stimulation are applied, a value m (the total number of stimulation per cycle), and an n value (storage interval).

That is, in the case of the embodiment of FIG. 5B, it is possible to set the storage interval (n value) to be larger and to set the difficulty more difficult than in the embodiment of FIG. 5B. As a result, it is possible to set a higher cognitive fatigue.

FIG. 3B is a flowchart illustrating a tactile stimulation-based cognitive fatigue measurement method according to another embodiment of the present invention.

According to another embodiment of the present invention, there is provided a fatigue-inducing task training step (S10) in which a fatigue-inducing task for generating fatigue is provided in advance by providing visual and auditory stimulation in a predetermined pattern of difficulty levels before the fatigue measurement step .

In the fatigue-induced task training step S10, more accurate results can be obtained by adapting to the dual n-back task.

In this case, the value of n (memory interval) may vary depending on the measurement result of the subject. For example, if the subject has less than three mistakes in both modalities (visual and auditory stimuli), the n value is increased by one, but if the visual and auditory stimuli have more than five mistakes, the n value is reduced by one can do. Otherwise, the value of n may be set to be maintained.

In the training phase, 20 + n cycles can be performed according to the value n set per cycle. If the time required for one cycle is 3 seconds, it can take about 60 seconds for each cycle. And repeats the training until the n (memory interval) = 2 is stably maintained until the preset accuracy is maintained. (For example, it can be repeated until the minimum accuracy reaches 75% or more at n = 2.)

Further, in order to attach the EEG and the safety sensor after the fatigue-induced task training step (S10) and before the fatigue measurement step (S30), and to reduce the noise due to the safety through the pre-training, (EOG) training step (S20) that selectively gives attention to a second focus that changes in a time between 1 degree and 10 degrees with the eyes fixed with respect to one focus.

An EOG training step S20 may be performed to minimize the effect of electrooculogram (EOG) on the electroencephalogram (EEG) noise. This is done to minimize left and right eye movements, rapid eye movements and blinking of eyes in the fatigue measurement phase.

In the EOG training phase (S20), it can be measured by selectively paying attention to the second cross (white cross) which changes by 1 degree on the visual axis with the eye fixed to the cross of the monitor (red cross). (For example, according to the left and right random sequence, the second cross can be moved in 10 steps by 1 degree.) It starts moving from the center to the left or right and reaches the left (right) end of the monitor Can be moved back to the origin.

By performing the fatigue measurement step S30 and the first and second fatigue measurement and fatigue inducing steps S40 and S50 after passing through the fatigue-induced task training step S10 and the EOG training step S20, Can be obtained.

According to an embodiment of the present invention, fatigue inducing factors can be evaluated and feedback can be provided through a tactile stimulation based cognitive fatigue measuring apparatus and method. In addition, cognitive fatigue can be measured by using an independent and simple tactile stimulus to the mechanism of the poison, which is mainly related to task commitment.

In addition, it is possible to evaluate the change of only one significant component without analyzing various components of the event evoked potential, and it is possible to shorten the conventional contrast evaluation time and real-time cognitive fatigue evaluation becomes possible.

In addition, by estimating the fatigue inducing factor in real time while deriving the user's immersion degree to the maximum, deriving the optimum design parameter can contribute to establishment of low-fatigue content and sensitive UX architecture.

FIG. 6 is a flowchart illustrating a tactile stimulation-based cognitive fatigue measurement method according to an embodiment 1 of the present invention.

 [Example 1]

The cognitive fatigue based on the tactile stimulus was measured according to the flowchart shown in FIG. For the pre-training, the fatigue-induced task training step (S110) was performed first, the sensor was attached, the EOG training step (S120) was performed, and the fatigue measurement step (S130) The first and second fatigue measurement and fatigue inducing steps (S150 and S170) in which the fatigue measurement task and the fatigue inducing task are simultaneously performed in the fatigue induced state of the two difficulty levels are performed.

(S140), a first fatigue measurement and fatigue induction step (S150), a second fatigue measurement and fatigue induction step (S170), and a second fatigue measurement step (S170) Next, the second and third subject questionnaire evaluations (S160, S180) were further performed, respectively.

Fatigue-inducing task training step (S110)

In the fatigue-inducing task training step (S110) for the pre-training, the subject is allowed to react to the continuous visual stimulation and auditory stimulation.

Visual and auditory stimuli appeared simultaneously at 3 second intervals. (Duration of 0.5 seconds, interval of 2.5 seconds). If the current stimulus and the stimulus (square position) presented before the nth are memorized on the continuously changing visual stimulation sequence, enter 'A' If they do not match, the task is ignored.

In the eight alphabet auditory stimulation sequences presented sequentially, if the current stimulus (alphabet) and the stimulus (alphabet) presented before the nth memory are stored and matched, the 'S' key is input to the left hand detection.

The value of n was measured differently according to the performance of the subjects for each block. In the auditory stimulus and the visual stimulus, the n value increased by 1 when the real number was less than 3, and when the real number was 5 or more in the auditory stimulus and the visual stimulus, Was decreased by 1, and in the other cases, the value of n was maintained.

The training was 20 + n auditory stimulation and visual stimulation according to the set n values per block, and each block thus took about 60 seconds.

In addition, pre-training was repeated until n = 2 was stably maintained. That is, for example, pre-training was repeated until n = 2 showed an accuracy of 75% or more.

EOG training step (S120)

The EEG electrode part of the cognitive fatigue measuring device using the tactile stimulus was attached to the subject's scalp according to the international 10-20 electrode system (international 10-20 electrode system).

EEG electrodes are attached to four points AFz, Fz, Cz, and Pz (see FIG. 4), and A1 point is designated as a ground electrode and A2 point is designated as a reference electrode.

Vertical EOG (VEOG) electrode and horizontal EOG (HEOG) electrodes were attached to each other at a distance of about 2.5 cm below the left eye pupil and 2 cm horizontally to the tip of both eyes to attach the EOG electrode. Then, a safety grounding electrode was attached at a distance of 2 cm from the vertical.

And, EOG training was performed to minimize the effect of the safety (EOG) on the noise of the EEG. With the eye fixed to the red cross in the monitor, with a selective attention to the white cross that changes by 1 degree on the visual axis, the white cross is moved from the center to the left or to the right by 1 degree in the left and right random sequence We started to move, and we trained at the end of the left (or right) monitor to reach 10 degrees and return to the origin.

In the reference state fatigue measurement step (S130)

Fatigue measurements were performed in a reference state without additional fatigue-induced tasks. For the fatigue measurement task, a belt with two vibrating haptic sensors was worn on the waist (for example, left and right).

A tactile stimulus of 66 dB was applied, and a fixed bib was worn to minimize the noise generated by body movements.

Subjects were exposed to the tactile stimulus with their eyes fixed on the red cross that appeared on the front of the monitor.

The tactile stimulation was performed a plurality of times at intervals of at least one of a vibration time of 128 ms and a stimulation interval of 12 to 60 seconds, that is, at random intervals, and the actual vibration range was performed with a radius of 6 mm. Twenty vibrations constituted one cycle at the upper stimulation interval, and a total of 80 cycles (4 cycles) were presented.

In order to expose the subjects to the same stimulation time, the execution time per block was set at a maximum random interval of 60 seconds × 20 times = 20 minutes. The time difference between the subjects due to the random interval was treated as a dormant state, After 20 minutes, the next block was started automatically.

First fatigue induction and fatigue measurement step (S150) (primary fatigue induction)

While performing the same fatigue measurement task as in the fatigue measurement step (S130), the subject wore a belt including two vibrating haptic sensors at the waist for fatigue measurement task.

As shown in FIG. 5A, the currently presented stimulus (position of a square) and the previous stimulus (position of a square) are stored in the continuously changing visual stimulation sequence (eight square positions on the monitor) 'Key and ignore it if they do not match.

The stimulus (alphabet) presently presented and the stimulus (alphabet) preceding the stimulus are stored in the audible stimulation sequence of the eight alphabets C, F, H, M, P, Q, If there is a match, enter the 'S' key, otherwise ignore.

The first fatigue induction and fatigue measurement tasks were performed 80 times in total for 21 times per block to induce cognitive fatigue, and each block took 60 seconds. Then, the response time to the stimulation and the tactile evoked potential from the subject were detected.

Second fatigue induction and fatigue measurement task (S170) (secondary fatigue induction)

In performing the same fatigue measurement task as in the fatigue measurement step (S130), a belt including two (left and right) vibrating haptic sensors was worn on the waist for fatigue induction task.

As shown in FIG. 5B, the currently presented stimulus (the position of the square) and the stimulus (the position of the square) two times before are stored in the continuously changing visual stimulation sequence (eight square positions on the monitor) 'Key and ignore it if they do not match.

The stimuli (alphabet) presently presented and the stimuli (alphabet) two times before are presented in the audible stimulation sequence of the eight alphabets C, F, H, M, P, If there is a match, enter the 'S' key, otherwise ignore.

The second fatigue measurement and fatigue inducing task was performed for 80 times for 22 times per block for inducing cognitive fatigue, and each block took about 60 seconds. Then, reaction time and evoked potential corresponding to stimulation were detected from the subject.

The first, second and third subject questionnaire evaluations (S140, S160, S180)

The subject's subjective evaluation was performed immediately after each task was completed using the NASA-TLX questionnaire which can reflect cognitive fatigue.

In NASA-TLX questionnaires, 6 items (mental requirement, physical requirement, time requirement, task achievement, effort, embarrassment / frustration) are explained and subjective evaluation is made.

In order to calculate the weight of six questionnaire items, 15 cards were created using the number of cases, and the items that were considered to have had a greater effect among the two items were selected to perform the task.

The final NTSA-TLX subjective rating was calculated by applying a weight to the NASA-TLX questionnaire after evaluating the score between 0 and 100 points.

Data processing and analysis

The EEG data was amplified 50,000 times after applying a digital bandpass filter of 0.1 to 35 Hz. EOG data is also 0.05 to 35 Hz band-pass filter. The P300 component was determined to be the highest amplitude in the static potential range (400 to 600 ms) after stimulus presentation.

For each experimental condition, the evoked potential corresponding to each tactile stimulus was detected. After separating EEG signals of 200 ms and +1000 ms intervals from the target onset time and the presentation time recorded in three states of reference state, primary fatigue induction and secondary fatigue induction (Epoch Separation EEG lesions contaminated by eye blinking in excess of ± 80 μV and horizontal eye movements in excess of ± 30 μV were removed.

Detection intervals of each EEG measurement point (AFz, Fz, Cz, Pz) including eye blinking and left and right eye movements not exceeding ± 80 μV and ± 30 μV are detected by principal component analysis to detect the event- Respectively.

Provision of stimulus - After removing the contaminated intervals by subtracting the average potential between the presentation time and the presentation point by the total potential value, correct the P300 amplitude to the baseline and calculate the average potential value between the intervals, Were calculated.

Data were analyzed by using AcqKnowledge 4.2 (BIOPAC Systems Inc., Goleta, CA, USA) and Visual Studio 2008 with Microsoft Foundation Class (MFC) software for data processing and analysis. Statistical data analysis was performed using IBM SPSS Statistics 17.0 , Illinois, USA). At the same time, we used a periapical Peroni calibration method to reduce the increased probability of one type error that can be generated by statistical multiple comparison of measured signals.

FIG. 7 is a graph showing reaction accuracy and reaction time according to the conditions of the fatigue-induced task measured according to the embodiment of FIG.

Referring to FIG. 7, it can be seen that the second-order fatigue-inducing condition with a high degree of difficulty is significantly lower than the first-order fatigue inducing condition. Also, it was found that the reaction time increased significantly compared to the first fatigue inducing condition under the second fatigue inducing condition.

The decrease in reaction accuracy and the increase in reaction time reflects the decrease in the response speed and accuracy of the behavioral mechanism due to the cognitive fatigue induced in the second fatigue inducing condition.

FIG. 8 is a graph comparing P300 average potential values at AFz, Fz, Cz, and Pz points of EEG measurement points measured in the reference state, primary fatigue induced state, and secondary fatigue induced state measured according to the embodiment of FIG. 6 . FIG. 9 is a graph obtained by visualizing the graph of FIG. 8 as an EEG waveform. FIG. 9 is a graph showing the Cz point and the Cz point of the EEG measurement points measured in the reference state, the primary fatigue inducing state and the secondary fatigue induced state measured according to the embodiment of FIG. A graph showing the tactile evoked potential waveform at the Pz point, the reference state, the primary fatigue inducing state, and the difference of the average potential values between the primary fatigue inducing state and the secondary fatigue inducing state.

Fig. 8 shows the results of cognitive fatigue evaluation using a tactile stimulus.

After setting the reference condition, primary fatigue induction and secondary fatigue induction conditions as factors in the subjects, we performed the ANOVA analysis. The degree of freedom was corrected through Greenhouse-Geisser because it did not meet the hypothesis.

There was no significant difference at AFz, Fz, and Cz points, but post-test was performed because there was a significant difference in P300 amplitude between the three conditions at Pz point. (Specifically, Tukey's HSD post hoc test was conducted.)

At the Pz point, there was no significant difference under the T-only and primary fatigue inducing conditions, but the P300 amplitude was significantly decreased in the secondary fatigue inducing conditions when compared with the reference fatigue and the primary fatigue inducing conditions .

A statistically significant decrease in the amplitude of P300, a late event-induced potential component, implies that the selective attention mechanism is temporarily impaired and the concentration is reduced. This means that the degree of activation of the neural cell and the specific neural network involved in perception of a particular target stimulus and for cognitive information processing has been temporarily reduced. In other words, it is a triggering EEG indicator only when cognitive fatigue is induced.

FIG. 10 is a graph comparing subjective fatigue results with reference conditions, primary fatigue inducing, and secondary fatigue inducing conditions according to the embodiment of FIG.

After setting the baseline condition, primary fatigue induction, and secondary fatigue induction as factors within the subject, analysis was performed through one - way repeated measures analysis.

Also, post - test was performed because the scores of subjects' questionnaires were significantly different in each of the three conditions.

As shown in FIG. 10, the subject's subjective evaluation result and the P300 amplitude showed a significant pattern in the post test.

There was no significant difference between baseline and primary fatigue inducing conditions. This suggests that fatigue does not occur in the primary fatigue inducing conditions of easy difficulty as in the reference condition in which no triggering task is performed.

However, in the case of the second fatigue inducing condition, the score of the questionnaire survey was significantly increased. The significant increase in the score of the NASA TLX questionnaire, which can reliably and subjectively evaluate the cognitive load on the cognitive task, suggests that the task difficulty set to induce fatigue was appropriate, and the subjective cognitive This means that the rod can be measured objectively by measuring the evoked potential.

10: Subjects
20: EOG & EEG measuring unit
30: visual & auditory stimulation output unit
40:
50: Support
60: tactile stimulation output unit
70:
80: Data processing and analysis unit

Claims (15)

A fatigue measurement step in which a fatigue measurement task for measuring an EEG and a safety level signal is performed based on tactile stimulation; And
Fatigue causing fatigue by providing visual and auditory stimuli in a pattern of predetermined difficulty, fatigue measuring the EEG and safety signal based on tactile stimulation in a state where the task progresses, fatigue measurement and fatigue induction step Wherein the tactile stimulation-based cognitive fatigue measurement method comprises the steps of:
The method according to claim 1,
The fatigue measurement task may include:
A tactile stimulation output device is worn on the subject,
While applying a tactile stimulus of predetermined intensity at a predetermined time interval,
Wherein the tactile stimulation-based cognitive fatigue measuring method is performed by measuring the electroencephalogram and the safety signal of the subject.
3. The method of claim 2,
And the tactile stimulus of the predetermined intensity is 25 dB to 66 dB.
The method according to claim 1,
The fatigue-
Visual stimulation or auditory stimulation, or both at a predetermined time interval m times per cycle,
Characterized in that it is carried out by checking whether or not the mn th stimulus (n is a natural number of 1 or more) and the m th stimulus coincide with each other in the provided stimulus.
5. The method of claim 4,
The degree of difficulty in the fatigue-
Wherein the at least one of the visual stimulation and the auditory stimulation is set by at least one of a time interval, an m value, and an n value.
The method according to claim 1,
The fatigue measurement and fatigue induction step may include:
A first fatigue measurement and fatigue inducing step in which a fatigue measurement task is performed while a first fatigue inducing task is progressed at a first degree of difficulty,
And a second fatigue measurement and a fatigue induction step in which a fatigue measurement task is performed while a second fatigue-inducing task of a second difficulty different from the first difficulty is being performed.
The method according to claim 1,
Before the fatigue measurement step,
Further comprising a fatigue-inducing task training step in which a fatigue-inducing task for causing fatigue is previously performed by providing visual and auditory stimuli in a predetermined difficulty pattern.
The method of claim 1, wherein
Before the fatigue measurement step, an electroencephalogram and a safety level sensor are attached,
And a safety degree training step of selectively paying attention to a second focus which changes with a time between 1 degree and 10 degrees with the eyes fixed to the first focus of the display part. How to measure.
A tactile stimulation output unit for outputting a visual and auditory stimulation output unit and a tactile stimulation output unit for outputting visual stimulation and auditory stimulation in a predetermined difficulty pattern;
An EEG and EOG measurement unit for measuring the electroencephalogram and the safety;
An input unit for inputting a response of the subject to the visual stimulation and auditory stimulation;
A fatigue measurement task for measuring an EEG and a safety signal based on a tactile stimulus and a task for visual and auditory stimuli in a predetermined difficulty pattern to generate an EEG signal and a safety signal based on a tactile stimulus in a fatigue- A control unit for controlling fatigue measurement and fatigue-inducing tasks to be measured; And
And a data processing and analyzing unit for measuring perceived fatigue according to the fatigue-induced task result and the first and second fatigue-induced task results measured by the EOG and EEG measuring unit. .
10. The method of claim 9,
Further comprising a support for supporting a subject's head to minimize noise generated by movement of the subject with respect to the tactile stimulus.
10. The method of claim 9,
Wherein the tactile stimulation output unit is configured in the form of a belt including at least one vibrating haptic sensor at the waist of the subject.
10. The method of claim 9,
Wherein the data processing and analysis unit comprises:
The EEG and EEG measurement unit amplifies the electrocardiogram signal data by applying a band pass filter of 0.1 to 35 Hz and amplifies and records the safety signal by applying a band pass filter of 0.05 to 35 Hz. A tactile stimulus based cognitive fatigue measuring device.
13. The method of claim 12,
Wherein the data processing and analysis unit comprises:
After separating the signals of the interval of -200 ms to 1000 ms from the point of presentation of each stimulus,
A blinking of the eye in excess of ± 80 μV, and an elimination of the EEG interval due to horizontal eye movements in excess of ± 30 μV.
14. The method of claim 13,
Wherein the data processing and analysis unit comprises:
The baseline is calibrated by subtracting the P300 amplitude of the separated and removed intervals from the presentation time of the stimulus, and subtracting the mean potential value of the time interval of -200 ms before the presentation of the stimulus from the potential value of each interval to measure the tactile stimulation based cognitive fatigue Device.
15. The method of claim 14,
Wherein the data processing and analyzing unit corrects the baseline,
The P300 average potential value of each section is calculated,
Wherein the tactile stimulus-based cognitive fatigue measuring device is characterized in that a point at which the amplitude is the highest in a static potential range of 400 to 600 ms generated after the presentation of the tactile stimulus is set to an amplitude of P300.

Images