KR20180124327A - wearable device and method for measuring the degree of drowsiness or concentration of the user - Google Patents

Abstract

According to the present invention, a wearable device includes: a first sensor unit for acquiring first data related to an electroencephalography (EEG) signal by sensing a biometric signal of a user; a second sensor unit for acquiring second data related to a heartbeat signal by sensing the biometric signal of the user; and a processor for measuring the degree of concentration or drowsiness of the user using the first data and the second data. Accordingly, the present invention can measure the degree of the concentration or drowsiness of the user.

Description

[0001] The present invention relates to a wearable device for measuring the degree of drowsiness or concentration of a wearer and a wearable device for the same,

The present invention relates to a method for measuring the degree of drowsiness or concentration of a user and a wearable device therefor.

Electronic learning (e-learning) and smart learning are increasingly used to achieve effective learning for adolescents. In recent years, smart learning, which learns with wearable devices and smart devices, has become popular with a broader opportunity in the framework of e-learning.

However, an e-learning system that does not have two-way communication with a learner may result in poor learner's concentration and ineffective learning results. Especially, the existing e - learning system was not able to grasp whether the learner was faithful to the class at the moment, and it was not possible to objectively measure learner 's concentration and drowsiness in e - learning class through smart learning.

Considering that one of the most important factors in learning is concentration and sleepiness is one of the most detrimental factors of concentration, existing E - learning systems or smart learning systems have big problems in learning effect.

In this way, existing e-learning system or smart learning system can not learn the concentration of learners and thus the learning effect is reduced. However, it is still necessary to enhance the concentration or drowsiness of the user in relation to education and quantitatively measure There is no means.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a wearable device for measuring the degree of drowsiness or concentration of a user.

Another object of the present invention is to provide a method for measuring the degree of drowsiness or concentration of a user.

According to another aspect of the present invention, there is provided a computer-readable recording medium recording a program for causing a computer to execute a method for measuring a degree of drowsiness or concentration of a user.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, unless further departing from the spirit and scope of the invention as defined by the appended claims. It will be possible.

According to an aspect of the present invention, there is provided a wearable device including a first sensor unit sensing a user's bio-signal to acquire first data related to an EEG signal, A second sensor unit for sensing the user's bio-signal to acquire second data related to a heart rate signal; And a processor for measuring the degree of concentration or drowsiness of the user using the first data and the second data.

Wherein the processor is configured to perform prefiltering on the first data and the second data and to generate at least one first feature for calculating the degree of concentration or the degree of drowsiness of the user from the preprocessed first data And calculating at least one second feature value for calculating the degree of concentration or the degree of drowsiness of the user from the preprocessed second data, and calculating the at least one first feature value and the calculated at least one The degree of concentration of the user or the degree of drowsiness can be measured based on the second characteristic value of the user.

Wherein the processor is configured to perform prefiltering for extracting a signal corresponding to a predetermined first frequency component from the first data and to extract a signal corresponding to a predetermined second frequency component from the second data, A preprocess can be performed. The signal corresponding to the predetermined first frequency component may be a signal corresponding to 1 Hz to 50 Hz. The signal corresponding to the predetermined second frequency component may be a signal corresponding to 1 to 100 Hz branches.

The first sensor unit includes two electrodes, and the first sensor unit senses a signal changed between the Fp1 channel and the A1 channel through the first electrode, thereby extracting the EEG signal. The first sensor unit converts the extracted EEG signal into a digital signal, and the first data may include the converted EEG signal.

The second sensor unit may include one electrode, and the one electrode may be disposed on the wearable device so as to be attached to the forehead of the user.

The first sensor unit may include an electroencephalography (EEG) sensor, and the second sensor unit may include a photoplethysmograph (PPG) sensor.

The wearable device may further include a stimulus generator for generating a stimulus for giving feedback to the user based on the measured degree of concentration or degree of drowsiness of the user.

The processor transmits a notification of the degree of concentration of the user or the degree of drowsiness to a device that the user learns when the degree of concentration of the user is less than a predetermined threshold value or the degree of drowsiness is equal to or greater than a predetermined threshold value As shown in FIG.

Wherein the at least one first characteristic value includes at least one of an average signal intensity value of the preprocessed EEG signal, a standard deviation value of the preprocessed EEG signal intensity, a number of zero crossings when the preprocessed EEG signal is second- Or the like. The at least one second characteristic value may include a value related to a distance between a peak value of the preprocessed heart rate signal and a next peak value.

The processor may convert the second data acquired from the second sensor unit into a digital signal to perform preprocessing.

According to another aspect of the present invention, a wearable device for measuring a degree of drowsiness or a concentration of a user of a wearable device includes sensing a user's bio-signal to acquire first data related to an electroencephalogram signal; Sensing biometric signals of the user to obtain second data related to a heart rate signal; And measuring the degree of concentration or the amount of drowsiness of the user using the first data and the second data to measure the degree of drowsiness or concentration of the user.

The measuring step may include: performing prefiltering on the first data and the second data; Calculating at least one first characteristic value for calculating the degree of concentration or the degree of drowsiness of the user from the preprocessed first data and a degree of concentration or drowsiness of the user from the preprocessed second data; Calculating at least one second characteristic value; And measuring the degree of concentration of the user or the degree of drowsiness based on the calculated at least one first feature value and the calculated at least one second feature value.

The performing the preprocessing may include extracting a signal corresponding to a predetermined first frequency component from the first data and extracting a signal corresponding to a predetermined second frequency component from the second data.

The acquiring of the first data may include extracting the EEG signal in the Fp1 channel and the A1 channel. The obtaining of the second data may further include extracting the heart rate signal through an electrode attached to the forehead of the user.

The method may further include the step of generating a stimulus for giving feedback to the user based on the measured degree of concentration of the user or the degree of drowsiness.

Alternatively, the method may further include, when it is determined that the degree of concentration of the user is below a predetermined threshold value or the degree of drowsiness is equal to or greater than a predetermined threshold value, To the device.

Wherein the at least one first characteristic value includes at least one of an average signal intensity value of the preprocessed EEG signal, a standard deviation value of the preprocessed EEG signal intensity, a number of zero crossings when the preprocessed EEG signal is second- Or the like. The at least one second characteristic value may include a value related to a distance between a peak value of the preprocessed heart rate signal and a next peak value.

The wearable device according to an embodiment of the present invention can accurately measure drowsiness and concentration by simultaneously measuring the EEG and the heart rate (PPG) in a multimodal manner.

In addition, the system for feeding back the degree of drowsiness and / or concentration to the user who is learning according to the present invention induces efficient participation and learning of the online lecture in which the concentration of the user is likely to be low, By quantifying the measurements, teachers or administrators can objectively and accurately check student participation on-line. Also, if the system according to the present invention is used for an on-line lecture in which it is difficult for users to receive feedback, the learning efficiency can be improved.

The effects obtained by the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description will be.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.
1 is a view showing a 10-20 EEG system as an example of an EEG sensor system.
2 is a block diagram illustrating the configuration of a wearable device for measuring the drowsiness and / or concentration of a user according to the present invention.
3 is a graph showing the degree of drowsiness and / or concentration of a user in data (raw data) received from the brain wave sensor unit 210 and the heart rate sensor unit 220 by the processor 230 of the wearable device 200 according to the present invention Fig. 2 is a diagram showing a method for performing measurement for measurement.
Fig. 4 is a diagram illustrating an exemplary number of wavelets. Fig.
5 is a diagram illustrating a method by which the processor 230 determines the degree of drowsiness and / or concentration of power using a classifier of a deep learning algorithm as an example of a machine learning algorithm.
6 is a diagram illustrating an example of the wearable device 200 according to the present invention.
FIG. 7 is an illustration of a system 500 for providing feedback to a learning user of the degree of drowsiness and / or concentration.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following detailed description, together with the accompanying drawings, is intended to illustrate exemplary embodiments of the invention and is not intended to represent the only embodiments in which the invention may be practiced. The following detailed description includes specific details in order to provide a thorough understanding of the present invention. However, those skilled in the art will appreciate that the present invention may be practiced without these specific details.

In some instances, well-known structures and devices may be omitted or may be shown in block diagram form, centering on the core functionality of each structure and device, to avoid obscuring the concepts of the present invention. In the following description, the same components are denoted by the same reference numerals throughout the specification.

 Since the conventional device for measuring concentration and predicting drowsiness relied solely on brain wave information, accurate concentration measurement and drowsiness measurement (or prediction) were impossible. There is no device that simultaneously measures brain waves and heart rate and predicts drowsiness among learning aids. In addition, existing products did not provide feedback or only measure concentration in the field of learning or prediction of drowsiness.

The present invention proposes a wearable device capable of effectively teaching interactive communication by a method in which a learner actually plays a video only when the learner concentrates on the lecture and thereby plays a role of enhancing the learning effect of the learner .

The wearable device according to the present invention measures the index of the activity of the motor cell responding to the autonomic and central nerve stimulation of the human body, establishes the index of the evaluation of the human autonomic nerve, Monitoring and management of the e-learning classes, so as to enable efficient interactive communication between students and lecturers.

An electroencephalographic sensor (hereinafter referred to, for example, as an electroencephalography (EEG) sensor) measures the flow of electricity generated when a signal is transmitted between neurons in the nervous system. Neurophysiological methods for electrical activity of the brain include recording through electrodes attached to the scalp.

1 is a view showing a 10-20 EEG system as an example of an EEG sensor system.

To measure the EEG, the electrode position on the scalp is needed, and a 10-20 EEG system can be used as the system for that measurement. The 10-20 EEG system is the same as shown in Figure 1 and the name is given descriptively. 10 and 20 of "10-20" refer to 10% and 20% of the distance between NASION (anterior) and INION (posterior).

In the channel name shown in Fig. 1, the odd number always refers to the electrode position on the left side of the head, and the even number refers to the electrode position on the right side of the head. The letter "z" is used to indicate a point between the center line of the center line between NASION (Nz) and INION (Iz). F is the frontal, C is the central (central), P is the temporal (Parietal), O is the occipital (O), T is the occipital, Represents the temporal.

In FIG. 1, NASION refers to a small notch above the nose, below the forehead, and INION refers to a small bump or ridge at the neck, occiput base. The distance between these two points can be measured using a tape measure, etc., and it is accurate to measure in centimeters and a typical measurement is 36 centimeters.

At NASION, 10% of this distance is the midpoint between the two prefrontal lobe sites called FP1 and FP2. Also, Fz is 20% of the distance between NASION and INION. NASION and INION move 20% more in Fz, which is the middle point between NASION and INION.

In general, brain waves are artificially delta (δ) waves (0.2 to 3.99 Hz), theta waves (4 to 7.99 Hz), alpha waves (8 to 12.99 Hz) (13 to 29.99 Hz), and gamma-g waves (30 to 50 Hz).

Delta waves appear predominantly in the deep sleep of a normal person or in a newborn. Theta pa is a wave that occurs mainly in the process of emotional stability or sleep, and is related to various states of memory, creativity, and concentration. Alpha waves appear mainly in relaxed conditions, such as relaxation, and the amplitude increases with more stable and relaxed conditions. Beta waves appear predominantly in the frontal areas and appear when performing all conscious activities, such as when awake, when speaking. Especially in an unstable state or tension, it appears to be dominant in complex calculation processing. Gamma waves are more emotionally more irritated in the form of vibrating faster than the beta waves, or related to advanced cognitive information processing such as reasoning and judgment.

1 illustrates an EEG measurement channel measured by an EEG sensor of a wearable device according to the present invention. EEG1 measures the drowsiness and concentration in Fp1 channel, EEG2 measures the concentration in the Cz channel (EEG2), the sleepiness and the concentration in the brain, . A1 channel is used as REF (Reference) and A2 channel is used as GND (Ground).

Next, a PPG (Photoplethysmograph) sensor will be described as an example of the heart rate sensor. The PPG sensor is called an optical pulse wave sensor. Photoplethysmograph (PPG) is a pulse wave measurement method that can measure heart rate activity or heart rate by measuring blood flow through blood vessels using the optical characteristics of living tissue. Pulse is a pulsating wave of blood that the blood swells in the heart. It can be measured by changing the blood flow, which is caused by the relaxation and contraction of the heart, that is, the volume change of the blood vessel. Pulsed pulse wave measurement is a method of measuring pulse waves using light. The optical sensor senses and measures changes in optical characteristics such as reflection, absorption and transmission ratios of the biomedical tissue when the volume changes, thereby enabling pulse measurement . This method is widely used because it has advantages such as non-invasive pulse measurement, miniaturization and ease of use, and can be used as a bio-signal detection sensor in a wearable device.

2 is a block diagram illustrating the configuration of a wearable device for measuring the drowsiness and / or concentration of a user according to the present invention.

2, the wearable device 200 for measuring drowsiness and / or concentration of a user includes a brain wave sensor unit 210, a heart rate sensor unit 220, a processor 230, a power supply unit 240, and a communication unit 250, . ≪ / RTI >

The brain-wave sensor unit 210 can sense or extract the bio-signal of the user under study and extract or acquire data on the brain-wave signal. The electroencephalogram sensor unit 210 may include at least two electrodes (for example, the present invention may be referred to as two electrodes) for sensing the body of a human body. In the case where the EEG sensor 210 includes two electrodes, the EEPROM 210 senses signals that change between the Fp1 channel and the A1 channel through the first electrode EEG1 of the two electrodes, (The second electrode may be a reference electrode). The brain wave sensor unit 210 may transmit data or information on the calculated or extracted brain wave signal to the processor 230. At this time, the brain-wave sensor unit 210 may convert the data of the extracted EEG signal into digital signal data and transmit the converted data to the processor 230. Thus, the brain-wave sensor unit 210 can measure an EEG signal based on a 10-20 EEG system using two electrodes.

The EEG sensor unit 210 may be implemented as a 4-bit ADC converter (ADS 1299) as an example. The 4-bit ADC converter (ADS 1299) enables real-time conversion and digital bio-signal monitoring. The 4-bit ADC converter (ADS 1299) is a small chip that converts analog signals to digital signals, obtains clean signals with low noise, and can operate with low power, making it easy to use for a long time. In addition, the 4-bit ADC converter (ADS 1299) can measure 8 channels, so the channel is easy to use and supports up to 24bit resolution.

The heart rate sensor unit 220 can sense or extract the bio-signal of the user during learning and extract or acquire data on the heart rate signal. The heart rate sensor unit 220 may transmit data on the extracted or obtained heart rate signal to the processor 240 in the form of an analog signal. The heart rate sensor unit 220 can measure the heart rate using one electrode. Since the biological signal is sensitive to the movement of the human body, it is important to measure the biological signal in a situation where the movement is small. In the case of the present invention, it is preferable to measure the bio-signal in a situation where the user is learning less, such as a situation where the user is learning, but it is preferable to sense the bio-signal in the forehead of the user with less motion. The heart rate sensor unit 220 may include one electrode, and one electrode may be attached to the forehead to sense the heart rate. The heart rate sensor unit 220 may also be referred to as a forehead PPG sensor. As an example, the heart rate sensor 230 may calculate the actual heart rate by detecting a peak occurring every interval in the PPG.

3 is a graph showing the degree of drowsiness and / or concentration of a user in data (raw data) received from the brain wave sensor unit 210 and the heart rate sensor unit 220 by the processor 230 of the wearable device 200 according to the present invention Fig. 2 is a diagram showing a method for performing measurement for measurement.

Referring to FIG. 3, the processor 230 receives raw data (Raw data) received from the brain-wave sensor unit 210 (S310). The processor 230 may perform prefiltering to extract a frequency component corresponding to a predetermined frequency to be used for drowsiness and concentration measurement in Raw data (S320). At this time, it may be a frequency component (for example, a frequency component in the range of 1 Hz to 50 Hz) corresponding to a predetermined frequency to be used for calculating the degree of drowsiness and concentration, and in this case, a sampling frequency of at least 100 Hz is used. Most of the biological signals that can be acquired from the human body have nonlinear and nonstationary characteristics. That is, the intensity and frequency of the bio-signal changes with time, and in the case of brain waves, the conventional Fourier-based frequency analysis method using the fixed basis function is not suitable. For frequency analysis of such nonlinear and abnormal signals, time frequency analysis is used. The processor 230 adds a time axis to the existing frequency analysis method to observe a change in frequency component with time and corresponds to a frequency range (from 3 Hz to 50 Hz) that can be used for drowsiness and / or concentration measurement / analysis And extracts the frequency component.

Hereinafter, various methods of prefiltering the EEG signal data (raw data) will be briefly described. Specifically, there are a short-time Fourier transform (STFT) method, a wavelet method, and a multivariate empirical mode decomposition (MEMD) method as a method of performing pre-processing of brain wave data. The STFT method is a Fourier analysis method that can observe the change of the frequency and phase information of the signal over time by applying the Fourier transform to the signal divided into several short sections. When a signal is divided into a short section and a Fourier transform is applied, a single Fourier spectrum is generated in each section. Generally, the Fourier spectra thus generated are represented by a spectrum change over time.

The wavelet represents a function whose start and end intensity of the signal is 0, and the wavelet transform performs the frequency analysis of the signal by using such a wavelet function as a basis. The wavelet function can be recursively defined from a basic function called a mother wavelet. The Mexican-hat function as shown in FIG. 4 is used as the number of the mother wavelet. However, assuming that the definition of the wavelet function described above is satisfied It is acceptable to use any number of masters. Wavelet transform has the advantage of using free number of moons as compared to Fourier analysis using sinusoidal wave basis. It is mainly used for analysis of signal frequency in audio and image processing fields. Comparing the time-frequency analysis results of the EEG signals using the STFT method and the wavelet transform method, the signal intensity of the STFT is widely dispersed throughout the Mu and beta regions, while the wavelet is relatively concentrated in each region And it is easier to analyze the frequency.

Next, the MEMD method will be described.

The EMD (Empirical Mode Decomposition) method analyzes data-driven signals using data base derived from data without using fixed base functions such as those used in Fourier and wavelets for the effective analysis of nonlinear and abnormal biosignals. Technology. By performing frequency analysis using data without using a fixed base such as a sine wave, it is possible to prevent the signal intensity of the frequency spectrum from being dispersed widely so that the signal intensity of the intrinsic mode function (IMF) The effect can be obtained.

The MEMD method is a frequency analysis method used to apply EMD to a bio signal using a plurality of channels, and the Intrinsic Mode Function (IMF) generated in each channel is designed to share the same frequency domain. In the results of frequency analysis using MEMD, it is difficult to confirm the exact frequency of each component because the frequency components of the Mu and Beta regions are dispersed in STFT and wavelet-based frequency analysis methods. In MEMD, however, The frequency analysis can be performed more precisely and it can be easily used for instantaneous frequency analysis such as Hilbert transform.

As such, the processor 230 may perform pre-processing on brain wave data corresponding to a digital signal using a Short-Time Fourier Transform (STFT) method, a Wavelet method, or a Multivariate Empirical Mode Decomposition (MEMD) method.

Thereafter, the processor 230 may calculate at least one feature value to be used for calculating the degree of drowsiness or concentration of concentration from the preprocessed EEG data (S330). Twelve feature values used for drowsiness and concentration analysis are extracted or calculated from the pre-processed EEG data using STFT, wavelet or MEMD method, and each feature value can be defined as follows.

Mean: The mean signal intensity of EEG signals.

2. Standard deviation: Standard deviation of EEG signal intensity.

3. Zero Crossing: indicates the number of zero crossings when the EEG signal is second-differentiated.

4. Kurtosis: The value obtained by normalizing the fourth order momentum of the signal to the square of the standard deviation, which indicates the peakedness of the probability distribution of the signal intensity.

5. Crest factor: The ratio of the peak value of the signal. It is similar to Kurtosis and is a feature value that describes the signal's impulse response.

6. Power Spectrum Density: The power spectrum density generated by the Fourier transform of EEG signals.

Correlation: It shows the auto-correlation of EEG signals.

8. Threshold Crossing: It shows the number of filtered raw data over a certain threshold in one Epoch.

9. Skewness: The third momentum of the EEG signal, indicating the degree to which the distribution of signal strength is biased.

10. Entropy: Indicates the predictability of the signal.

11. Band energy: Represents energy in the frequency domain associated with stress.

12. Spectral Flux: PSD (t) / PSD (t-1) means the rate of change of the power spectrum over time when analyzing the signal's frequency.

The processor 230 may measure or calculate the degree of drowsiness and / or concentration of the user under study by applying at least one of the twelve calculated feature values to a predetermined machine learning algorithm (S340). Here, as an example, a predetermined machine learning algorithm includes a random forest algorithm, a random subspace algorithm, or a bagging algorithm.

The processor 230 may classify at least one characteristic value extracted from the preprocessed EEG data through a machine learning (machine learning) algorithm to measure the degree of drowsiness and / or the degree of concentration. Hereinafter, the machine learning algorithm will be briefly described.

The random subspace method randomly divides the subspace of the feature value space and uses a plurality of classifiers trained from each divided subspace. The result generated through the plurality of classifiers is used for voting or using the maximum value Generate final classification results. The random forest method is a decision tree-based algorithm. Forests are generated through a plurality of decision trees trained using randomly reconstructed extracted feature value samples. The final classification result is determined in a similar manner as in the random subspace method. The result of classification of the tree is collected and generated.

Bagging is an abbreviation for bootstrap aggregating, which means that several weak classifiers are generated and the final decision is a sorting method that combines the classification results of many weak classifiers into voting. By combining the classification results in this manner, the bagging algorithm can obtain the stability of classification results, prevent overfitting, and improve accuracy.

The n-fold crossover verification method is used to evaluate the classification performance and to avoid over sum. The performance evaluation criteria are the accuracy, recall, precision, and receiver operating characteristic (ROC) AUC (area under ROC curve) can be used.

Next, the processor 230 receives heart rate signal data (Raw data) from the heart rate sensor unit 220 (S310), and the processor 230 calculates a predetermined value of the heart rate signal data (Raw data) A prefiltering process for extracting a frequency component corresponding to a frequency may be performed (S320). At this time, the processor 230 may be a frequency component corresponding to a predetermined frequency (for example, a frequency component in a range of 1 Hz to 100 Hz), and in this case, a sampling frequency of at least 200 Hz is used. The processor 230 may extract at least one of the following characteristic values from the preprocessed data (S330).

1. Peak value of the measured PPG signal (Peak of PPG and R waveform of ECG are deeply related)

2. A signal obtained by dividing peak value data of the measured PPG signal by a predetermined window

3. Obtain the following characteristic values from the peak values of the PPG signal (NN interval is the interval between the peak point of the PPG signal and the next peak point)

(1) meanNN: expected value of NN Interval

(2) SDNN: standard deviation of NN interval

(3) SDSD: Standard deviation of the difference of NN Interval

(4) NNx_Count: Tested 10, 20, 30, 40, 50 as the value of x, which is derived from intervals (NN)

(5) pNNx: Percentage of absolute differences in successive NN values> x (ms) Test 10, 20, 30, 40,

(6) TF: power of the TF (total frequency) band (0.14-0.4 Hz) at the NN interval PSD

(7) VLF: Power of VLF (very low frequency) band (0-0.04Hz)

(8) LF: Power of LF (low frequency) band (0.04-0.15Hz)

(9) HF: Power of HF (high frequency) band (0.15-0.4Hz)

(10) LFn: the ratio of the LF band to the LF + HF band (0.04 to 0.4 Hz)

(11) HFn: ratio of HF band to LF + HF band (0.04-0.4 Hz)

(12) LFHF: ratio of LF band to HF band

The processor 230 substitutes the values of 1, 2, and 3 corresponding to the twelve characteristic values calculated from the heart rate data received from the heart rate sensor 220 into a predetermined machine learning algorithm to calculate the degree of drowsiness and / (S340).

The processor 230 performs preprocessing using the data of the EEG signals received from the EEG sensor 210 as well as the data of the heart rate signal received from the heart rate sensor 220, Using the characteristic values of the EEG signal and the heart rate signal, the degree of drowsiness and / or the degree of concentration can be measured through a predetermined machine learning algorithm.

5 is a diagram illustrating a method by which the processor 230 determines the degree of drowsiness and / or concentration of power using a classifier of a deep learning algorithm as an example of a machine learning algorithm.

The feature values calculated from the raw data itself or the raw data corresponding to the data received from the brain wave sensor unit 210 and the feature values calculated from the raw data itself or the raw data corresponding to the data received from the heart rate sensor unit 220 Are input to the input layer of the deep learning algorithm shown in FIG. 5 at the same time. Label information is given to the output layer. The weights of each layer of the deep learning algorithm can be continuously updated according to learning data input using an algorithm such as a restricted Boltzman machine or a convolutional neural network or a recurrent neural network.

When the new test data is received from the brain-wave sensor unit 210 and the heart-rate sensor unit 220 using the deep learning classifier that has been finally learned according to the input data, the processor 230 It is possible to determine whether the current state of the user is in a drowsy state or in a state of low concentration.

As described above, the processor 230 calculates feature values for data received from the brain-wave sensor unit 210 and the heart rate sensor unit 220, applies the feature values to a predetermined machine learning algorithm, And the degree of concentration or concentration can be measured.

As an example for determining the degree of drowsiness and / or concentration of the user of the processor 230, drowsiness experiment data previously secured may be used. For example, when at least 10 experimental groups are performing the PPG and EEG measurements by the EEG sensor 210 and the heart rate sensor 220, information on specific EEG signals and heart rate when sleepiness occurs is shown as a sleepiness test Data can be recorded, and the processor 230 can judge based on the drowsiness test data defined beforehand when it judges the degree of drowsiness of the user.

As an example, 10 or more experimental group data may be used to find out the singularities of the drowsiness and concentration-related PPG and EEG signals using a machine learning algorithm, and the degree of drowsiness and concentration may be quantified, The degree of drowsiness and / or concentration can be determined based on quantitative drowsiness, concentration information, and information on measured EEG signals and heart rate signals.

The power supply unit 240 can supply power to the wearable device 200 through the charging circuit. The communication unit 250 is a component for transmitting and receiving signals with other devices. As an example, the communication unit 250 may be a Bluetooth device 250 for enabling communication with other wireless communication devices in a Bluetooth manner at a short distance.

6 is a diagram illustrating an example of the wearable device 200 according to the present invention.

As an example of the wearable device 200, there is a headset device that can be used by a user who is learning. The headset device may include an EEG sensor part (a first electrode 213 and a second electrode 215) and a heart rate sensor part (one electrode 220) in the same manner as the components included in the wearable device. 6, the head set device may further include a processor 230, a power supply 240, and a Bluetooth device 250 as an example of a communication unit. The headset device can be configured such that the heart rate sensor portion (one electrode 220) can attach the electrode 220 to the forehead to calculate the heart rate. The headset device shown in Fig. 6 can perform the same function as an example of the wearable device shown in Fig.

FIG. 7 is an illustration of a system 500 for providing feedback to a learning user of the degree of drowsiness and / or concentration.

Referring to FIG. 7, the system 500 may include a user 300 according to the present invention and a device 400 that a user may use for learning.

The wearable device 200 may further include a stimulus generator 260. [ The processor 230 of the wearable device 200 measures the degree of drowsiness and / or concentration of the user 300 and if the user 300 sleeps or the degree of concentration decreases below a predefined threshold, And notify the user of the notification to the device 400 used by the user 300 for learning. Alternatively, the processor 230 may forward the notification to the stimulus generator 260 and forward the stimulus generator 260 to the device 400 that the user 300 uses for learning.

When the user 300 is notified from the processor 230 or the stimulus generator 260 that the user 300 is sleeping or the concentration of the user is low, the device 400 that the user 300 uses for the learning Stopping the drowsiness and controlling the player to play again in a state of concentrating. Alternatively, the device 400 used by the user 300 for learning may allow the user 300 to change the medium, such as brightening the computer screen or adjusting the size of the sound, in order to break the drowsiness and concentrate on the lecture, 300). ≪ / RTI >

The stimulus generator 260 of the wearable device 200 receives a notification from the processor 230 indicating that the user 300 is sleeping or the degree of concentration is lowered below a predefined threshold, Stimulation may be generated and feedback may be given to the user 300.

As described above, the wearable device 200 according to the present invention can quantitatively measure drowsiness and concentration by simultaneously measuring the EEG and the heart rate (PPG) in a multimodal manner. In addition, since the headset device according to an exemplary embodiment of the wearable device 200 according to the present invention measures less motion, it is possible to measure motion sensitive living body signals with high accuracy.

In addition, the system 500 for feedbacking the degree of drowsiness and / or concentration to the user in learning according to the present invention induces efficient participation and learning of the online lecture, By quantifying concentration and drowsiness measurements, teachers or administrators can objectively and accurately check student participation on-line. In addition, if the system 500 according to the present invention is used for an on-line lecture in which it is difficult for users to receive feedback, online learning efficiency can be enhanced.

The wearable device 200 and the system 500 according to the present invention quantitatively measure user concentration and drowsiness and provide feedback to the user, thereby maximizing the academic achievement of the user or the learner.

The embodiments described above are those in which the elements and features of the present invention are combined in a predetermined form. Each component or feature shall be considered optional unless otherwise expressly stated. Each component or feature may be implemented in a form that is not combined with other components or features. It is also possible to construct embodiments of the present invention by combining some of the elements and / or features. The order of the operations described in the embodiments of the present invention may be changed. Some configurations or features of certain embodiments may be included in other embodiments, or may be replaced with corresponding configurations or features of other embodiments. It is clear that the claims that are not expressly cited in the claims may be combined to form an embodiment or be included in a new claim by an amendment after the application.

It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the essential characteristics thereof. Accordingly, the above description should not be construed in a limiting sense in all respects and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

Claims (25)

A first sensor unit sensing a user's bio-signal to acquire first data related to an EEG signal;
A second sensor unit for sensing the user's bio-signal to acquire second data related to a heart rate signal; And
And a processor for measuring the degree of concentration or drowsiness of the user using the first data and the second data. The method according to claim 1,
The processor comprising:
Performing prefiltering on the first data and the second data,
Calculating at least one first characteristic value for calculating the degree of concentration or the degree of drowsiness of the user from the preprocessed first data and a degree of concentration or drowsiness of the user from the preprocessed second data; Calculating at least one second characteristic value,
And measures the degree of concentration or the degree of drowsiness of the user based on the calculated at least one first feature value and the calculated at least one second feature value. The method according to claim 1,
The processor comprising:
Performing prefiltering for extracting a signal corresponding to a predetermined first frequency component from the first data,
And performs a pre-processing for extracting a signal corresponding to a predetermined second frequency component from the second data. The method of claim 3,
Wherein the signal corresponding to the predetermined first frequency component is a signal corresponding to 1 Hz to 50 Hz. The method of claim 3,
Wherein the signal corresponding to the predetermined second frequency component is a signal corresponding to 1 Hz to 100 Hz. The method according to claim 1,
The first sensor unit includes two electrodes,
Wherein the first sensor unit senses a signal changed between the Fp1 channel and the A1 channel through the first electrode to extract the EEG signal. The method according to claim 1,
The first sensor unit converts the extracted EEG signal into a digital signal,
Wherein the first data comprises the converted digital brain wave signal. The method according to claim 1,
Wherein the second sensor unit includes one electrode,
Wherein the one electrode is disposed on the wearable device such that it can be attached to the wearer's forehead. The method according to claim 1,
Wherein the first sensor unit comprises an Electroencephalography (EEG) sensor. The method according to claim 1,
Wherein the second sensor unit comprises a PPG photoplethysmograph (PPG) sensor. The method according to claim 1,
And a stimulation generating device for generating stimulation for giving feedback to the user based on the measured degree of concentration of the user or degree of drowsiness. The method according to claim 1 or 11,
The processor transmits a notification of the degree of concentration of the user or the degree of drowsiness to a device that the user learns when the degree of concentration of the user is less than a predetermined threshold value or the degree of drowsiness is equal to or greater than a predetermined threshold value The wearable device. 3. The method of claim 2,
Wherein the at least one first characteristic value includes at least one of an average signal intensity value of the preprocessed EEG signal, a standard deviation value of the preprocessed EEG signal intensity, a number of zero crossings when the preprocessed EEG signal is second- The wearable device comprising: 3. The method of claim 2,
Wherein the at least one second characteristic value comprises a value related to a distance between a peak value of the preprocessed heart rate signal and a next peak value. The method according to claim 1,
Wherein the processor converts the second data acquired from the second sensor unit into a digital signal to perform preprocessing. A wearable device for measuring the degree of drowsiness or concentration of a user,
Sensing a user's bio-signal to obtain first data related to an EEG signal;
Sensing biometric signals of the user to obtain second data related to a heart rate signal; And
Measuring a degree of concentration or a degree of drowsiness of the user using the first data and the second data; and measuring the degree of drowsiness or concentration of the user. 17. The method of claim 16,
Wherein the measuring step comprises:
Performing prefiltering on the first data and the second data;
Calculating at least one first feature value for calculating the degree of concentration or the degree of drowsiness of the user from the preprocessed first data and a degree of concentration or drowsiness of the user from the preprocessed second data; Calculating at least one second characteristic value; And
And measuring the degree of concentration or the degree of drowsiness of the user based on the calculated at least one first feature value and the calculated at least one second feature value, / RTI > 18. The method of claim 17,
The pre-processing may include extracting a signal corresponding to a predetermined first frequency component from the first data and extracting a signal corresponding to a predetermined second frequency component from the second data. A method for measuring the degree of drowsiness or concentration. 17. The method of claim 16,
Wherein the acquiring of the first data comprises extracting the EEG signal in the Fp1 channel and the A1 channel. 17. The method of claim 16,
Wherein the step of acquiring the second data further comprises extracting the heart rate signal via an electrode attached to the forehead of the user. 17. The method of claim 16,
And generating a stimulus for giving feedback to the user based on the measured degree of concentration or degree of drowsiness of the user. 17. The method of claim 16,
When the degree of concentration of the user is less than or equal to a predetermined threshold value or when the degree of drowsiness is determined to be equal to or greater than a predetermined threshold value, the step of notifying the user about the degree of concentration or the degree of drowsiness of the user A method for measuring the degree of drowsiness or concentration of a user. 18. The method of claim 17,
Wherein the at least one first characteristic value includes at least one of an average signal intensity value of the preprocessed EEG signal, a standard deviation value of the preprocessed EEG signal intensity, a number of zero crossings when the preprocessed EEG signal is second- The degree of drowsiness or the degree of concentration of the user. 18. The method of claim 17,
Wherein the at least one second feature value comprises a value related to a distance between a peak value of the preprocessed heart rate signal and a next peak value. A computer-readable recording medium having recorded thereon a program for causing a computer to execute a method of measuring the degree of drowsiness or degree of concentration of a user according to any one of claims 16 to 24.

Images