KR20190140864A - Method and apparatus for determining concentration level of user - Google Patents

Abstract

Provided are a method for determining a concentration level of a user and a device thereof. The method for determining a concentration level provides content to a user through a head mounted display (HMD) for determining a concentration level, obtains at least one of brainwave information and eye gaze information of the user watching the content through the HMD, and determines the concentration level of the user based on the at least one of first brainwave information and first eye gaze information.

Description

METHOD AND APPARATUS FOR DETERMINING CONCENTRATION LEVEL OF USER}

The technical field relates to a technology for determining a level of concentration of a user, and more particularly, to an apparatus and a method for determining a level of concentration of a user based on brain wave information and gaze information of a user.

Development refers to the functional development process as it grows, usually a dynamic process that proceeds in a constant and predictable manner. A developmental disability does not refer to a specific disease or disorder, but refers to a case in which development to be performed at a corresponding age is not achieved, and the developmental screening test is 25% behind the normal expectation of the age. Overall developmental delay is a delay in two or more of the gross motor, fine motor, cognition, language, sociality and everyday life. As described above, the developmental disorder is delayed compared with age, and thus can be improved by learning and learning with the body. Therapeutic simulators are used to treat people with developmental disabilities suffering from these developmental disorders. In general, as a method for treating developmental disorders, a method for improving cognitive ability by providing visual and auditory stimuli, such as a previously filed Patent Application No. 10-1008541, [a cognitive ability improving device for providing an auditory stimulus], and a display device Through the use of simulation to provide treatment, visual stimulation and physical vestibular stimulation to the brain, treatment, and drug inhalation are being applied.

One embodiment may provide an apparatus and method for determining a level of concentration of a user.

One embodiment may provide an apparatus and method for improving the level of concentration of a user.

In accordance with an aspect, the concentration level determining method may include providing first content to a user through a head mounted display (HMD), first brain wave information and a first brain wave information of the user viewing the first content through the HMD; Obtaining at least one of gaze information, and determining a concentration level of the user based on at least one of the first brain wave information and the first gaze information.

Acquiring at least one of the first brain wave information and the first gaze information of the user who views the first content through the HMD is based on at least one of the first brain wave information and the first gaze information. Changing the object in the first content, and stopping the providing of the first content when the change of the object corresponds to a preset change.

The change of the object may change at least one of the shape of the object and the position of the object.

The changing of the object in the first content may include determining whether the gaze of the user corresponds to the object among a plurality of objects of the first content based on the first gaze information, and the gaze is If the object corresponds to the object may include the step of changing the object.

Acquiring at least one of brain wave information and gaze information of the user who views the first content through the HMD may include setting a preset timer for the first content even if the change of the object does not correspond to a preset change. The method may further include stopping the providing of the first content when it is finished.

The method for determining the concentration level may include providing second content to the user when the first content is stopped, and second brain wave information of the user who views the second content through the HMD. And obtaining at least one of second gaze information, and the calculating of the concentration level of the user may include calculating the concentration level based on at least one of the second brain wave information and the second gaze information of the user. Calculating may be included.

The calculating of the concentration level of the user may include dividing the first EEG information into brain wave signals of a plurality of frequency bands, and calculating the concentration level based on a target EEG signal among the brain wave signals. It may include.

The step of calculating the concentration level of the user may be performed using a weighted fuzzy membership function-based deep neural network (NEF) based on at least one of the first brain wave information and the first gaze information. Calculating the concentration level.

The NEWFM classifies a class of Takagi-Sugeno defuzzification values and Takagi-Sugeno inverse fuzzy values when receiving at least one of the first EEG information and the first gaze information. The concentration level can be calculated by outputting the result and the weighted fuzzy membership function.

According to another aspect, a server for determining a concentration level of a user includes a memory in which a program for determining a concentration level of a user is recorded, and a processor for executing the program, wherein the program includes a head mounted display (HMD). Providing a first content to a user through, Acquiring at least one of the first brain wave information and the first gaze information of the user viewing the first content through the HMD, The first brain wave information and the Determining a concentration level of the user based on at least one of first gaze information.

Acquiring at least one of the first brain wave information and the first gaze information of the user who views the first content through the HMD is based on at least one of the first brain wave information and the first gaze information. Changing the object in the first content, and stopping the providing of the first content when the change of the object corresponds to a preset change.

The changing of the object in the first content may include determining whether the gaze of the user corresponds to the object among a plurality of objects of the first content based on the first gaze information, and the gaze is If the object corresponds to the object may include the step of changing the object.

The calculating of the concentration level of the user may include dividing the first EEG information into brain wave signals of a plurality of frequency bands, and calculating the concentration level based on a target EEG signal among the brain wave signals. It may include.

The step of calculating the concentration level of the user may be performed using a weighted fuzzy membership function-based deep neural network (NEF) based on at least one of the first brain wave information and the first gaze information. Calculating the concentration level.

The NEWFM classifies a class of Takagi-Sugeno defuzzification values and Takagi-Sugeno inverse fuzzy values when receiving at least one of the first EEG information and the first gaze information. The concentration level can be calculated by outputting the result and the weighted fuzzy membership function.

An apparatus and method may be provided for determining the level of concentration of a user.

An apparatus and method may be provided that enhance the level of concentration of a user.

1 is a block diagram of a system for determining a level of concentration of a user according to an example.
2 is a block diagram of a server according to an exemplary embodiment.
3 is a flowchart of a method of determining a level of concentration of a user according to an embodiment.
4 illustrates content in which an object changes in a user's concentration according to an example.
5 is a flowchart illustrating a method of acquiring EEG information and gaze information based on a change of an object according to an example.
6 is a flowchart of a method of changing an object in content based on eye gaze information according to an example.
7 illustrates gaze information of a user who views content according to an example.
8 illustrates divided EEG signals of a plurality of frequency bands based on EEG information and EEG information according to an example.
9 is a flowchart illustrating a method of providing second content to a user and obtaining second brain wave and second gaze information of the user, according to an exemplary embodiment.
10 is a flowchart of a method of calculating a concentration level based on brain wave signals according to an example.
11 is a flowchart illustrating a class classification method using a weighted fuzzy membership function based deep neural network, according to an embodiment.
12 to 14 illustrate a weighted fuzzy membership function based deep neural network according to an example.
15 is a block diagram illustrating a user device according to one embodiment.

Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings. However, the scope of the patent application is not limited or limited by these embodiments. Like reference numerals in the drawings denote like elements.

Various modifications may be made to the embodiments described below. The examples described below are not intended to be limited to the embodiments and should be understood to include all modifications, equivalents, and substitutes for them.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of examples. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "comprise" or "have" are intended to indicate that there is a feature, number, step, action, component, part, or combination thereof described on the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art, and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

In addition, in the description with reference to the accompanying drawings, the same components regardless of reference numerals will be given the same reference numerals and duplicate description thereof will be omitted. In the following description of the embodiment, when it is determined that the detailed description of the related known technology may unnecessarily obscure the gist of the embodiment, the detailed description thereof will be omitted.

1 is a block diagram of a system for determining a level of concentration of a user according to an example.

According to an aspect, the system 100 may include a server 110 that determines a level of concentration of a user, a user device 120 that outputs content, and an analysis terminal 130 of a medical institution.

The server 110 may provide pre-produced content to the user device 120 to determine the level of concentration of the user. For example, the content may be virtual reality (VR) content. The content is described in detail with reference to FIG. 4 below.

The user device may include a head mounted display (HMD) 120, and may further include a user terminal 130 that may be connected to the HMD 120 when there is no processing device in the HMD 120. The HMD 120 may measure a user's response to the content while the content is output. When the VR content is provided through the HMD 120, the user can watch the content in an immersive manner.

According to an aspect, the HMD 120 may include a camera and generate gaze information of a user using the camera. For example, the camera may be an infrared camera, and by using an infrared camera, the movement of the user's eye (specifically, the pupil) may be detected even in a dark situation in which the inside is sealed through the HMD.

In addition, the HMD 120 may further include an EEG sensor, and may generate EEG information of the user using the EEG sensor.

The server 110 obtains gaze information and brain wave information from the user device 120, and determines the level of concentration of the user based on the gaze information and the brain wave information.

The server 110 may transmit the gaze information, the EEG information, and the level of concentration to the analysis terminal 130. For example, the user of the analysis terminal 130 may be a doctor, the doctor may adjust the content of the content provided to the user based on the transmitted information, or adjust the algorithm for determining the level of concentration. For example, if the level of concentration determined through the algorithm is higher than the level of concentration determined by the doctor based on the information, the user (or doctor) may adjust the algorithm to lower the level of concentration.

Hereinafter, a method of determining the level of concentration of the user will be described in detail with reference to FIGS. 2 to 15.

2 is a block diagram of a server according to an exemplary embodiment.

The server 200 includes a communication unit 210, a processor 220, and a memory 230. For example, the server 200 may be the server 110 described above with reference to FIG. 1.

The communication unit 210 is connected to the processor 220 and the memory 230 to transmit and receive data. The communication unit 210 may be connected to another external device to transmit and receive data. Hereinafter, the expression “transmitting / receiving“ A ”may refer to transmitting / receiving“ information or data indicating A ”.

The communication unit 210 may be implemented as a circuit in the server 200. For example, the communication unit 210 may include an internal bus and an external bus. As another example, the communication unit 210 may be an element connecting the server 200 and an external device. The communication unit 210 may be an interface. The communication unit 210 may receive data from an external device and transmit data to the processor 220 and the memory 230.

The processor 220 processes data received by the communication unit 210 and data stored in the memory 230. A "processor" may be a data processing device implemented in hardware having circuitry having a physical structure for performing desired operations. For example, desired operations may include code or instructions included in a program. For example, data processing devices implemented in hardware may include a microprocessor, a central processing unit, a processor core, a multi-core processor, and a multiprocessor. , An application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA).

The processor 220 executes computer readable code (eg, software) stored in the memory (eg, memory 230) and instructions caused by the processor 220.

The memory 230 stores data received by the communication unit 210 and data processed by the processor 220. For example, the memory 230 may store a program (or an application or software). The stored program may be a set of syntaxes that are coded to determine the level of concentration of the user and executable by the processor 220.

According to one aspect, the memory 230 may include one or more volatile memory, nonvolatile memory and random access memory (RAM), flash memory, hard disk drive, and optical disk drive.

The memory 230 stores a set of instructions (eg, software) for operating the server 200. The instruction set for operating the server 200 is executed by the processor 220.

The communication unit 210, the processor 220, and the memory 230 will be described in detail with reference to FIGS. 3 to 14 below.

3 is a flowchart of a method of determining a level of concentration of a user according to an embodiment.

Steps 310 to 330 below are performed by the server 200 described above with reference to FIG.

In step 310, the server 200 provides content to the user through the user device. For example, content may be provided to the user through the HMD of the user device. The user device may obtain the user's response to the content while the content is provided to the user. The response of the user may be brain wave information and gaze information. Various contents for testing the concentration of the user may be pre-produced and provided to the user

4 illustrates content in which an object changes in a user's concentration according to an example.

According to one aspect, the content may be content that changes the displayed object. For example, the content may be content that changes the position of the second object 412 placed on the first object 411. In the first state 410 of the content initially provided to the user, the user may focus on the second object 412 to move the second object 412. The concentration of the user changes the brain waves of the user, and the gaze of the user is directed toward the second object 412. If it is determined that the user has focused, a second state 420 may be provided in which the position of the second object 412 is changed. The position of the second object 412 in the second state 420 may vary depending on the degree of concentration of the user.

According to an aspect, an embodiment of changing the position of an object in the content is illustrated, but as another example, content using super powers such as content that a user flies, content moving, speech content, and object perspective content may be considered. It is not limited to the described embodiment.

According to another aspect, the content may be sequential memory content, numeric memory content, pattern memory content, face memory content, or the like. The content may be added to obstruct the user's memory. For example, sound, flash light can be added as a disturbing element.

According to another aspect, the content may be content that finds a card meeting a condition among the presented phrases. The above content requires logical reasoning, and a personal pattern of logical reasoning of the user may be derived through tracking the eyes of the user. Through the above contents, a portion of the user's difficulty can be identified, and an index can be established accordingly.

According to another aspect, the content may be content for spatial perception. Depending on the difficulty, the complexity of the space may increase.

According to another aspect, the content may be content that provides a virtual reality to the user. For example, a situation within the content may be set to a special situation, such as a classroom lesson or a test time, and content that solves the problem within the situation may be provided. It is possible to increase adaptability in reality by solving a problem through a set situation. According to another aspect, content may be provided to a user in a way of solving a problem such as a quiz. For example, a plurality of objects may be output to the user and content may be provided to request the user to arrange the order of the output objects. The user may concentrate and observe the arrangement order of the plurality of objects, and arrange the objects through selection according to the line of sight. In addition, the selection of the object according to the gaze occurs through the EEG information obtained from the user. For example, even if the gaze corresponds to the object, selection of the object may not occur if the concentration level does not exceed a predetermined level through the EEG information. Content providing a way of solving a problem such as a quiz is not limited to the described embodiments, but may be produced in various ways.

Since the user views only objects output through the HMD, the user may be less affected by external environments other than the content. In addition, since the content is produced in the first-person view, the user's feeling and immersion are improved.

Referring back to FIG. 3, in step 320, the server 200 acquires brain wave information and gaze information of the user. For example, the server 200 may receive brain wave information and gaze information generated by the user device from the user device. A method of obtaining brain wave information and gaze information of a user will be described in detail with reference to FIGS. 5 to 8 below.

5 is a flowchart illustrating a method of acquiring EEG information and gaze information based on a change of an object according to an example. According to one aspect, step 320 may include steps 510 and 520 below.

In operation 510, the server 200 may change an object in the content based on the EEG information and the gaze information. When the acquired EEG information and the gaze information satisfy the conditions preset for the content, the object may change according to the degree. A method of changing an object is described in detail with reference to FIG. 6 below.

According to another aspect, the content may be provided such that an object corresponding to the user's gaze is located at a preset position of the content (for example, the center of the screen of the outputted content). That is, when there are a plurality of objects output on the screen, and the user focuses the gaze on the first object of the plurality of objects, the screen of the content may be modified so that the first object is output at the center of the screen. That is, the screen of the content may be rendered differently according to the eyes of the user.

In operation 520, the server 200 stops providing content when the change of the object corresponds to a preset change. For example, when the object moves to a preset position, the content may be terminated. The faster and more intensely focused the user is, the faster content can be terminated. The presentation time of the content may be used to determine the concentration level of the user.

Although not shown, step 320 may stop providing the content when the preset timer for the content expires even if the change of the object does not correspond to the preset change. That is, the content may be terminated when the goal of the content is not reached within the time.

6 is a flowchart of a method of changing an object in content based on eye gaze information according to an example. According to one aspect, step 510 may include steps 610 and 620 below.

In operation 610, the server 200 determines whether the user's gaze corresponds to an object among a plurality of objects of the content based on the gaze information. In the example of FIG. 4, it may be determined whether a user's gaze corresponds to the second object 412 of the plurality of objects 411 and 412. A method of determining whether a user's gaze corresponds to an object among a plurality of objects of content based on the gaze information will be described in detail with reference to FIG. 7 below.

In operation 620, the server 200 changes the object when the gaze corresponds to the object. For example, the object may be changed when the gaze corresponds to the object and it is determined that the user concentrates based on the brain wave information.

7 illustrates gaze information of a user who views content according to an example.

For example, the gaze information 710 may be a gaze plot or heat map of a user who views content. The gaze plot may indicate the position of the gaze stayed at a specific position of the content, repair, time, and the like. Time can be represented by the size of a circle. The heat map may indicate how the gaze of the content is distributed.

For example, when the user focuses the gaze on the target object 712 of the plurality of objects, the records 713 may be concentrated around the target object 712. When the concentration of the records 713 is equal to or greater than a preset value, the gaze of the user may be determined to correspond to the target object 712.

8 illustrates divided EEG signals of a plurality of frequency bands based on EEG information and EEG information according to an example.

The EEG information 810 generated by the EEG sensor of the user device may be an analog signal. In order to extract the features of the EEG, processing of the EEG information 810 is required. For example, the EEG signals 820 may be divided into a plurality of frequency bands by Fourier transforming the EEG information 810.

An EEG characteristic can be expressed by the magnitude of the component (or power) that appears at a particular frequency. The frequency band of 1 to 4 Hz is of delta type and may be a characteristic of sleeping, repairing, and complex problem solving. The frequency band of 4 to 8 Hz is theta type and may be a feature that appears in creativity, insight, and deep states. The frequency band of 8 to 12 Hz is of alpha type and may be a feature that appears during meditation. The frequency band of 13 to 21 Hz is a beta (β) type and may be a feature that appears when thinning and focusing. The frequency band of 12-15 Hz is of SMR type and may be a characteristic manifested in mental alertness, physical relaxation. The frequency band of 20 to 32 Hz is a gamma (γ) type and may be a feature that appears during cognitive processing and learning.

According to an embodiment, the EEG concentration index may be calculated by Equation 1 below.

[Equation 1]

In Equation 1, T is an concentration indicator, SMR may be the size of the component for the SMR type of the EEG signals 820, β may be the size of the component for the beta type, the size of the component for theta type .

The concentration index for providing the content to the user may be calculated in real time, and the object in the content may change according to the concentration index. For example, when the concentration indicator is large, the object may change greatly. When the concentration indicator is greater than or equal to a preset value, the goal of the content is achieved, and the content may end.

9 is a flowchart illustrating a method of providing second content to a user and obtaining second brain wave and second gaze information of the user, according to an exemplary embodiment.

According to one aspect, steps 910 and 920 below may be further performed after step 320 described above with reference to FIG. 3 is performed.

In operation 910, when the provision of the first content is stopped (or terminated), the server 200 provides the second content to the user. For example, the second content may be the same type as the first content. As another example, the second content may be of a different type from the first content. The first content and the second content may be contents described with reference to FIG. 4, but are not limited to the described embodiment.

In operation 920, the server 200 acquires second brain wave information and second gaze information of the user.

Although not shown, according to an example, additional contents may be continuously provided even when the provision of the second contents is stopped. The required number of contents may be provided to determine the level of concentration of the user.

Referring back to FIG. 3, in step 330, the server 200 determines the concentration level of the user based on the EEG information and the gaze information. A method of determining the concentration level of the user based on the EEG information and the gaze information will be described in detail with reference to FIGS. 10 to 14 below.

According to an aspect, any one level among a plurality of preset levels may be determined. For example, the plurality of levels may include severe ADHD patient level, mild ADHD patient level, attention level, and normal level, and are not limited to the described embodiments.

The current state of the user may be determined according to the determined concentration level, and when the user repeatedly uses the content, the concentration level may be improved in a good direction. In other words, the user can train the concentration through the provided content.

10 is a flowchart of a method of calculating a concentration level based on brain wave signals according to an example.

According to one aspect, step 330 described above with reference to FIG. 3 may include steps 1010 and 1020 below.

In step 1010, the server 200 divides the EEG information into the EEG signals of the plurality of frequency bands. For example, as described with reference to FIG. 8, the EEG signals 820 may be divided into a plurality of frequency bands by Fourier transforming the EEG information 810.

In operation 1020, the server 200 calculates a concentration level based on the target EEG signal among the EEG signals. For example, the target EEG signal may include an SMR type, a beta type, and theta type among the EEG signals 820 described above with reference to FIG. 8.

According to an aspect, any one level among a plurality of preset levels may be determined based on the target EEG signal. For example, the plurality of levels may include severe ADHD patient level, mild ADHD patient level, attention level, and normal level, and are not limited to the described embodiments.

According to one embodiment, the concentration level may be calculated through a neural network or algorithm that has been previously trained to calculate the concentration level. For example, a Neural Network with Weighted Fuzzy Membership Function (NEWFM) may be used to calculate the concentration level. The NEWFM can calculate the concentration level as an output by using at least one of the first brain wave information and the first gaze information as an input. When NEWFM receives at least one of the first EEG information and the first gaze information, the result of classifying the Takagi-Sugeno defuzzification values, the class of Takagi-Sugeno inverse fuzzy values, and The concentration level can be calculated by outputting the weighted fuzzy membership function.

The class classification method through NEWFM and the structure of NEWFM are described in detail with reference to FIGS. 11 to 14 below.

11 is a flowchart illustrating a class classification method using a weighted fuzzy membership function based deep neural network, according to an embodiment.

According to one aspect, step 1020 described above with reference to FIG. 10 may include steps 1110 to 1160 below. That is, step 1020 may be performed by performing steps 1110 to 1160.

In step 1110, the server 200 inputs input data into the NEWFM. For example, the input data can include target EEG signals. In addition, the input data may further include gaze information.

NEWFM receives n input data, classifies Takagi-Sugeno defuzzification values, Takagi-Sugeno inverse fuzzy values for the input data, and weighted fuzzy membership function. Outputs For example, the input data of the I ₁ to I ₁₀ a is input to the NEWFM, of I ₁ to I ₁₀ classified in the output of NEWFM class, Takagi for I ₁ to I ₁₀ - can halogeno station fuzzy values and weighting Output the fuzzy membership function.

In operation 1120, the server 200 classifies the output data into a plurality of groups.

The server 200 compares the class of records of the input data with the classes of the records of the output data output as a result of step 1110, and classifies the output data into a plurality of groups. The output data may be classified into the first and second groups according to whether the class classified for the output data, that is, the output Takagi-Sugeno fuzzy values matches the class of records of the corresponding input data. Here, the output data are the Takagi-Sugeno inverse fuzzy values corresponding to the input data, and the class of the records of the output data is the Takagi-Sugeno inverse fuzzy output based on the records of the input data in the learning process of NEWFM. The result of classifying a class of values. For example, if the class of records of I ₁ and I ₁₀ of the input data I ₁ to I ₁₀ is different from the class of records of output data corresponding to I ₁ and I ₁₀ of the output data, I ₁ and I _10. Are classified into a first group, and I ₂ to I ₉ may be classified into a second group. That is, the first group is a group different from the class of each record of the input data to which the class of the record corresponds among the classes of records of the output data, and the second group is the input data to which the class of the record among the records of the output data corresponds. Can be the same group as the class of the record.

In operation 1130, the server 200 inputs output data included in the plurality of groups into the NEWFM. Since output data included in the plurality of groups is inputted to the NEWFMs for each group, the weighted fuzzy membership functions corresponding to each group may be output.

The NEWFM to which the output data included in the first and second groups are input in step 1130 is the same as the NEWFM to which the input data are input in step 1110, but the data input to the NEWFM is the input data and the output data. Because the results are different in groups, the output results are not the same.

In step 1140, server 200 calculates the accuracy of class classification of records of input data.

As a result of step 1130, the weighted fuzzy membership functions for each of the first and second groups are output, and by matching the input data of step 1110 with the output weighted fuzzy membership functions, the class for the input data. Can be classified.

By comparing the classes of records of the sorted input data with the actual classes of the input data, the accuracy of the classes for the records of the sorted input data can be calculated. For example, a result of comparing classes of records of input data I ₁ to I ₁₀ with classes of records classified for I ₁ to I ₁₀ output as a result of step 1130 may be calculated with a first accuracy. Can be.

In operation 1150, the server 200 outputs the weighted fuzzy membership functions output from each of the input data and the NEWFMs corresponding to the plurality of groups. That is, after step 1140, the server 200 performs the weighted fuzzy membership function output from the NEWFM according to the input in step 1110 and the weighted fuzzy output from each of the NEWFMs corresponding to the groups in step 1130. Print membership functions. For example, each weighted fuzzy membership function output may be used as a step function when new data is input to classify the class of records of the new data with high accuracy.

When step 1150 is repeatedly performed, if the accuracy calculated as a result of step 1150, that is, the last calculated accuracy is equal to the last calculated accuracy, the last calculated accuracy is applied to the input data. This final accuracy can be output, since

In operation 1160, when the accuracy calculated in operation 1150 satisfies a preset condition, the server 200 defines output data output as a result of operation 1130 as input data. Steps 1110 to 1150 may be repeatedly performed by inputting the defined input data back to step 1110.

Step 1160 is to increase the accuracy of class classification of records of the input data, and may be repeatedly performed until the accuracy calculated in step 1140 is the highest.

Step 1160 is the accuracy of which the accuracy has not been calculated before calculating the accuracy in step 1140, or the accuracy of the class classification of the records of the input data calculated as a result of step 1150 immediately before the calculation. If less than. For example, if the first order accuracy for class classification of the records of the input data I ₁ to I ₁₀ is calculated through step 1140, since the accuracy has not been calculated before calculating the first order accuracy, Steps 1110 to 1150 may be performed by treating the Takagi-Sugeno inverse purge values calculated as a result of step 1130 as input data.

In addition, even if the second degree of accuracy calculated through step 1160 exceeds the first degree of accuracy, it is not known whether the calculated second degree of accuracy is the highest accuracy. 1110 to 1150 may be performed again.

If the calculated third degree of accuracy is the same as the second degree of accuracy, the second and third degree of accuracy are regarded as the highest accuracy, and the third degree of accuracy is the final accuracy of class classification for the records of the input data. Can be output as

For example, when the weighted fuzzy membership functions are extracted using the EEG information (more specifically, target EEG signals) of the actual ADHD patient as input data of the class classification method using NEWFM, the extracted weighted fuzzy membership function is extracted. Since these are functions having high accuracy with respect to EEG information for ADHD patients, they can be used as functions for classifying classes of EEG information for ADHD patients whose classes are not defined.

12 and 14 illustrate a weighted fuzzy membership function based deep neural network according to an example.

Specifically, FIG. 12 briefly illustrates an overall flowchart of a class classification method using NEWFM according to an example, FIG. 13 illustrates a flowchart of first steps 1110 to 1130, and FIG. 14 illustrates step 1140. Shows a flow chart of.

Table 1 below shows the values and class of input data 1201, Table 2 shows the class for Takagi-Sugeno inverse fuzzy values output via NEWFM 1203, and Table 3 ] Shows the Takagi-Sugeno inverse purge values included in the first group 1205, and [Table 4] shows the Takagi-Sugeno inverse purge values included in the second group 1207.

input 1 input 2 ... class One 11 One 2 12 One 3 13 2 4 14 One 5 15 2 6 16 2 7 17 One 8 18 One 9 19 One 10 20 2

One 2 One One 2 One One 2 One One

input 1 input 2 ... class 2 12 One 3 13 2 6 16 2 8 18 One 10 20 2

input 1 input 2 ... class One 11 One 4 14 One 5 15 2 7 17 One 9 19 One

12 to 14 and Tables 1 to 4, input data 1201 is input to the NEWFM 1203.

The NEWFM 1203 is a weighted class classification result of the Takagi-Sugeno inverse fuzzy values, Takagi-Sugeno inverse fuzzy values corresponding to the input data 1201 and [Table 1], and the weighting. Output fuzzy membership function.

As a result of comparing the class classification result by record ([Table 2]) and the class of the input data 1201 and [Table 1], it was classified into a class different from the class for the input data 1201 and [Table 1]. Takagi-Sugeno inverse fuzzy values are classified into a first group 1205 and [Table 3] and classified into a class different from the class for input data 1201 and [Table 1]. Fuzzy values are classified into a second group 1207 and Table 4.

For example, since the actual class of the first record of input data 1201 and [Table 1] and the classified class of the first record of Takagi-Sugeno inverse fuzzy values are the same, the Takagi-Sugeno inverse fuzzy The first record of values may be classified into a second group 1207 and Table 4.

However, since the actual class of the second record of the input data 1201 and [Table 1] differs from the classified class of the second record of the Takagi-Sugeno inverse fuzzy values, the Taggi-Sugeno inverse fuzzy values of The second record may be classified into the first group 1207 and Table 4.

By inputting the Takagi-Sugeno inverse purge values included in the first group 1205 and Table 3 and the second group 1207 and Table 4 into the NEWFM 1203 for each group, the first group ( 1205 and the weighted fuzzy membership function corresponding to [Table 3] and the weighted fuzzy membership function corresponding to the second group 1207 and [Table 4] may be output.

The input data 1201 and [Table 1] are assigned to the weighted fuzzy membership function corresponding to the first group 1205 and [Table 3] and the weighted fuzzy membership function corresponding to the second group 1207 and [Table 4]. By substituting for each, the accuracy for class classification of the input data 1201 and [Table 1] can be calculated.

When the calculated accuracy satisfies the preset condition, steps 1110 to 1150 may be repeatedly performed through step 1160.

When the steps 1110 to 1150 are repeated, the Takagi-Sugeno inverse purge values included in the first group 1205 and Table 3 and the second group 1207 and Table 4 are each grouped. The Takagi-Sugeno inverse purge values output as a result of inputting to NEWFM 1203 may be treated as input data 1201 and [Table 1].

15 is a block diagram illustrating a user device according to one embodiment.

According to an aspect, the user device 1500 may include a communication unit 1510, a processor 1520, a memory 1530, an EEG sensor 1540, and a camera 1550. The user device 1500 may correspond to the user device 120 described with reference to FIG. 1. For example, the user device 1500 may be an HMD.

The communication unit 1510 is connected to the processor 1520, the memory 1530, the EEG sensor 1540, and the camera 1550 to transmit and receive data. The communication unit 1510 may be connected to another external device to transmit and receive data.

The communicator 1510 may be implemented as a circuit in the user device 1500. For example, the communication unit 1510 may include an internal bus and an external bus. As another example, the communication unit 1510 may be an element connecting the user device 1500 and an external device. The communication unit 1510 may be an interface. The communication unit 1510 may receive data from an external device and transmit data to the processor 1520 and the memory 1530.

The processor 1520 processes the data received by the communication unit 1510 and data stored in the memory 1530. The processor 1520 executes instructions dictated by the processor 1520 and computer readable code (eg, software) stored in the memory (eg, memory 1530).

The memory 1530 stores data received by the communication unit 1510 and data processed by the processor 1520. For example, the memory 1530 may store a program (or an application or software). The stored program may be a set of syntaxes that are coded to provide content to a user and obtain brain wave information and gaze information, and may be executed by the processor 1520.

According to one aspect, memory 1530 may include one or more volatile memory, nonvolatile memory and RAM, flash memory, hard disk drive, and optical disk drive.

The memory 1530 stores a set of instructions (eg, software) for operating the user device 1500. The instruction set for operating the user device 1500 is executed by the processor 1520.

The EEG sensor 1540 may be located in the user device 1500 to acquire the EEG information of the user when the user device 1500 is in close contact with the head of the user. In detail, when the user wears the user device 1500, the EEG sensor 1540 may be disposed on the user device 1500 to be positioned at the user's forehead.

The camera 1550 may be an imaging device capable of capturing an eye of a user. The position of the pupil of the user may be determined based on the image generated by the camera 1550. The gaze of the user may be determined based on the position of the pupil of the user. When the user wears the user device 1500, the user camera 1550 may be disposed on the user device 1500 to capture the eyes of the user.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable arrays (FPAs), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. Or may be permanently or temporarily embodied in a signal wave to be transmitted. The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

Method according to the embodiment is implemented in the form of program instructions that can be executed by various computer means may be recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine code, such as produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different manner than the described method, or other components. Or even if replaced or replaced by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims which follow.

100: system
110: server
120: user device
130: analysis terminal

Claims (16)

Providing first content to a user via a head mounted display (HMD);
Acquiring at least one of first brain wave information and first gaze information of the user who views the first content through the HMD; And
Determining a concentration level of the user based on at least one of the first EEG information and the first gaze information;
Including,
How to determine your level of concentration.
The method of claim 1,
Acquiring at least one of first brain wave information and first gaze information of the user who views the first content through the HMD,
Changing an object in the first content based on at least one of the first brain wave information and the first gaze information; And
Stopping providing the first content when the change of the object corresponds to a preset change;
Including,
How to determine your level of concentration.
The method of claim 2,
The change of the object,
At least one of the shape of the object and the position of the object is changed,
How to determine your level of concentration.
The method of claim 2,
Changing the object in the first content,
Determining whether the gaze of the user corresponds to the object among a plurality of objects of the first content based on the first gaze information; And
Changing the object when the gaze corresponds to the object
Including,
How to determine your level of concentration.
The method of claim 2.
Acquiring at least one of brain wave information and gaze information of the user who views the first content through the HMD,
Stopping providing of the first content when the timer set for the first content expires even if the change of the object does not correspond to a preset change;
Further comprising,
How to determine your level of concentration.
The method of claim 2,
Providing second content to the user when the first content is stopped; And
Acquiring at least one of second brain wave information and second gaze information of the user who views the second content through the HMD;
More,
Calculating the concentration level of the user,
Calculating the concentration level based on at least one of the second brain wave information and the second gaze information of the user.
Including,
How to determine your level of concentration.
The method of claim 1,
Calculating the concentration level of the user,
Dividing the first EEG information into EEG signals of a plurality of frequency bands; And
Calculating the concentration level based on a target EEG signal among the EEG signals
Including,
How to determine your level of concentration.
The method of claim 1,
Calculating the concentration level of the user,
Calculating the concentration level using a Neural Network with Weighted Fuzzy Membership Function (NEWFM) based on at least one of the first EEG information and the first gaze information;
Including,
How to determine your level of concentration.
The method of claim 8,
The NEWFM classifies a class of Takagi-Sugeno defuzzification values and Takagi-Sugeno inverse fuzzy values when receiving at least one of the first EEG information and the first gaze information. Calculating the concentration level by outputting a result and a weighted fuzzy membership function,
How to determine your level of concentration.
A computer-readable recording medium containing a program for performing the method of any one of claims 1 to 9.
The server that determines the level of concentration of the user,
A memory in which a program for determining a concentration level of a user is recorded; And
A processor that executes the program
Including,
The program,
Providing first content to a user via a head mounted display (HMD);
Acquiring at least one of first brain wave information and first gaze information of the user who views the first content through the HMD;
Determining a concentration level of the user based on at least one of the first EEG information and the first gaze information;
To do,
server.
The method of claim 11,
Acquiring at least one of first brain wave information and first gaze information of the user who views the first content through the HMD,
Changing an object in the first content based on at least one of the first brain wave information and the first gaze information; And
Stopping providing the first content when the change of the object corresponds to a preset change;
Including,
server.
The method of claim 12,
Changing the object in the first content,
Determining whether the gaze of the user corresponds to the object among a plurality of objects of the first content based on the first gaze information; And
Changing the object when the gaze corresponds to the object
Including,
How to determine your level of concentration.
The method of claim 11,
Calculating the concentration level of the user,
Dividing the first EEG information into EEG signals of a plurality of frequency bands; And
Calculating the concentration level based on a target EEG signal among the EEG signals
Including,
How to determine your level of concentration.
The method of claim 11,
Calculating the concentration level of the user,
Calculating the concentration level using a Neural Network with Weighted Fuzzy Membership Function (NEWFM) based on at least one of the first EEG information and the first gaze information;
Including,
How to determine your level of concentration.
The method of claim 15,
The NEWFM classifies a class of Takagi-Sugeno defuzzification values and Takagi-Sugeno inverse fuzzy values when receiving at least one of the first EEG information and the first gaze information. Calculating the concentration level by outputting a result and a weighted fuzzy membership function,
How to determine your level of concentration.

Images