KR20220047187A - Server and method for cognitive function testing using feature combination - Google Patents

Abstract

The present invention relates to a method for testing a cognitive function using a feature combination, wherein the method is configured to comprise: a step of providing a plurality of consecutive event sequences through an HMD device; a step of obtaining, from the HMD device, at least one time stamp for each of the plurality of consecutive event sequences; a step of obtaining, from the HMD device, a biosignal corresponding to the plurality of consecutive event sequences; a step of recording the at least one time stamp on the biosignal; and a step of generating a cognitive function evaluation result of a user based on the biosignal according to the time stamp. Therefore, the present invention is capable of allowing changes in the biosignal of the user to be obtained more precisely.

Description

Cognitive function test method and server using feature combination

The present invention relates to a method and server for cognitive function testing using feature combination.

As the proportion of the elderly population continues to increase as we enter an aging society, interest in geriatric diseases such as dementia, which is more likely to develop as we enter old age, is increasing.

Dementia has various and complex symptoms and causes, and there is no clearly defined treatment. Early diagnosis of dementia is the best method, but due to the lack of professional manpower, there is a limit to periodic examination and early diagnosis of dementia by all users of a certain age.

Accordingly, a digital content-based medical service technology that has less burden on medical expenses and makes it easier to secure professional manpower is emerging. Digital content refers to VR, AR, and MR images that enable users to experience spatial and temporal similarity to reality, and digital content-based medical services use digital equipment (HMD devices) that output these images. It refers to a service that can perform capability diagnosis, screening, and characteristic analysis of disease groups.

In particular, in recent years, medical research and services for using the collected bio-signals by attaching a user's bio-signal collecting equipment to a digital device (HMD device) providing digital content are also increasing.

The description underlying the invention has been prepared to facilitate understanding of the invention. It should not be construed as an admission that the matters described in the background technology of the invention exist as prior art.

In the conventional medical research and service, after a series of digital contents are provided, all of the user's bio-signals are collected and analysis is performed based on them, so the accuracy is lowered even though the bio-signals are analyzed.

Accordingly, there is a need for a method for accurately examining the user's cognitive function by precisely analyzing the user's bio-signals.

As a result, the inventors of the present invention have attempted to develop a method for synchronizing various event sequences provided through the HMD device with time stamps of a user's bio-signals and a server for performing the same.

In this case, the inventors of the present invention have configured a method capable of distinguishing between a plurality of event sequences and providing times of events included in the plurality of event sequences while a cognitive function test is in progress, and synchronizing them with biosignals.

The problems of the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above-described problems, there is provided a cognitive function test method according to an embodiment of the present invention. The method includes providing a plurality of consecutive sequences of events through an HMD device, obtaining from the HMD device at least one timestamp for each of the plurality of consecutive sequences of events, and from the HMD device the plurality of sequences of events. acquiring a bio-signal corresponding to a continuous sequence of events, recording the at least one time stamp in the bio-signal, and generating a user's cognitive function evaluation result based on the bio-signal according to the time stamp is configured to include

According to a feature of the present invention, the obtaining of the at least one time stamp may further include obtaining a time stamp for a sub-event pre-stored in each of the plurality of consecutive event sequences.

According to another feature of the present invention, the pre-stored sub-event provides at least one of guide text for testing a cognitive function, guide audio, a graphic object positioned in a 3D space as a background, motion of the graphic object, and sound effect can correspond to

According to another aspect of the present invention, obtaining a time stamp for input data corresponding to the plurality of consecutive event sequences from an input device connected to the HMD device, and adding a time stamp for the input data to the biosignal It may further include the step of recording.

According to another feature of the present invention, generating an evaluation result for any one event sequence among the plurality of continuous event sequences based on the input data, and the pre-stored sub-event or next based on the evaluation result The method may further include determining whether to provide an event sequence.

According to another feature of the present invention, the biosignal may be obtained from at least one of a heartbeat sensor, an electrocardiogram sensor, an electroencephalogram sensor, an electromyogram sensor, a temperature sensor, and a camera sensor connected to the HMD device.

According to another feature of the present invention, the generating of the cognitive function evaluation result comprises dividing the EEG signal obtained from the EEG sensor based on the time stamp, and in the divided EEG signal, PSD (Power Spectral Power Spectral) for each frequency band. Density) may be a step of generating a user's cognitive function evaluation result.

According to another feature of the present invention, the generating of the cognitive function evaluation result comprises classifying a change in the user's pupil movement speed or size obtained from the camera sensor based on the time stamp, and based on the divided change value may be a step of generating a user's cognitive function evaluation result.

According to another feature of the present invention, before the step of providing the plurality of event sequences, time synchronization of the biosignals obtained from the at least two or more sensors is performed using at least two or more synchronization sensors related to the biosignals. It may further include the step of performing.

According to another feature of the present invention, the synchronization sensor is disposed adjacent to a sensor for acquiring the biosignal, and may include at least one of a motion sensor, an illuminance sensor, an optical sensor, and a sound wave sensor.

According to another feature of the present invention, the HMD device includes a first sensor for acquiring the biosignal, a second sensor, a first synchronization sensor disposed adjacent to the first sensor, and adjacent to the second sensor and a second synchronization sensor disposed thereon.

According to another aspect of the present invention, the performing of the time synchronization includes calculating time difference information for the first and second synchronization sensing signals and the first biosignal based on the calculated time difference information. and performing time synchronization with respect to the second biosignal.

According to another feature of the present invention, any one of the first bio-signal obtained through the first sensor and the second bio-signal obtained through the second sensor may include an image signal.

According to another feature of the present invention, the calculating of the time difference information includes: identifying characteristic information for each of the first and second synchronization sensing signals; and calculating the time difference information based on the identified characteristic information. It may further include the step of calculating. Cognitive function test method using feature combination comprising

According to another feature of the present invention, the characteristic information may include at least one of a peak value of a synchronization sensing signal and a preset threshold value.

According to another feature of the present invention, the sequence of events includes the user's short-term memory, long-term memory, reading ability, concentration, depth perception, and reflexes together with a sequence configured to acquire the biosignal of the user's resting state. , at least one of Visual perception, Spatial perception, Working memory, Focused attention, Sustained attention, Reading comprehension, and Selective attention. It may include sequences configured to test capabilities.

In order to solve the above problems, there is provided a cognitive function test server according to another embodiment of the present invention. The server includes a communication interface, a memory, and a processor operatively coupled to the communication interface and the memory, the processor to provide a sequence of a plurality of consecutive events through an HMD device, and to provide a sequence of a plurality of events from the HMD device. obtaining at least one time stamp for each successive sequence of events, obtaining a biosignal corresponding to the plurality of successive event sequences from the HMD device, and recording the at least one time stamp in the biosignal; and generate a user's cognitive function evaluation result based on the biosignal according to the time stamp.

In order to solve the above problems, there is provided a cognitive function test method using feature combination according to another embodiment of the present invention. The method includes: providing a plurality of consecutive event sequences through a display screen; obtaining at least one time stamp for each of the plurality of consecutive event sequences; Acquiring a signal, recording the at least one time stamp in the bio-signal, and providing a result of evaluating a user's cognitive function based on the bio-signal according to the time stamp through the display screen do.

In order to solve the above problems, an HMD device according to another embodiment of the present invention is provided. The apparatus comprises a communication interface, a memory, a display, and a processor operatively coupled to the communication interface, the memory and the display, the processor to provide a continuous sequence of a plurality of events through a screen of the display; obtaining at least one time stamp for each of the plurality of successive event sequences, obtaining a biosignal corresponding to the plurality of successive event sequences, recording the at least one time stamp in the biosignal, and and providing a result of evaluating the user's cognitive function based on the biosignal according to the time stamp through the screen of the display.

Specific details of other embodiments are included in the detailed description and drawings.

According to the present invention, a series of biosignals can be classified based on the event sequence by recording a time stamp for an event provision time in biosignals obtained from a user while a plurality of event sequences are provided.

In addition, according to the present invention, by recording the events (eg, stimuli, features) that are provided in detail in the event sequence configured to test the cognitive function, in the bio-signals, it is possible to obtain more detailed changes in the user's bio-signals. .

In addition, according to the present invention, it is possible to perform a professional cognitive function test as diagnosed by an expert through an HMD device through a detailed analysis of changes in biosignals.

In addition, the present invention performs a cognitive function test using various biological signals such as pupil, brain wave, heartbeat, electromyography, electroencephalogram, body temperature, etc., so that a disease with an unclear cause, such as dementia, can be more easily diagnosed.

In addition, according to the present invention, since time synchronization between sensors for acquiring biosignals and an event sequence is accurately performed in the HMD device, a time error in milliseconds between a plurality of acquired biosignals and a timing of providing the event sequence can be corrected. In particular, the present invention can improve the reliability of the cognitive function test using the event sequence through error correction.

The effect according to the present invention is not limited by the contents exemplified above, and more various effects are included in the present invention.

1 is a schematic diagram of a cognitive function test system using feature combination according to an embodiment of the present invention.
2 is a block diagram of an HMD device according to an embodiment of the present invention.
3 is a flowchart of a cognitive function test method using an HMD device according to an embodiment of the present invention.
4 is a block diagram of a cognitive function test server according to an embodiment of the present invention.
5 is a flowchart of a cognitive function test method using a cognitive function test server according to an embodiment of the present invention.
6 and 7 are schematic diagrams for explaining a method of matching a feature of a sequence provided by an HMD device with a biosignal according to an embodiment of the present invention.
8A to 8F are schematic diagrams for explaining a sequence interface provided in an HMD device according to an embodiment of the present invention.
9 and 10 are graphs for explaining that the accuracy of the cognitive function test is increased through feature combination of sequences provided by the HMD device according to an embodiment of the present invention.

Advantages and features of the present invention, and a method for achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in a variety of different forms, only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the technical field to which the present invention pertains It is provided to fully inform the possessor of the scope of the invention, and the present invention is only defined by the scope of the claims. In connection with the description of the drawings, like reference numerals may be used for like components.

In this document, expressions such as "have," "may have," "includes," or "may include" refer to the presence of a corresponding characteristic (eg, a numerical value, function, operation, or component such as a part). and does not exclude the presence of additional features.

In this document, expressions such as “A or B,” “at least one of A or/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, "A or B," "at least one of A and B," or "at least one of A or B" means (1) includes at least one A, (2) includes at least one B; Or (3) it may refer to all cases including both at least one A and at least one B.

As used herein, expressions such as "first," "second," "first," or "second," may modify various elements, regardless of order and/or importance, and refer to one element. It is used only to distinguish it from other components, and does not limit the components. For example, the first user equipment and the second user equipment may represent different user equipment regardless of order or importance. For example, without departing from the scope of rights described in this document, the first component may be named as the second component, and similarly, the second component may also be renamed as the first component.

A component (eg, a first component) is "coupled with/to (operatively or communicatively)" to another component (eg, a second component); When referring to "connected to", it will be understood that the certain element may be directly connected to the other element or may be connected through another element (eg, a third element). On the other hand, when it is said that a component (eg, a first component) is "directly connected" or "directly connected" to another component (eg, a second component), the component and the It may be understood that other components (eg, a third component) do not exist between other components.

The expression "configured to (or configured to)" as used in this document, depending on the context, for example, "suitable for," "having the capacity to ," "designed to," "adapted to," "made to," or "capable of." The term “configured (or configured to)” may not necessarily mean only “specifically designed to” in hardware. Instead, in some circumstances, the expression “a device configured to” may mean that the device is “capable of” with other devices or parts. For example, the phrase "a processor configured (or configured to perform) A, B, and C" refers to a dedicated processor (eg, an embedded processor) for performing the corresponding operations, or by executing one or more software programs stored in a memory device. , may mean a generic-purpose processor (eg, a CPU or an application processor) capable of performing corresponding operations.

Terms used in this document are only used to describe specific embodiments, and may not be intended to limit the scope of other embodiments. The singular expression may include the plural expression unless the context clearly dictates otherwise. Terms used herein, including technical or scientific terms, may have the same meanings as commonly understood by one of ordinary skill in the art described in this document. Among terms used in this document, terms defined in a general dictionary may be interpreted with the same or similar meaning as the meaning in the context of the related art, and unless explicitly defined in this document, ideal or excessively formal meanings is not interpreted as In some cases, even terms defined in this document cannot be construed to exclude embodiments of this document.

Each feature of the various embodiments of the present invention may be partially or wholly combined or combined with each other, and as those skilled in the art will fully understand, technically various interlocking and driving are possible, and each embodiment may be implemented independently of each other. It may be possible to implement together in a related relationship.

1 is a schematic diagram of a cognitive function test system using feature combination according to an embodiment of the present invention.

Referring to FIG. 1 , the cognitive function testing system 1000 may be a system capable of combining a plurality of continuous event sequences provided to a user and a biosignal based on a feature, and then examining the user's cognitive function through this.

In the present invention, a feature may be understood as a generic term for all kinds of events given to a user in order to evaluate various kinds of stimuli given to the user and cognitive functions.

The cognitive function test system 1000 combines the HMD device 100 mounted on the user's head and various signals provided from the HMD device 100 based on the features, and the cognitive function test server 200 checks the cognitive function. and a manager device 300 that determines a plurality of consecutive event sequences to be provided to the HMD device 100 .

The HMD device 100 may be mounted on a user's head, and may provide a spatial and temporal experience similar to reality to the user. The HMD device 100 may be a complex virtual experience device capable of detecting physical, cognitive, and emotional changes of a user who is undergoing a virtual experience.

The HMD device 100 may receive and output a plurality of continuous event sequences for testing a cognitive function, and simultaneously acquire biosignals such as a user's gaze, brain waves, and heart rate. According to an embodiment, the input device 105 and at least one sensor 120 may be connected to the HMD device 100 .

Meanwhile, in the present invention, an event sequence may be understood as a task for examining a cognitive function. According to embodiments, the sequence of events may include a sequence configured to cause the user to meditate, the user's short-term memory, long-term memory, reading ability, concentration, depth perception, agility, visual perception, spatial perception ( It may include tasks to test spatial perception, working memory, focused attention, sustained attention, reading comprehension and selective attention.

In addition, the event sequence may include images, audio, video, vibration, motion, etc., output through the HMD device 100 and an output device connected thereto, and may include various types of content and images that a user can recognize.

The input device 105 may acquire input data by the user's left hand and right hand. Specifically, the input device 105 may acquire input data in response to a user's response to an event sequence provided by the HMD device 100 .

The sensor 120 may measure changes in various biosignals of the user with respect to a plurality of consecutive event sequences output from the HMD device 100 . According to an embodiment, the sensor 120 may include a sensor for acquiring various signals and a synchronization sensor for time synchronization between various signals, a more detailed description will be provided later.

The HMD device 100 is implemented in a form that can process virtual, augmented, and mixed (VR, AR, MR) images on the display 150 inside the device, or provides the above-described image to a part of the HMD device 100 . A separate output device (not shown) may be mounted, and an image may be processed through it. For example, the HMD device 100 receives and outputs an event sequence for testing a cognitive function, and transmits various data acquired in relation to the output event sequence to the manager device 300 or the cognitive function testing server 200 . can send

According to an embodiment, the HMD device 100 may provide an event sequence for testing the user's cognitive functions, such as short-term memory, long-term memory, reading ability, concentration, depth perception, and reflexes of the user. The HMD device 100 obtains input data based on an event sequence and a series of biosignals (eg, EEG signals) from the user, and outputs a cognitive function evaluation result determined based on this from the cognitive function test server 200 . can

Meanwhile, when the user detaches the device, the HMD device 100 may transmit a device detachment signal to the cognitive function test server 200 . If the same or different user wants to perform a cognitive function test after the device detachment signal, the HMD device 100 receives and outputs a sequence for acquiring the user's biosignal in a resting state from the cognitive function test server 200 . can do.

The cognitive function test server 200 may generate a cognitive evaluation result by combining an event sequence and a biosignal based on a feature, and performing a cognitive function test. To this end, the cognitive function test server 200 may be implemented as a general-purpose computer, a laptop, and a cloud server.

According to an embodiment, the cognitive function test server 200 receives input data and bio-signals from the HMD device 100 or the manager device 300, and evaluates the cognitive function test based on the received input data and bio-signals. results can be generated. Thereafter, the cognitive function test server 200 may provide the generated evaluation result data to the HMD device 100 or the manager device 300 .

The manager device 300 may provide a user interface for supporting a user's cognitive function test. The manager device 300 may be implemented as a smart phone, a tablet PC, a PC, etc. capable of outputting an interface screen.

The manager device 300 may be connected to the HMD device 100 to exchange various data for a cognitive function test. The manager device 300 may support device connection, communication connection, tutorial progress, cognitive function test and analysis result, etc. with the HMD device 100 while sharing the screen of the HMD device 100 worn by the user.

Hereinafter, with reference to FIG. 2 , the HMD device 100 mounted on the user's head in order to test the cognitive function will be described in more detail.

Meanwhile, the HMD device 100 described in the present invention may be a stand-alone type in which the HMD device 100 is connected to sensors alone. Also, the HMD device 100 may be of a type in which only a sensing module is provided and a separate image output device is connected to the HMD device.

2 is a block diagram of an HMD device according to an embodiment of the present invention.

Referring to FIG. 2 , the HMD device 100 may include a communication interface 110 , a sensor 120 , a memory 130 , a processor 140 , and a display 150 .

The communication interface 110 may connect the HMD device 100 to communicate with an external device. Specifically, the communication interface 110 may receive an image from the outside or transmit/receive data related to sensing through wired/wireless communication. Here, wireless communication is WiFi (wireless fidelity), Bluetooth, Bluetooth low energy (BLE), Zigbee (Zigbee), NFC (near field communication), magnetic secure transmission (Magnetic Secure Transmission), radio frequency (RF) and body area network ( BAN), and the like, and wired communication includes at least one of universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard232 (RS-232), power line communication, and plain old telephone service (POTS). may contain one.

The sensor 120 may acquire synchronization sensing signals regarding a feature (stimulus and event) provided to the user while the user views a plurality of event sequences displayed on the display 150 . For example, the event is a stimulus given to the user, such as an event related to a user's movement, an event related to a change in ambient brightness, an event in which a preset amount of light is detected, or an event in which a sound (or voice) of a preset volume is detected It may include related events. For another example, the event may include, in relation to a cognitive ability task, an event related to the start or end of an event sequence, guide text for testing cognitive function, guide audio, a graphic object positioned in a three-dimensional space as a background, the graphic It may include an event related to provision of a cognitive function task, such as an object's motion and sound effect.

For synchronization between signals acquired from the HMD device 100 , the sensor 120 includes a first sensor 120 - 1 , a second sensor 120 - 2 , and a first corresponding to the first sensor 120 - 1 . It may include a synchronization sensor 120-10 and a second synchronization sensor 120-20 corresponding to the second sensor 120-2.

The first sensor 120-1 may be disposed adjacent to the first synchronization sensor 120-10, and the second sensor 120-2 may be disposed adjacent to the second synchronization sensor 120-20, The first sensor 120 - 1 and the first synchronization sensor 120 - 10 may be physically spaced apart from the second sensor 120 - 2 and the second synchronization sensor 120 - 20 .

Specifically, the first sensor 120-1 may output a first sensing signal, and the second sensor 120-2 may output a second sensing signal. The first sensor 120-1 and the second sensor 120-2 may be at least two or more bio-sensors for obtaining a bio-signal indicating the user's bio-information.

According to an embodiment, the first sensor 120-1 and the second sensor 120-2 may include a heartbeat sensor, an electrocardiogram sensor, an electroencephalogram sensor, an EMG sensor, a temperature sensor, and a camera sensor connected to the HMD device 100. and periodically output a sensing signal.

For example, when the user wears the HMD device 100, the at least one sensor 120 for acquiring the user's bio-signal is in contact with the user's skin to obtain a user's heartbeat signal (Photoplethysmogram, PPG), an electrocardiogram signal ( Electrocardiogram, ECG), electroencephalography (EEG), electromyography (EMG), and skin temperature can be measured. Meanwhile, the EEG sensor used for the cognitive function test may not be disposed on the entire head of the user, but may be disposed on a portion of the user's frontal lobe region.

For another example, the sensor 120 may photograph the user's face when the HMD device 100 is worn, and through this, the user's gaze movement, the size of the iris region around the pupil, and facial expression through the movement of muscles around the eyes heart rate information due to changes in blood flow in the blood vessels around the eyes can be acquired.

The first synchronization sensor 120-10 may output a first synchronization sensing signal, and the second synchronization sensor 120-20 may output a second synchronization sensing signal related to the second sensing signal. The first synchronization sensor 120-10 and the second synchronization sensor 120-20 may be related to at least two or more series of signals. Here, the series of signals include a first sensing signal output from the first sensor 120-1, a second sensing signal output from the second sensor 120-2, an image signal of a virtual reality image, and various signals in the system. may include at least one of

According to an embodiment, the first synchronization sensor 120 - 10 and the second synchronization sensor 120 - 20 may include a motion sensor connected to the HMD device 100 , an illuminance sensor, an optical sensor, and a sound wave sensor. The sensor may include at least one of an acceleration sensor, a gyroscope, and a geomagnetic sensor.

For example, the first synchronization sensor 120-10 and the second synchronization sensor 120-20 may acquire a synchronization sensing signal representing user movement information through a motion sensor, and obtain ambient brightness information through an illuminance sensor. It is possible to obtain a synchronization sensing signal representing can be obtained.

The memory 130 may store various data used for time synchronization between a series of signals. For example, the memory 130 may store a plurality of event sequences received through the communication interface 110 and at least one time stamp for each of the plurality of event sequences. As another example, the memory 130 may store a sensing signal obtained through the sensor 120 . As another example, the memory 130 may store time difference information between the synchronized sensing signals calculated by the sensor 120 or the processor 140 .

According to an embodiment, the memory 130 is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg, SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, network storage. It may include a storage medium of at least one type of storage, cloud, and block chain database.

The processor 140 is operatively connected to the communication interface 110 , the sensor 120 , the memory 130 , and the display 150 , and executes various commands for displaying a plurality of continuous event sequences on the display 150 . can be done Specifically, the processor 140 may display a plurality of consecutive event sequences received through the communication interface 110 or stored in the memory 130 on the display 120, and perform time synchronization between the series of signals. can do.

According to an embodiment, the processor 140 receives a first synchronization sensing signal related to an event from the first synchronization sensor 120-10 and a second synchronization sensing signal related to the corresponding event from the second synchronization sensor 120-20 signal can be received. The processor 140 may perform time synchronization between two or more series of signals based on the first synchronization sensing signal and the second synchronization sensing signal. For example, the two or more series of signals may include a first sensing signal and a second sensing signal.

Specifically, the processor 140 may check characteristic information of each of the first and second synchronization sensing signals, and calculate time difference information between the first and second synchronization sensing signals based on the identified characteristic information. Here, the characteristic information may include any one of a peak value of the synchronization sensing signal or a preset threshold value.

According to an embodiment, the processor 150 checks at least one peak value among the measurement values of the first synchronization sensing signal and at least one peak value among the measurement values of the second synchronization sensing signal, and at least the first synchronization sensing signal. A difference value between the first time corresponding to one peak value and the second time corresponding to at least one peak value of the second synchronization sensing signal may be calculated as time difference information.

According to another embodiment, the processor 140 may measure a first time value corresponding to a preset threshold value among measurement values of the first synchronization sensing signal and a measurement corresponding to a threshold value among measurement values of the second synchronization sensing signal The second time of the value may be checked, and a difference value between the checked first time and the second time may be calculated as time difference information.

As described above, after calculating the time difference information, the processor 140 may perform time synchronization on two or more series of signals based on the calculated time difference information. For example, when the at least two series of signals are the first sensing signal and the second sensing signal, the processor 140 determines the time for each of the measurement values of the first sensing signal and the second sensing signal. Time correction can be performed by summing or subtracting differences.

In addition, the processor 140 may perform time synchronization between the first synchronization sensing signal and the second synchronization sensing signal based on the calculated time difference information. For example, the processor 140 may perform time correction by adding or subtracting a time difference calculated in time for each of the measurement values of the first synchronization sensing signal and the second synchronization sensing signal.

According to an embodiment, when time synchronization is performed using a motion sensor, the motion sensor may equally reflect the user's dynamics even in different system environments. Accordingly, the processor 140 records the time for each of the first motion sensor and the second motion sensor for measuring the user's motion information using a timer, and the first motion sensor measured according to the time recorded through the first motion sensor. Time difference information may be calculated by comparing the values with the second values measured according to the time recorded by the second motion sensor. For example, the processor 140 may calculate a time difference between a peak value among the first values and a peak value among the second values as time difference information. As another example, the processor 140 may calculate a time difference between any one of the first values and any one of the second values corresponding to a preset threshold value as the time difference information. The processor 140 may perform time synchronization between a series of signals using the calculated time difference information.

According to another embodiment, when time synchronization is performed using a sound wave sensor, a dynamic change of an environment given to a user may be equally reflected by the sound wave sensor in different system environments. For example, when the user shouts a sound, the change may be equally measured by the biometric sensor and the sound wave sensor. In this case, the HMD device 100 or a separate output device (not shown) connected to the HMD device 100 may include a microphone. Accordingly, the processor 140 records the time for each of the first sound wave sensor and the second sound wave sensor for measuring sound (or voice) information using a timer, and the measured time is measured according to the time recorded through the first sound wave sensor. Time difference information may be calculated by comparing the first values with the second values measured according to the time recorded by the second sound wave sensor. For example, the processor 140 may calculate a time difference between a peak value among the first values and a peak value among the second values as time difference information. As another example, the processor 140 may calculate a time difference between any one of the first values and any one of the second values corresponding to a preset threshold value as the time difference information. The processor 140 may perform time synchronization between a series of signals using the calculated time difference information.

According to another embodiment, when time synchronization is performed using an illuminance sensor or an optical sensor, the dynamic change of the environment given to the user may be equally reflected in the illuminance sensor or the optical sensor in different system environments. For example, changes in the biometric sensor and the illuminance sensor, or the biometric sensor and the optical sensor may be measured in the same way by a light stimulus according to a change in the surrounding environment or a user's movement. In various embodiments, the HMD device 100 or a separate output device (not shown) connected to the HMD device 100 may be provided with a separate light emitting unit for light stimulation such as infrared or visible light in order to provide light stimulation at the same timing. can Accordingly, the processor 140 records the time for the first illuminance or optical sensor and the second illuminance or optical sensor for measuring infrared or visible light using a timer, and the time recorded through the first illuminance or optical sensor The time difference information may be calculated by comparing the first values measured according to the illuminance and the second values measured according to the second illuminance or the time recorded through the optical sensor. For example, the processor 140 may calculate a time difference between a peak value among the first values and a peak value among the second values as time difference information. As another example, the processor 140 may calculate a time difference between any one of the first values and any one of the second values corresponding to a preset threshold value as the time difference information. The processor 140 may perform time synchronization between a series of signals using the calculated time difference information.

In various embodiments, the processor 140 may record a time stamp for the event sequence provided from the HMD device 100 in the biosignal and check the cognitive function, which will be described in more detail with reference to FIG. 3 . .

3 is a flowchart of a cognitive function test method using an HMD device according to an embodiment of the present invention.

Referring to FIG. 3 , the processor 140 of the HMD device 100 may provide a plurality of consecutive event sequences through the screen of the display 150 ( S110 ). The event sequence can be output through the HMD device 100 (or an output device connected separately from the HMD device 100), such as images, audio, video, vibration, motion, etc., and various types of content that the user can recognize; It may include video.

For example, the HMD device 100 may receive a user manipulation signal, that is, a cognitive function test start signal from the input device 105 connected to the HMD device 100 , and provide an event sequence stored in the memory 130 . . As another example, the HMD device 100 may provide a plurality of continuous event sequences to the user through a method of streaming a plurality of event sequences received from the cognitive function test server 200 to be described later.

According to an embodiment, the sequence of events may include short-term memory, long-term memory, reading ability, concentration, depth perception, agility, visual perception of the user, together with a sequence configured to obtain a biosignal of the user's resting state. ), spatial perception, working memory, focused attention, sustained attention, reading comprehension, and selective attention. may include

Meanwhile, before step S110 , the HMD device 100 may perform time synchronization between at least two synchronization sensors related to a biosignal. Specifically, the processor 140 may perform time synchronization between at least two or more sensors for acquiring biosignals and at least two or more synchronization sensors associated with the corresponding sensors.

After step S110, the HMD device 100 may obtain at least one time stamp for each of a plurality of consecutive event sequences (S120). In detail, the processor 140 may acquire a time stamp for a stimulus given to a user wearing the HMD device 100 or a time stamp for a cognitive ability task according to an event sequence. Here, the stimulus may include, for example, a visual stimulus provided to the user by an image or video included in the event sequence, and an auditory stimulus provided to the user by a sound effect included in the event sequence. Cognitive skills tasks may also include guiding text for a specific action that the user must perform (eg, “Please select an object you saw on the previous screen”, a graphical object “Gaze at the red dot”, etc.).

That is, the processor 140 may pre-record all types of events that can be seen, heard, or felt by the user in a series of processes of providing the event sequence, and may obtain a time stamp for the recorded event.

According to an embodiment, the event sequence may include an event corresponding to provision of at least one of guide text for testing a cognitive function, guide audio, a graphic object positioned in a 3D space as a background, motion of the graphic object, and sound effect. . For example, one sequence of events would include “time stamps for presentation of events related to initiation”, “at least one timestamp for presentation of stimuli or cognitive ability tasks”, and “time stamps for presentation of events related to termination”. can

After step S120 , the processor 140 may acquire biosignals corresponding to a plurality of consecutive event sequences ( S130 ). Specifically, the processor 140 may obtain a biosignal from at least one of a heartbeat sensor, an electrocardiogram sensor, an electroencephalogram sensor, an electromyogram sensor, a temperature sensor, and a camera sensor connected to the HMD device 100 .

After step S130 , the processor 140 may identify the biosignal using the time stamp obtained through step S120 . That is, after step S130 , the processor 240 may record at least one time stamp obtained in step S120 on the biosignal ( S140 ).

After step S140 , the processor 140 may provide the user's cognitive function evaluation result based on the biosignal according to the time stamp through the screen of the display 150 ( S250 ). Specifically, the processor 140 transmits the biosignal divided by the time stamp to the cognitive function test server 200 through the communication interface 110 , and evaluates the cognitive function based on this from the cognitive function test server 200 . results can be received.

However, the method of providing the evaluation result is not limited thereto, and the processor 140 may generate a cognitive function evaluation result based on the divided bio-signal on its own. For example, the processor 140 classifies the EEG signal obtained from the EEG sensor based on the time stamp, and generates the user's cognitive function evaluation result based on the PSD (Power Spectral Density) for each frequency band in the divided EEG signal. can

Referring back to FIG. 2 , the display 150 may display various contents received through the communication interface 110 and a series of event sequences. For example, various contents may include text, an image, an image, an icon, a banner, or a symbol, and according to an embodiment, the display 150 may display a virtual reality image received through the communication interface 110 . can

So far, the HMD device 100 according to an embodiment of the present invention has been described. Hereinafter, with reference to FIG. 4 , time stamps for a plurality of consecutive event sequences are recorded, and cognitive function is tested based on this. The cognitive function test server 200 will be described in more detail.

4 is a block diagram of a cognitive function test server according to an embodiment of the present invention.

Referring to FIG. 4 , the cognitive function test server 200 may include a communication interface 210 , a memory 220 , an I/O interface 230 and a processor 240 , and each configuration includes one or more communication buses. Alternatively, they may communicate with each other via signal lines.

The communication interface 210 may be connected to the HMD device 100 and the manager device 300 through a wired/wireless communication network to exchange data. For example, the communication interface 210 provides a plurality of continuous event sequences to the HMD device 100 , and input data based on the plurality of continuous event sequences from the HMD device 100 , biosignals (eg. Pupil velocity, brain wave signals) and sensing signals can be received. For another example, the communication interface 210 receives user data for performing a cognitive function test from the manager device 300 , and transmits the cognitive function test history, recovery history, diagnosis result, etc. of the corresponding user to the manager device 300 . can send

Meanwhile, the communication interface 210 that enables transmission and reception of such data may include a communication pod 211 and a wireless circuit 212 . Here, the wired communication port 211 may include one or more wired interfaces, for example, Ethernet, Universal Serial Bus (USB), FireWire, and the like. Also, the wireless circuit 212 may transmit/receive data to and from an external device through an RF signal or an optical signal. In addition, the wireless communication may use at least one of a plurality of communication standards, protocols and technologies, such as GSM, EDGE, CDMA, TDMA, Bluetooth, Wi-Fi, VoIP, Wi-MAX, or any other suitable communication protocol.

The memory 220 may store various data used in the cognitive function test server 200 . For example, the memory 220 may include a sequence configured to cause the user to meditate, the user's short-term memory, long-term memory, reading ability, concentration, depth perception, agility, visual perception, and spatial perception. It can store sequences of events configured to test spatial perception, working memory, focused attention, sustained attention, reading comprehension and selective attention.

As another example, the memory 220 may store at least one timestamp log for each of a plurality of events provided to the user. Also, the memory 220 may store a time stamp log for a sub-event pre-stored in each of a plurality of event sequences according to the type of event. Here, the pre-stored sub-event may correspond to provision of at least one of guide text for testing a cognitive function, guide audio, a graphic object positioned in a 3D space as a background, motion of the graphic object, and sound effect.

As another example, the memory 220 stores synchronization sensing information obtained by the sensor 120 of the HMD device 100 or between synchronization sensing signals calculated by the processor 140 of the HMD device 100 . Time difference information can be stored. In addition, the memory 230 may store sensing information or synchronization sensing information received from the HMD device 100 .

As another example, the memory 220 may include a comprehensive cognitive impairment, a cognitive evaluation score reference value for diagnosing a cognitive impairment for each cognitive function, and an EEG signal (eg, EEG signal) between a normal group and a patient group with reduced cognitive function. ) to store differences, patterns, etc.

In various embodiments, the memory 220 may include a volatile or non-volatile recording medium capable of storing various data, commands, and information. For example, the memory 220 may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg, SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, network storage storage. , cloud, and may include at least one type of storage medium of a blockchain database.

In various embodiments, the memory 220 may store a configuration of at least one of the operating system 221 , the communication module 222 , the user interface module 223 , and one or more applications 224 .

Operating system 221 (eg embedded operating systems such as LINUX, UNIX, MAC OS, WINDOWS, VxWorks, etc.) controls and manages various software for general system operations (eg memory management, storage device control, power management, etc.) It may include components and drivers, and may support communication between various hardware, firmware, and software components.

The communication module 223 may support communication with other devices through the communication interface 210 . The communication module 220 may include various software components for processing data received by the wired communication port 211 or the wireless circuit 212 of the communication interface 210 .

The user interface module 223 may receive a user's request or input from a keyboard, a touch screen, a microphone, etc. through the I/O interface 230 , and may provide a user interface on a display.

Applications 224 may include programs or modules configured to be executed by one or more processors 240 . Here, an application for recording a time stamp in a biosignal based on a plurality of consecutive event sequences and generating a cognitive evaluation result based on the time stamp may be implemented on a server farm.

The I/O interface 230 may connect an input/output device (not shown) of the cognitive function test server 200 , for example, at least one of a display, a keyboard, a touch screen, and a microphone, with the user interface module 223 . The I/O interface 230 may receive a user input (eg, a voice input, a keyboard input, a touch input, etc.) together with the user interface module 223 and process a command according to the received input.

The processor 240 is connected to the communication interface 210 , the memory 220 , and the I/O interface 230 to control the overall operation of the cognitive function test server 200 , and an application stored in the memory 220 or A program may perform various commands for mapping features between sequences and determining whether a user has a cognitive impairment.

The processor 240 may correspond to a computing device such as a central processing unit (CPU) or an application processor (AP). In addition, the processor 240 may be implemented in the form of an integrated chip (IC), such as a system on chip (SoC) in which various computing devices are integrated. Alternatively, the processor 240 may include a module for calculating an artificial neural network model, such as a Neural Processing Unit (NPU).

According to an embodiment, the processor 240 may provide a plurality of successive event sequences to the HMD device 100 and perform time synchronization between at least two or more series of signals based on this.

Specifically, the processor 240 may calculate time difference information between the first synchronization sensing signal and the third synchronization sensing signal received from the HMD device 100 . Here, the third synchronization sensing signal may be obtained from a separate output device connected to the HMD device 100 . That is, when the HMD device 100 includes only the first sensor 120-1 and the first synchronization sensor 120-10 for obtaining a biosignal, the second sensor 120-1 and the second synchronization sensor Reference numeral 120-20 may be provided in a separate output device, and accordingly, the third synchronization sensing signal may be received from the separate output device.

When the first and third synchronization sensing signals related to the event are received, the processor 240 checks characteristic information of each of the received first and third synchronization sensing signals, and based on the identified characteristic information, the first and third synchronization sensing signals Time difference information between synchronization sensing signals may be calculated.

For example, the processor 240 checks at least one peak value among the measurement values of the first synchronization sensing signal and at least one peak value among the measurement values of the third synchronization sensing signal, and at least one of the first synchronization sensing signal A difference value between the first time corresponding to the peak value of , and the third time corresponding to at least one peak value of the third synchronization sensing signal may be calculated as time difference information.

For another example, the processor 240 may measure a first time of a measurement value corresponding to a preset threshold value among measurement values of the first synchronization sensing signal and a measurement corresponding to a threshold value among measurement values of the third synchronization sensing signal A third time of the value may be checked, and a difference value between the checked first time and the third time may be calculated as time difference information.

The processor 240 may perform time synchronization between at least two or more series of signals based on the calculated time difference information. Specifically, the processor 240 may perform time correction by adding the time difference and time difference of each measurement value with respect to the first sensing signal or subtracting the time difference from the time of each measurement value with respect to the first sensing signal. Alternatively, the processor 240 may perform time correction by adding the time and time difference of a plurality of consecutive event sequences (images) or subtracting the time difference from the image signal.

In various embodiments, the processor 240 may time-synchronize the system time of the HMD device 100 and the system time of the image output device 200 using the calculated time difference information. Specifically, the processor 240 transmits the calculated time difference information to the HMD device 100 so that the HMD device 100 corrects the system time or sums the time difference calculated in the system time of the image output device 200 . Alternatively, time correction may be performed by subtraction.

In various embodiments, the processor 150 may perform time synchronization between the first synchronization sensing signal and the third synchronization sensing signal based on the calculated time difference information. For example, the processor 150 may perform time correction by adding or subtracting a time difference calculated in time for each of the measured values of the first synchronization sensing signal and the third synchronization sensing signal.

Hereinafter, with reference to FIGS. 5 to 8F , a method in which the processor 240 of the cognitive function test server 200 records time stamps for a plurality of successive event sequences in a biosignal, and performs a cognitive function test to be explained.

5 is a flowchart of a cognitive function test method using a cognitive function test server according to an embodiment of the present invention.

Referring to FIG. 5 , the processor 240 may provide a plurality of consecutive event sequences through the HMD device 100 ( S210 ). The event sequence may include images, audio, video, vibration, motion, etc., output through the HMD device 100 and an output device connected thereto, and may include various types of content and images that a user can recognize.

According to an embodiment, the sequence of events may include short-term memory, long-term memory, reading ability, concentration, depth perception, agility, visual perception of the user, together with a sequence configured to obtain a biosignal of the user's resting state. ), spatial perception, working memory, focused attention, sustained attention, reading comprehension, and selective attention. may include

The processor 240 may variously combine the above-described event sequence according to which cognitive function is tested through the HMD device 100 . The processor 240 receives a selection of a type of cognitive function test to be performed by the user from the HMD device 100 or the manager device 300 through the communication interface 210 , and transmits a plurality of event sequences to the HMD device 100 accordingly. can provide

Meanwhile, before step S210 , the processor 240 may perform time synchronization between at least two synchronization sensors related to the biosignal through the HMD device 100 . Specifically, the processor 240 may perform time synchronization between at least two or more sensors for obtaining a biosignal and at least two or more synchronization sensors associated with the corresponding sensors.

After step S210 , the processor 240 may obtain at least one time stamp for each of a plurality of consecutive event sequences from the HMD device 100 ( S220 ). In detail, the processor 240 may acquire a time stamp for a stimulus given to a user equipped with the HMD device 100 or a time stamp for a cognitive ability task according to an event sequence. Here, the stimulus may include, for example, a visual stimulus provided to the user by an image or video included in the event sequence, and an auditory stimulus provided to the user by a sound effect included in the event sequence. Cognitive skills tasks may also include guiding text for a specific action that the user must perform (eg, “Please select an object you saw on the previous screen”, a graphical object “Gaze at the red dot”, etc.).

That is, the processor 240 may pre-record all types of events that can be seen, heard, or felt by the user in a series of processes of providing the event sequence, and may obtain a time stamp for the recorded event.

According to an embodiment, the event sequence may include an event corresponding to provision of at least one of guide text for testing a cognitive function, guide audio, a graphic object positioned in a 3D space as a background, motion of the graphic object, and sound effect. . For example, one sequence of events would include “time stamps for presentation of events related to initiation”, “at least one timestamp for presentation of stimuli or cognitive ability tasks”, and “time stamps for presentation of events related to termination”. can

After step S220 , the processor 240 may obtain biosignals corresponding to a plurality of consecutive event sequences from the HMD device 100 ( S230 ). Specifically, the processor 240 may obtain a biosignal from at least one of a heart rate sensor, an electrocardiogram sensor, an electroencephalogram sensor, an electromyogram sensor, a temperature sensor, and a camera sensor connected to the HMD device 100 .

In other words, the processor 240 may continuously acquire biosignals from the HMD device 100 while providing a series of event sequences for testing the cognitive function.

On the other hand, in order to classify the continuous biosignals according to the event sequence, when the provision of one event sequence is completed, the cognitive function test is temporarily stopped, and a simple method of matching and storing one event sequence with the corresponding biosignal can also be used. However, in this case, the concentration of the user's cognitive function test is lowered, and when the number of events (features) present in the event sequence is large, the number of files to be stored increases too much.

In addition, events other than pre-stored events may be generated depending on what actions the user performs for each event sequence, so that while a plurality of event sequences are provided, the event sequence is provided for feature matching between the event sequence and biosignals It is not advisable to stop for a while.

Accordingly, the processor 240 may identify the biosignal using the time stamp obtained in step S220. That is, after step S230, the processor 240 may record at least one time stamp obtained in step S220 in the biosignal (S240).

6 and 7 are schematic diagrams for explaining a method of matching a feature of a sequence provided by an HMD device with a biosignal according to an embodiment of the present invention.

Referring to FIG. 6 , the processor 240 records time stamps of each of a plurality of consecutive event sequences being provided to the HMD device 100 in continuous biosignals acquired through the sensor 120 of the HMD device 100 . can do. For example, the plurality of consecutive sequences of events may include a sequence of events configured to proceed with meditation, a sequence configured to test sequence memory, and the ability to remember.

The processor 240 may obtain a time stamp related to the provision of the main event stored in each event sequence, and record it in the biosignal. Here, the time stamp for providing the main event may be, for example, a time stamp for providing an event related to the start or end of an event sequence.

Meanwhile, in FIG. 6 , although it is illustrated that the time stamps related to the start and the end are the same time in two event sequences, the time stamps related to the start may be earlier than the time stamps related to the end. For example, two timestamps may have a time difference of ms.

Referring to FIG. 7 , the processor 240 records time stamps for sub-events pre-stored in each of a plurality of continuous event sequences in the continuous bio-signals acquired through the sensor 120 of the HMD device 100 . can For example, the event sequence configured to test concentration may include a plurality of main events (①②) and sub-events (①②③④⑤⑥) as follows, and the processor 240 records a time stamp for each event in the biosignal and biosignals can be classified for each section.

8A to 8F are schematic diagrams for explaining a sequence interface provided in an HMD device according to an embodiment of the present invention.

Referring to FIG. 8A , the processor 240 considers the provision of guide text 11 related to the start of the event sequence, such as “a game to remember the ball that has been blinked,” provided to the HMD device 100 as ① the main event, and , a time stamp for this can be recorded in the biosignal.

Next, referring to FIGS. 8B to 8E , the processor 240 provides the graphic object 12, the guide text 13, and the motion of the graphic object 14 and 15 in the HMD device 100, The provision of the sound effect 16 may be regarded as a plurality of sub-events, and a time stamp for this may be recorded in the biosignal.

Finally, referring to FIG. 8f , the processor 240 regards the provision of the guide text 17 related to the end of the sequence informing that it is the last round in the HMD device 100 as the main event (②), and converts the time stamp for this as a main event. signal can be recorded.

Meanwhile, the events (features) provided in the event sequence are divided into main events and sub-events according to the pre-stored time sequence. . For example, serial numbers having the same regularity may be matched to both the main event and the sub-event, and a start time and an end time of the corresponding event may be recorded in each event.

Meanwhile, while a cognitive function test is performed through an event sequence, a time stamp for providing an event related to termination may vary according to a user's cognitive function evaluation result. For example, in the event sequence consisting of a total of 9 times, a pre-stored main time stamp may be a time stamp for providing a guide text indicating that the 9th time execution is completed. However, if the user does not succeed in the test until the ninth time, the timestamp for transition to the next sequence of events may be the timestamp for providing events related to termination.

To this end, the processor 240 may obtain a time stamp for the input data obtained from the input device 105 connected to the HMD device 100 based on a plurality of consecutive event sequences. The processor 240 may additionally record a time stamp for input data in the biosignal.

The processor 240 may generate an evaluation result for any one event sequence among a plurality of continuous event sequences based on the input data, and determine whether to provide a pre-stored sub-event or a next event sequence based on the evaluation result. can For example, the processor 240 does not provide a pre-stored sub-event to the HMD device 100 when the user makes a mistake 3 or more times in the ball matching game (event sequence) based on the input data, but the next game (next event) sequence) can be provided.

5 and 7 , the processor 240 may generate the user's cognitive function evaluation result based on the biosignal according to the time stamp (S250). Specifically, the processor 240 may divide the biosignal into a plurality of sections according to a time stamp recorded in the biosignal.

In various embodiments, the processor 240 classifies the EEG signal (biometric signal) obtained from the EEG sensor based on the time stamp, and in the divided EEG signal, the user's recognition based on the PSD (Power Spectral Density) for each frequency band Functional evaluation results can be generated. Specifically, the processor 240 determines the PSD value of the reference EEG signal in each of the 1-3 Hz delta band, the 4-7 Hz theta band, the 8-14 Hz alpha band, and the 15-30 Hz beta band and the EEG signal obtained from the user. The difference in PSD values can be obtained. The processor 240 may determine whether the user has cognitive impairment (a cognitive evaluation result) by comparing the difference between the PSD value of the user and the PSD value obtained from the normal group. For example, in the case of depth perception, in patients with cognitive dysfunction, the PSD value in the alpha band may be lower than the PSD value in the normal group.

9 and 10 are graphs for explaining that the accuracy of the cognitive function test is increased through feature combination of sequences provided by the HMD device according to an embodiment of the present invention.

Referring to FIG. 9 , the difference in PSD values between the normal group (gray bar) and the patient group with cognitive decline (white bar) for each frequency band of the delta band, theta band, alpha band, and beta band is shown as the following graph. can be obtained Specifically, as a result of analyzing biosignals based on time stamps in various event sequences such as attention, working memory, spatial perception, etc., the patient group with reduced cognitive function compared to the normal group in a specific frequency band (alpha band, beta band) It can be seen that the PSD value of is lowered. In particular, it can be seen that the difference in PSD values in selective attention and depth perception remains at 0.08 and 0.01 levels. That is, the processor 240 subdivides and classifies a series of biosignals based on the time stamp for the event (feature), thereby obtaining the fine PSD value difference as described above, and through this, an accurate cognitive function evaluation result can be generated. there is.

In addition, the processor 240 may obtain a PSD value for each frequency band and generate a cognitive function evaluation result based on a ratio for each band. Specifically, the processor 240 calculates a delta/alpha ratio (DAR) or theta/alpha ratio (TAR), and then calculates the ratio value of the normal group and the calculated value of the current user. The ratio values can be compared, and this can generate a cognitive function evaluation result.

In various embodiments, the processor 240 classifies the user's pupil movement speed or size change (biometric signal) obtained from the camera sensor based on the time stamp, and the user's cognitive function evaluation result based on the divided change value can create Specifically, the processor 240 divides a region in which a time stamp for providing a visual event is recorded among a series of biosignals for pupil movement or size change, and analyzes the divided biosignals to generate a cognitive function evaluation result. can

Referring to FIG. 10 , in a state in which a visual event is provided through the HMD device 100 , a normal group (NC), a patient group with subjective cognitive decline (SCD), a patient group with mild cognitive decline (MCI), and a group with mild Alzheimer's disease (mild AD) The pupil movement speed can be obtained as the following graph. In particular, in the case of mild Alzheimer's patient group, it can be seen that the rate of change in pupil size is lower than the rate of change in the rest of the normal and patient groups. That is, the processor 240 subdivides and classifies a series of biosignals based on the time stamps for events (features), so that biosignals required for analysis can be obtained even through a small amount of data, through which accurate recognition Functional evaluation results can be generated.

On the other hand, the processor 240 generates the user's cognitive evaluation result by combining the cognitive function evaluation score obtained through the HMD device 100 in addition to generating the cognitive function evaluation result using the biosignal separated through the time stamp. can do. Specifically, the cognitive function evaluation score may mean a score calculated based on the user's input data in a plurality of consecutive event sequences provided to the user through the HMD device 100 . That is, the processor 240 may generate a user's cognitive function evaluation result (whether or not there is a cognitive function impairment) by comparing the reference score specified in each event sequence with a score calculated based on input data.

According to an embodiment, the processor 240 may generate one cognitive function evaluation result by complementing the evaluation result generated through the biosignal and the evaluation result generated through the input data, or provide two results to the user as it is. can That is, the processor 240 may transmit the user's cognitive evaluation result to the HMD device 100 or the manager device 300 through the communication interface 210 .

So far, the cognitive function test server 200 according to an embodiment of the present invention has been described. According to the present invention, a series of biosignals can be classified based on one event sequence by recording a time stamp for an event provision time in biosignals obtained from a user while a plurality of event sequences are provided. In particular, by subdividing bio-signals according to pre-stored events in the event sequence, bio-signal changes in response to short-term stimulation given to the user can be acquired, and through this, it is possible to accurately determine whether or not the user's cognitive function is impaired.

On the other hand, the time stamp recording in the biosignal performed by the HMD device 100 and the cognitive function test server 200 and the cognitive function test method using the same are performed on a display device (eg, a smart phone) separately connected to the HMD device 100 or It may be performed on a computing device connected thereto. In this case, a plurality of consecutive event sequences may be stored in the display device or a computing device connected thereto. In addition, the display device or a computing device connected thereto may record a time stamp on a biosignal obtained from a sensor connected to the HMD device 100 , and generate or provide a cognitive function test result based thereon.

Although one embodiment of the present invention has been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and various modifications may be made within the scope without departing from the technical spirit of the present invention. . Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

1000: cognitive function test system
100: HMD device
105: input device
110: communication interface 120: sensor
120-1: first sensor 120-2: second sensor
120-10: first synchronization sensor 120-20: second synchronization sensor
130: memory 140: processor
150: display
200: cognitive function test server
210: communication interface
211: wired communication port 212: wireless circuit
220: memory
221: operating system 222: communication module
223: user interface module 224: application
230: I/O interface 240: processor
300: manager device

Claims (26)

providing a plurality of consecutive sequences of events via the HMD device;
obtaining at least one time stamp for each of the plurality of consecutive event sequences from the HMD device;
obtaining biosignals corresponding to the plurality of consecutive event sequences from the HMD device;
recording the at least one time stamp on the biosignal; and
generating a user's cognitive function evaluation result based on the biosignal according to the time stamp; Cognitive function test method using feature combination comprising a. According to claim 1,
Obtaining the at least one time stamp comprises:
Cognitive function test method using feature combination further comprising the step of obtaining a time stamp for a sub-event pre-stored in each of the plurality of consecutive event sequences. 3. The method of claim 2,
The pre-stored sub-event is
A cognitive function test method using feature combination, which corresponds to providing at least one of guide text, guide audio, a graphic object positioned in a three-dimensional space as a background, motion of the graphic object, and sound effect for testing cognitive function. 3. The method of claim 2,
obtaining time stamps for input data corresponding to the plurality of consecutive sequence of events from an input device connected to the HMD device; and
Cognitive function test method using feature combination further comprising the step of recording a time stamp of the input data in the biosignal. 5. The method of claim 4,
generating an evaluation result for any one of the plurality of consecutive event sequences based on the input data; and
and determining whether to provide the pre-stored sub-event or the next event sequence based on the evaluation result. According to claim 1,
The biosignal is
Cognitive function test method using feature combination obtained from at least one of a heart rate sensor, an electrocardiogram sensor, an electroencephalogram sensor, an electromyography sensor, a temperature sensor, and a camera sensor connected to the HMD device. 7. The method of claim 6,
The step of generating the cognitive function evaluation result,
Classifying the EEG signal obtained from the EEG sensor based on the time stamp,
A cognitive function test method using feature combination, which is a step of generating a user's cognitive function evaluation result based on the PSD (Power Spectral Density) for each frequency band in the divided EEG signal. 7. The method of claim 6,
The step of generating the cognitive function evaluation result,
Classifying a change in the user's pupil movement speed or size obtained from the camera sensor based on the time stamp,
A cognitive function test method using feature combination, which is a step of generating a user's cognitive function evaluation result based on the differentiated change value. According to claim 1,
Prior to providing the plurality of event sequences,
Using at least two or more synchronization sensors related to the biosignal,
Cognitive function test method using feature combination, further comprising the step of performing time synchronization on the bio-signals obtained from the at least two or more sensors. 10. The method of claim 9,
The synchronization sensor is
It is disposed adjacent to the sensor for acquiring the bio-signal,
A cognitive function test method using feature coupling, comprising at least one of a motion sensor, an illuminance sensor, an optical sensor, and a sound wave sensor. 10. The method of claim 9,
The HMD device,
A first sensor for acquiring the biosignal, a second sensor, a first synchronization sensor disposed adjacent to the first sensor, and a second synchronization sensor disposed adjacent to the second sensor, using feature combination Cognitive function test method. 12. The method of claim 11,
The step of performing the time synchronization comprises:
calculating time difference information for the first and second synchronization sensing signals; and
Cognitive function test method using feature combination further comprising the step of performing time synchronization on the first bio-signal and the second bio-signal based on the calculated time difference information. 13. The method of claim 12,
One of the first bio-signal obtained through the first sensor and the second bio-signal obtained through the second sensor includes an image signal. 13. The method of claim 12,
Calculating the time difference information comprises:
Checking characteristic information for each of the first and second synchronization sensing signals, and
Cognitive function test method using feature combination further comprising the step of calculating the time difference information based on the identified characteristic information. 15. The method of claim 14,
The characteristic information is
A cognitive function test method using feature combination including at least one of a peak value of a synchronization sensing signal and a preset threshold value. According to claim 1,
The event sequence is
Short-term memory, long-term memory, reading ability, concentration, depth perception, agility, visual perception, spatial perception of the user together with a sequence configured to acquire the biosignal of the user's resting state , comprising a sequence configured to test cognitive abilities of at least one of working memory, focused attention, sustained attention, reading comprehension, and selective attention. Cognitive function test method. communication interface;
Memory; and
a processor operatively coupled to the communication interface and the memory; including,
The processor is
providing a sequence of a plurality of consecutive events through the HMD device;
obtaining at least one time stamp for each of the plurality of consecutive event sequences from the HMD device;
obtaining biosignals corresponding to the plurality of consecutive event sequences from the HMD device;
recording the at least one time stamp in the biosignal,
Cognitive function test server using feature combination, configured to generate a user's cognitive function evaluation result based on the biosignal according to the time stamp. providing a sequence of a plurality of consecutive events through a display screen;
obtaining at least one timestamp for each of the plurality of consecutive event sequences;
acquiring biosignals corresponding to the plurality of consecutive event sequences;
recording the at least one time stamp on the biosignal; and
providing a user's cognitive function evaluation result based on the biosignal according to the time stamp through the display screen; Cognitive function test method using feature combination comprising a. 19. The method of claim 18,
Obtaining the at least one time stamp comprises:
Cognitive function test method using feature combination further comprising the step of obtaining a time stamp for a sub-event pre-stored in each of the plurality of consecutive event sequences. 20. The method of claim 19,
The pre-stored sub-event is
A cognitive function test method using feature combination, which corresponds to providing at least one of guide text, guide audio, a graphic object positioned in a three-dimensional space as a background, motion of the graphic object, and sound effect for testing cognitive function. 19. The method of claim 18,
obtaining time stamps for input data corresponding to the plurality of consecutive event sequences from an input device interlocked with the display screen; and
Cognitive function test method using feature combination further comprising the step of recording a time stamp of the input data in the biosignal. 22. The method of claim 21,
providing an evaluation result for any one of the plurality of consecutive event sequences based on the input data; and
and determining whether to provide the pre-stored sub-event or the next event sequence based on the evaluation result. 19. The method of claim 18,
The biosignal is
Cognitive function test method using feature combination obtained from at least one of a heart rate sensor, an electrocardiogram sensor, an electroencephalogram sensor, an electromyography sensor, a temperature sensor, and a camera sensor connected to the HMD device. 19. The method of claim 18,
Prior to providing the plurality of event sequences,
Using at least two or more synchronization sensors related to the biosignal,
Cognitive function test method using feature combination, further comprising the step of performing time synchronization on the bio-signals obtained from the at least two or more sensors. 25. The method of claim 24,
The step of performing the time synchronization comprises:
calculating time difference information for the first and second synchronization sensing signals; and
Cognitive function test method using feature combination further comprising the step of performing time synchronization on the first bio-signal and the second bio-signal based on the calculated time difference information. communication interface;
Memory;
display; and
a processor operatively coupled to the communication interface, the memory, and the display; including,
The processor is
providing a sequence of a plurality of consecutive events through the screen of the display;
obtaining at least one timestamp for each of the plurality of consecutive event sequences;
acquiring biosignals corresponding to the plurality of consecutive event sequences;
recording the at least one time stamp in the biosignal,
The HMD device is configured to provide a user's cognitive function evaluation result based on the biosignal according to the time stamp through the screen of the display.

Images