KR20230052668A - A device for providing video and detecting brainwave and the method for operating the same - Google Patents

Abstract

In the apparatus for providing images and measuring brain waves according to various embodiments of the present disclosure, the apparatus for measuring brain waves comprises: a head-mounted display; earphone; touch screen display; electroencephalogram sensor; Memory; and a processor operably connected to the head mounted display, the earphone, the touch screen display, and the memory, wherein the processor: receives user information through the touch screen display; selecting a video file and a music file stored in a memory that match the information of the user and transmitting the video file to the head mounted display; playing video files and music files using the head mounted display; measuring the user's electroencephalogram using the electroencephalogram sensor after reproducing the video file and the music file; It may be set to display the electroencephalogram measurement result through the touch screen display.

Description

Image provision and brain wave measuring device and its operation method

The present application relates to an apparatus for providing images and measuring brain waves, and an operation method thereof. Specifically, in a device capable of checking the user's mental health by providing an image to the user of the device, measuring the EEG measured in the user's brain after the image is provided, checking the user's electroencephalogram measurement value, and providing feedback. related

Dementia is a deterioration in cognitive function, and rather than a disease in itself, damage to the brain due to various causes causes memory and various cognitive functions to be impaired, making it impossible to maintain daily life as much as the previous level. means It refers to complex symptoms in which the brain, once normally matured, is damaged or destroyed by external factors such as acquired trauma or disease, and overall cognitive functions such as intelligence, learning, language, and higher mental functions are reduced.

Modern society has entered an aging society, and as the elderly population increases, the incidence of dementia is also increasing. In accordance with this social trend, research on various methods for preventing dementia is being conducted. As part of the research, a program to improve the memory of dementia patients by stimulating the brain by having them listen to music is being used. In addition, the above-described program is not simply limited to patients with dementia, and can be used together to check the mental health of users of programs for preventing dementia.

However, it is necessary for the program operator to determine the music to be provided to the dementia patient or the user of the program on the program, but there may be difficulties in determining which music will be appropriate for the dementia patient or the user of the program. There was a downside to not being able to.

In order to solve the above-mentioned problems, the present invention can provide greater stimulation to users by maximizing user experience by providing not only music but also images to dementia patients or program users through a head-mounted display. In addition, it has an effect that the program can be operated more effectively by displaying electroencephalogram measurement values through a touch screen display and feeding back result values with high electroencephalogram measurement values.

In the apparatus for providing images and measuring brain waves according to various embodiments of the present disclosure, the apparatus for measuring brain waves comprises: a head-mounted display; earphone; touch screen display; electroencephalogram sensor; Memory; and a processor operably connected to the head mounted display, the earphone, the touch screen display, and the memory, wherein the processor: receives user information through the touch screen display; selecting a video file and a music file stored in a memory that match the information of the user and transmitting the video file to the head mounted display; playing video files and music files using the head mounted display; measuring the user's electroencephalogram using the electroencephalogram sensor after reproducing the video file and the music file; It may be set to display the electroencephalogram measurement result through the touch screen display.

According to an embodiment, the information of the user includes age information of the user, and the processor uses a learning model learned by a deep learning algorithm to select the user among music files and video files stored in the memory. It can be set to select a music file and a video file matching the age information.

According to an embodiment, the processor identifies a plurality of music files and a plurality of image files matching the user's age information, and uses a model learned by a deep learning algorithm to obtain the user history of the user's electroencephalogram. According to the information, a list is created by arranging the plurality of music files and the plurality of video files in ascending order, the electroencephalogram measurement value is converted into a parameter, and three music files and video files having high parameters are displayed on the touch screen display. It can be configured to display through.

According to an embodiment, the processor further includes a motion sensor wearable by a user, and the processor detects a motion of the user using the motion sensor, compares the detected motion of the user with a motion of an object in the video, A similarity parameter may be determined based on the comparison result using a model learned by a deep learning algorithm, and the similarity parameter may be displayed.

According to an embodiment, the processor is set to detect an electroencephalogram using the electroencephalogram sensor while detecting a motion using the motion sensor, and display the detected electroencephalogram along with the similarity parameter through the touch screen display. It can be.

According to an embodiment, the processor divides the user's motion into a plurality of partial motions, checks an electroencephalogram for each partial motion, and determines each of the partial motions using the electroencephalogram result value. It may be set to generate electroencephalogram parameters, respectively, and store the generated electroencephalogram parameters in memory for each operation.

According to an embodiment, the processor confirms that the video is a video conducted by a conductor using a model learned with a deep learning algorithm, and uses the motion sensor to check the motion and beat of the user's hand, It may be configured to compare the movement and tempo of the conductor's hand in the video, generate a similarity parameter based on the comparison result, quantify the similarity parameter, and display the similarity parameter on the touch screen display.

According to an embodiment, the processor classifies music files and video files for each age group of the user based on the age information of the user, and classifies the music file and video file for each age group based on taste information of the user stored in a memory. selected music and video files, a list of the selected music and video files is generated and displayed through a touch screen display, and each of the music and video files among the music and video files displayed through the touch screen display It may be set to receive a selection for, and play the music file and video file through the head mounted display.

According to an embodiment, the processor checks the user's electroencephalogram measurement results for each of the music files and the image files stored in the memory, creates a list by arranging the electroencephalogram measurement results in ascending order, and It can be set to store the created list in memory.

An operation of receiving user information through an image provision and a touch screen display according to an embodiment; selecting a video file and a music file stored in a memory that match the information of the user and transmitting the video file to the head mounted display; playing a video file and a music file using the head mounted display; measuring an electroencephalogram of the user by using the electroencephalogram sensor after reproducing the image file and the music file; An operation of displaying the electroencephalogram measurement result through the touch screen display may be included.

1 is a diagram illustrating a configuration in which a user wears and uses an apparatus for providing images and measuring brain waves according to various embodiments of the present disclosure.
2 is a block diagram of an apparatus for providing images and measuring brain waves according to various embodiments of the present disclosure.
3 is a flowchart of a method for providing an image and measuring an EEG according to various embodiments of the present disclosure.

It should be noted that the technical terms used in this specification are only used to describe specific embodiments and are not intended to limit the present invention. Also, singular expressions used in this specification include plural expressions unless the context clearly indicates otherwise. As used herein, “consists of.” or "includes." Such terms should not be construed as necessarily including all of the various components or steps described in the specification, and some of the components or steps may not be included, or additional components or steps may be included. It should be interpreted as being more inclusive.

In addition, in describing the technology disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the technology disclosed in this specification, the detailed description will be omitted.

In describing each figure, like reference numbers are used for like elements. In addition, terms such as first and second may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another.

Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in this application, it should not be interpreted in an ideal or excessively formal meaning. Should not be.

1 is a diagram illustrating a configuration in which a user wears and uses an apparatus for providing images and measuring brain waves according to various embodiments of the present disclosure. The user 100 may hold the display device 200 having a touch screen display and wear the head mounted display 101 to watch an image. The user 100 may listen to music by wearing earphones 102 and may listen to voice files through a speaker included in the head mounted display 101 . Although not shown in FIG. 1 , the user 100 may attach an electroencephalogram sensor for measuring the user's electroencephalogram. Through the dementia prevention and treatment program, music files and video files input from the user 100 are viewed and listened to using the head mounted display 101 and earphones 102, and video and music files are viewed and listened to. The changing electroencephalogram can be measured using an electroencephalogram sensor. The overall configuration of the apparatus for providing images and measuring brain waves will be described in detail with reference to FIG. 2 .

2 is a block diagram of an apparatus for providing images and measuring brain waves according to various embodiments of the present disclosure. 3 is a flowchart of a method for providing an image and measuring an EEG according to various embodiments of the present disclosure.

According to various embodiments, an apparatus for providing an image and measuring an EEG may include a head mount display (HMD) 201, an earphone 203, a display device 205, and a motion sensor 207. The head mounted display (HMD) 201, the earphone 203, and the motion sensor 207 may be operably connected to the display device 205 by wire or wirelessly. The display device 205 may include a processor 209 , a touch screen display 211 , and a memory 213 . The processor 209 may be connected to the touch screen display 211 and the memory 213 to control overall operations of the display device 205 .

In addition, the processor 209 may control the head mounted display 201, the earphone 203, the motion sensor 207, and the electroencephalogram sensor (not shown) connected to the display device 205 by wire or wirelessly. The touch screen display 211 may receive an input through a user's touch input and display measurement results and the like. The configuration shown in FIG. 2 is exemplary, and it will be apparent to those skilled in the art that components may be added or omitted as needed.

Referring to FIG. 3 , in operation 301, the processor 209 may receive user information through the touch screen display 211. User information may include various types of information related to the user, such as age information of the user and history information of the user's music files and video files.

In operation 303, the processor 209 may select an image file and a music file stored in a memory that match the information of the user and transmit the image file and the music file to the head mounted display. A plurality of image files and music files are stored in the memory, and the processor 209 may select a music file and a video file matching user information information from among the plurality of video files and music files by various methods.

In operation 305, the processor 209 may reproduce an image file and a music file using the head mounted display 201. In operation 307, the processor 209 may measure the user's electroencephalogram by using the electroencephalogram sensor after reproducing the video file and the music file. After a user watches and listens to an image file and a music file, a change in the user's electroencephalogram may occur, and the processor 209 may measure the above-described change in the user's electroencephalogram using the electroencephalogram sensor.

In operation 309, the processor 209 may display the electroencephalogram measurement result through the touch screen display 211. The electroencephalogram measurement result displayed through the touch screen display 211 may be displayed in the form of a graph, or may be displayed in the form of numbers by digitizing the electroencephalogram measurement result. In addition, it may be displayed in various ways.

According to an embodiment, the information of the user includes age information of the user, and the processor 209 uses a learning model learned by a deep learning algorithm before measuring the user's electroencephalogram, and the memory 213 It may be set to select a music file and a video file matching the age information of the user from among music files and video files stored in . For example, the processor 209 may select a music file and an image file corresponding to a specific period of time, such as a user's childhood, using a learning model. .

According to an embodiment, the processor 209 checks a plurality of music files and a plurality of image files matching the user's age information, and uses a model learned by a deep learning algorithm to determine the user's electroencephalogram. According to the user history information, a list is created by arranging the plurality of music files and the plurality of video files in ascending order, the electroencephalogram measurement value is converted into a parameter, and three music files and video files having high parameters are selected by the touch It may be configured to display through the screen display 211 . The user history information may refer to information about a user's electrocardiogram measurement result for each music file and video file.

According to an embodiment, the apparatus for providing images and measuring brain waves further includes a motion sensor 207 wearable by a user, and the processor 209 detects a motion of the user by using the motion sensor, A motion of a user and a motion of an object in a video may be compared, a similarity parameter may be determined based on the comparison result using a model learned by a deep learning algorithm, and the similarity parameter may be digitized and displayed. An apparatus for providing images and measuring brain waves may provide content through which a user simulates a motion of a person or a specific object in an image through an image. The processor 209 may display the similarity parameter through the touch screen display 211 by comparing how well the user imitates by the algorithm described above.

According to an embodiment, the processor 209 detects an electroencephalogram using the electroencephalogram sensor while detecting a motion using the motion sensor 207, and transmits the detected electroencephalogram together with the similarity parameter to the touch screen. It can be displayed through the display 211.

According to an embodiment, the processor 209 divides the user's motion into a plurality of partial motions, checks an electroencephalogram detection value for each partial motion, and uses the electroencephalogram detection value to determine each of the partial motions. Electroencephalogram parameters may be generated for each partial motion, and the generated electroencephalogram parameters may be stored in a memory for each partial motion.

According to an embodiment, the processor 209 confirms that the image is an image of a conductor conducting by using a model learned with a deep learning algorithm, and uses the motion sensor to determine the user's hand movement and beat. is checked and compared with the movement and tempo of the conductor's hand in the image, and the processor 209 generates a similarity parameter based on the comparison result, digitizes the similarity parameter, and displays it on the touch screen display. can

According to an embodiment, the processor 209 classifies music files and image files for each age group of the user based on the user's age information, and classifies the music file and video file based on the user's taste information stored in the memory 213. A video file is selected from music files and video files classified by age group, a list of the selected music file and video file is created and displayed through the touch screen display 211, and displayed through the touch screen display 211. It may be configured to receive a selection of each of the music file and the video file among the music file and the video file, and play the music file and the video file through the head mounted display 201 .

According to an embodiment, the processor 209 checks the user's electroencephalogram measurement results for each of the music files and the image files stored in the memory 213, and arranges the electroencephalogram measurement results in ascending order in ascending order. It can be set to create a list and store the created list in the memory 213 .

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. You can command the device. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (10)

In the image providing and brain wave measuring device, the brain wave measuring device:
head mounted display;
earphone;
touch screen display;
electroencephalogram sensor;
Memory; and
a processor operably connected to the head mounted display, the earphone, the touch screen display, and the memory;
The processor:
receiving user information through the touch screen display;
selecting a video file and a music file stored in a memory that match the information of the user and transmitting the video file to the head mounted display;
playing video files and music files using the head mounted display;
measuring the user's electroencephalogram using the electroencephalogram sensor after reproducing the video file and the music file;
Characterized in that it is set to display the electroencephalogram measurement result through the touch screen display, image providing and brain wave measuring device.
According to claim 1,
The information of the user includes age information of the user,
The processor is set to select a music file and a video file matching the age information of the user from among the music files and video files stored in the memory using a learning model learned by a deep learning algorithm. Provision and EEG measurement device.
According to claim 2,
The processor checks a plurality of music files and a plurality of video files matching the user's age information, and uses a model learned by a deep learning algorithm to determine the plurality of the plurality of images according to the user history information on the user's electroencephalogram. A music file and the plurality of video files are arranged in ascending order to create a list, convert the electroencephalogram measurement value into a parameter, and display three music files and video files with high parameters through the touch screen display. To do, video provision and EEG measurement device.
According to claim 3,
Further comprising a motion sensor wearable by a user,
The processor detects a user's motion using the motion sensor, compares the detected user's motion with the motion of an object in the video, and based on the comparison result using a model learned by a deep learning algorithm An image providing and brain wave measuring device characterized in that it is set to determine a similarity parameter and display the similarity parameter.
According to claim 4,
Wherein the processor is configured to detect a motion using the motion sensor, detect an electroencephalogram using the electroencephalogram sensor, and display the detected electroencephalogram along with the similarity parameter through the touch screen display. and an EEG measuring device.
According to claim 5,
The processor divides the motion of the user into a plurality of partial motions, checks an electroencephalogram for each partial motion, and generates an electroencephalogram parameter for each partial motion using an electroencephalogram result value, , Image providing and EEG measurement apparatus, characterized in that set to store the generated electroencephalogram parameters in memory for each operation.
According to claim 5,
The processor confirms that the video is a video in which the conductor is conducting by using a model learned with a deep learning algorithm, and uses the motion sensor to check the motion and beat of the user's hand to determine the conductor's hand in the video. compared to the movement and tempo of
Characterized in that the process is set to generate a similarity parameter based on the comparison result, digitize the similarity parameter, and display it on the touch screen display.
According to claim 5,
The processor classifies music files and video files for each age group of the user based on the user's age information, and classifies the music file and video file for each age group based on the user's taste information stored in a memory. select from among them, create a list of the selected music files and video files, and display them through a touch screen display;
characterized in that it is set to receive a selection for each of the music file and video file among the music file and video file displayed through the touch screen display, and play the music file and video file through the head mounted display, characterized in that, .
According to claim 1,
The processor checks the user's electroencephalogram measurement results for each of the music files and the image files stored in the memory, creates a list by arranging the electroencephalogram measurement results in ascending order, and stores the created list in memory. Characterized in that it is set to store, image providing and EEG measuring device.
receiving user information through a touch screen display;
selecting a video file and a music file stored in a memory that match the information of the user and transmitting the video file to the head mounted display;
playing a video file and a music file using the head mounted display;
measuring an electroencephalogram of the user by using the electroencephalogram sensor after reproducing the video file and the music file;
A method for providing images and measuring brain waves, comprising an operation of displaying the electroencephalogram measurement result through the touch screen display.

Images