KR20210103816A - System and Method for Diagnosing and Caring Mental health using EEG information and face emotion recognition technology - Google Patents

Abstract

The present invention relates to a system and method for diagnosing and taking care of mental health using brainwave information and facial expression recognition technology, which are able to efficiently take care of mental health by using the brainwave information and facial expression recognition through a camera image. The system for diagnosing and taking care of mental health using brainwave information and facial expression recognition technology comprises: a diagnostic device which simultaneously collects the brainwave of a user and an image information through a camera; a control unit which stores and manages a transaction book including personal information of the user by using a smart contract based on the blockchain technology, analyzes the collected data provided from the diagnostic device, and generates and provides a first analysis report to a medical specialist; a remote diagnosis unit which provides the first analysis report provided by the control unit and brainwave information and facial expression recognition data so that the remote medical specialist can check the information, and supports so that the medical specialist can use the remote image diagnosis function with the user as needed; an application unit which mediates so that the user and the medical specialist can consult with each other; and a contribution compensation unit which provides a voucher to a mobile phone of the user for compensating the performance contributed for the activation of the services for diagnosing and taking care of mental health to the user who uses the diagnostic device.

Description

System and Method for Diagnosing and Caring Mental health using EEG information and face emotion recognition technology

The present invention relates to mental health diagnosis and care, and specifically, for mental health diagnosis and care using brain wave information and facial expression recognition technology that enables efficient mental health management using brain wave information and facial expression recognition through camera images It relates to systems and methods.

People living in modern society are exposed to various situations, and there is a lot of psychological stress caused by this. Such stress causes mental diseases such as depression, and the resulting social problems are reaching a serious point.

For example, mental illness can be triggered in adulthood following Childhood trauma. Childhood trauma is considered a risk factor for the development of various affective disorders in childhood as well as in adulthood.

Children who have experienced trauma are at higher risk of developing depression and anxiety disorders and may be more emotionally anxious than children who have not experienced it. In such a case, continuous tracking management is required.

The youth suicide rate per 100,000 population in 2011 jumped 57.2% from 3.19 to 5.58 compared to 2001, which is the highest suicide rate among OECD countries.

In addition, adolescents with mental health problems are 13 times more likely to choose suicide than general adolescents, with 1 in 12 adolescents having a mental illness.

In the past five years, the number of adolescent mental health counseling cases has increased by 580,000, reaching about 15 million. As of 2013, there are 5,519 counseling centers nationwide, but the number of students is 6,481,492.

On the other hand, with the recent increase in the aging population, depression of not only adolescents but also the elderly is greatly increasing.

In addition, as a result of a 2016 Daegu-Gyeongsan survey, it was found that the stress awareness rate among young people in their 20s was 41.5%.

As such, mental health risks are increasing in all generations, and this increase in mental health risks is putting a burden on local governments and citizens' finances.

However, most local governments are unable to provide health services in a timely manner due to lack of budget or manpower. In addition, individual citizens do not have time or financial space to receive counseling or diagnosis, and there are many cases where they have negative preconceived notions about psychiatric treatment, so proper mental health care is not being provided.

In order to respond to these social problems related to mental health, continuous measurement of brain activity and review by professional medical staff are required in everyday and universal environments, and there is a limit in the technical field for accurate mental health diagnosis and care in the prior art.

Therefore, there is a demand for development of a new technology that enables efficient mental health management through complex and efficient mental health diagnosis and care that can increase precision.

Republic of Korea Patent Registration No. 10-2074794 Republic of Korea Patent Publication No. 10-2019-0089678 Republic of Korea Patent Publication No. 10-2016-0031124

The present invention is to solve the problems of the mental health diagnosis and care technology of the prior art, and the mental health using EEG information and facial expression recognition technology that enables efficient mental health management using facial expression recognition through EEG information and camera images An object of the present invention is to provide a system and method for health diagnosis and care.

According to the present invention, a user can measure EEG along with a facial image recording for expression recognition and analysis using a diagnostic device placed in an individual's home or multi-user facility, and a specialist at a remote location analyzes the data to provide a diagnosis opinion. An object of the present invention is to provide a system and method for mental health diagnosis and care using EEG information and expression recognition technology that can be presented to users.

The present invention provides a wearable device that a user can use for EEG measurement by self-installing according to UI instructions and guidance of a system, and a signal measured by the device that can receive the first input and transmit it to a server for quantitative analysis of EEG EEG information and facial recognition technology that enables efficient mental health management by systematically connecting and establishing a service control unit platform to quantitatively analyze user-side diagnostic devices and communication results from diagnostic devices and provide the results to remote specialists. An object of the present invention is to provide a system and method for mental health diagnosis and care.

The present invention is presented as a systematic system UI report so that a specialist who has obtained the data of the primary quantitative analysis of the user's brain wave and behavior and facial expression analysis can reflect the information in the diagnosis, and the image with the user is determined by the expert. The purpose of this is to provide a remote diagnosis department that can respond promptly, such as counseling and interviewing, and recommending additional treatment thereafter.

The present invention relates to whether the user who performed self-eEG measurement using the diagnostic device of the present invention performed an accurate action according to the measurement guideline while the specialist who will perform the mental health diagnosis is at a remote location, measured and transmitted EEG data The purpose of this is to provide both user behavior and facial expression analysis results through images, which can determine whether the data is reliable for diagnosis.

In addition, the present invention is a blockchain-based distributed ledger for transparent management of system access accounts and personal measurement data for users and remote experts who participate in this system to perform self-measurement and data extraction, data transaction management technology and It aims to provide a performance reward system based on participation.

Other objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

A system for mental health diagnosis and care using brain wave information and facial expression recognition technology according to the present invention for achieving the above object is a diagnostic device that simultaneously collects image information through a user's brain waves and a camera; A control unit that stores and manages a transaction book using a smart contract based on blockchain technology, generates a primary analysis report by analyzing the collected data provided from the diagnostic device, and provides it to a specialist; Primary analysis report provided by the control unit and a remote diagnosis unit that provides EEG information and facial expression recognition data so that a remote specialist can check it, and supports to use a remote image diagnosis function with a user if necessary; Mediation so that the user and the specialist can consult with each other and an application unit; a contribution compensation unit that pays a voucher to the user's mobile phone in order to reward the user using the diagnosis device for achievements contributed to the activation of mental health diagnosis and care services.

Here, the diagnostic device is disposed at the user's home or multi-user facility, and pulse wave measurement and brain health score, concentration, brain stress, and brain activity information to analyze the user's autonomic nerve age, stress, cumulative fatigue, and heart health. It is characterized by measuring the EEG for analysis.

And the diagnosis device is characterized in that the wearable device that can be worn by the user.

In addition, the diagnostic device includes an EEG measuring device comprising an electrode for collecting EEG of the user and a wearable measuring device for performing analog-to-digital conversion, and a camera device capable of photographing the user's behavior and face when measuring EEG information. It is characterized in that it comprises an image information acquisition unit, and brain wave information and image information transmission unit for transmitting the brain wave information and expression recognition data of the user collected through the brain wave measuring means and the image information acquisition unit to the control unit.

And the control unit, a personal information storage unit for storing personal information registered for mental health diagnosis and care using brain wave information and facial expression recognition technology, and emotion analysis that performs a primary quantitative analysis of the user's brain waves and behavior and facial expression analysis The first analysis result from the account management unit that manages the accounts of each user and expert, which consists of a smart contract based on the block chain network technology, and the original data and emotion analysis unit provided from the diagnostic device, A data and analysis result storage unit that stores the diagnosis result by Accordingly, it is characterized in that it includes a video communication support unit that supports to use a remote video diagnosis function with a user.

And the application unit, a video consultation intermediary unit that mediates so that a user and a specialist can have video consultation with each other, and a high-risk group tracking management process by the control unit when it is determined as a high-risk group according to the diagnosis result stored in the data and analysis result storage unit A high-risk group tracking management unit that provides users with medical information including additional treatment by characterized in that

In addition, by providing results of user behavior and facial expression analysis through images in parallel, the user who performed self EEG measurement using a diagnostic device in a state where a specialist who will perform mental health diagnosis through the remote diagnosis unit is at a remote location follows the measurement instructions It is characterized in that it is possible to determine whether the correct action was performed and whether the measured and transmitted EEG collected data is reliable data for diagnosis.

And the contribution compensation unit, characterized in that the voucher is also paid to the specialist.

The method for mental health diagnosis and care using brain wave information and facial expression recognition technology according to the present invention to achieve another object is to manage each user and specialist account made of a smart contract based on block chain network technology. Collecting user's EEG information and user image when there is a user registration and a request for mental health diagnosis and care service through the user; providing a diagnosis by a specialist; mediating a video consultation between the user and the specialist through the application unit when remote image diagnosis with the user is required according to the diagnosis result of the specialist; providing the diagnosis result to the user; , providing a voucher to reward users and specialists using the diagnostic device for achievements contributed to the activation of mental health diagnosis and care services;

The system and method for mental health diagnosis and care using brain wave information and facial expression recognition technology according to the present invention as described above have the following effects.

First, it enables efficient mental health management by using EEG information and facial expression recognition through camera images.

Second, the user can measure EEG along with facial image recording for expression recognition and analysis by using a diagnostic device placed in an individual's home or multi-user facility, and a specialist at a remote location analyzes the data to provide a diagnostic opinion to the user. It has the effect of increasing the convenience of the user by making it possible to present it to the user.

Third, the user can measure EEG using a remote diagnostic unit placed in an individual's home or multi-user facility and a diagnostic device equipped with it, and the system analyzes user behavior and facial expression information during such self-measurement. Thus, the primary analysis is performed and transmitted to the specialist by the platform control unit, so that one's mind can be diagnosed more freely.

Fourth, since the user's EEG information is transmitted to a specialist and the analysis result is transmitted only to the user, even a user with a preconceived notion in mental treatment can receive a diagnosis when necessary.

Fifth, by providing performance reward vouchers using block chain technology to users and specialists, it is possible to increase the utility of the mental diagnosis system.

1 is an overall configuration diagram of a system for mental health diagnosis and care using EEG information and facial expression recognition technology according to the present invention;
2 is a detailed configuration diagram of a diagnostic device according to the present invention;
3 is a detailed configuration diagram of a control unit according to the present invention;
4 is a detailed configuration diagram of the application unit;
5 is a flowchart illustrating a method for mental health diagnosis and care using brain wave information and facial expression recognition technology according to the present invention;

Hereinafter, preferred embodiments of a system and method for mental health diagnosis and care using EEG information and facial expression recognition technology according to the present invention will be described in detail as follows.

Features and advantages of the system and method for mental health diagnosis and care using EEG information and facial expression recognition technology according to the present invention will become apparent through detailed description of each embodiment below.

1 is an overall configuration diagram of a system for mental health diagnosis and care using brain wave information and facial expression recognition technology according to the present invention.

The system and method for mental health diagnosis and care using EEG information and facial expression recognition technology according to the present invention enable efficient mental health management by using EEG information and facial expression recognition through camera images.

To this end, the present invention allows a user to measure EEG along with a facial image recording for expression recognition and analysis by using a diagnostic device placed in an individual's home or multi-user facility, and a specialist at a remote location analyzes the data for diagnosis It may include a configuration that allows comments to be presented to the user.

The present invention provides a wearable device that a user can use for EEG measurement by self-installing according to UI instructions and guidance of a system, and a signal measured by the device that can receive the first input and transmit it to a server for quantitative analysis of EEG It may include a configuration for systematically connecting and establishing a service control unit platform for quantitatively analyzing a user-side diagnostic device and a communication result from the diagnostic device and providing the result to a remote specialist.

The present invention is presented as a systematic system UI report so that a specialist who has obtained the data of the primary quantitative analysis of the user's brain wave and behavior and facial expression analysis can reflect the information in the diagnosis, and the image with the user is determined by the expert. It may include a remote diagnosis unit that can respond promptly, such as counseling and interviewing, and recommending additional treatment thereafter.

The present invention relates to whether the user who performed self EEG measurement using a diagnostic device performed an accurate action according to the measurement guideline while the specialist who will perform the mental health diagnosis is at a remote location, and whether the measured and transmitted EEG collected data is used for diagnosis. It may include a configuration that provides a result of analyzing user behavior and facial expression through an image, which can determine whether the data is reliable or not.

In addition, it is a blockchain-based distributed ledger for transparent management of system access accounts and personal measurement data for users and remote specialists who participate in the system according to the present invention and perform self-measurement and data extraction. It may include a configuration that provides a performance-based reward system for participation.

EEG measurement used in the present invention is to detect an electrical signal generated as a result of brain activity through a non-invasive method in order to understand the trend of brain activity, and to analyze the electrical signal measured by attaching an electrode to the scalp. Collectively, it refers to obtaining various indicators of brain activity and mental health.

At this time, according to the range of oscillating frequencies, the typical human brain waves are conventionally delta (δ) waves (0.2 to 3.99 Hz), theta (θ) waves (4 to 7.99 Hz), alpha (α) waves (8 to 12.99 Hz), beta (β) wave (13 ~ 29.99 Hz), and gamma (g) wave (30 ~ 50 Hz) are called qEEG technology, and analysis through various quantitative numerical analysis of these frequency band signals is collectively called qEEG technology. is called

And the expression recognition technology recognizes the user's head and face area through digital image processing technology from a digital image or digital moving image taken using a camera with a normal function, extracts feature points, detects the user's behavior, and visually It refers to the art of recognizing manifest mental and emotional states.

The system for mental health diagnosis and care using EEG information and expression recognition technology according to the present invention includes a diagnostic device 120 that provides EEG information and expression recognition data of a user to the control unit 110 as shown in FIG. 1 and , the remote diagnosis unit 130 by storing and managing the user's personal information using a smart contract based on block chain technology, and analyzing the collected data provided from the diagnosis device 120 to generate a primary analysis report. The control unit 110 provided to the specialist through A remote diagnosis unit 130 that supports the use of an image diagnosis function, an application unit 140 that mediates so that the user and the specialist can consult each other, and a mental health diagnosis for a user using the diagnosis device 120 and a contribution compensation unit 150 for providing a voucher to the user's mobile phone in order to compensate for the performance contributed to the activation of the care service.

Here, the diagnostic device 120 is disposed in the user's home or multi-user facility including a school, subway, and government building, measures the user's brain waves, and at this time, image data for state analysis through the user's actions and facial expressions record

The diagnostic device 120 may measure a pulse wave measurement for analyzing the user's autonomic nerve age, stress, cumulative fatigue, and heart health and an EEG for analyzing brain health score, concentration, brain stress, and brain activity information. , but not limited thereto.

The detailed configuration of the diagnostic device 120 is as follows.

2 is a detailed configuration diagram of a diagnostic device according to the present invention.

The diagnostic device 120 includes an EEG measuring means 21 comprising electrodes for collecting EEG of a user and a wearable measuring device that performs analog-to-digital conversion, and a camera capable of photographing a user's behavior and face when measuring EEG information. Brain wave information that transmits the user's brain wave information and facial expression recognition data collected through the image information acquisition unit 22 having the device, the brain wave measuring means 21 and the image information acquisition unit 22 to the control unit 110 . and an image information transmission unit 23 .

And the contribution compensation unit 150 remotely checks the primary analysis report and EEG information and expression recognition data provided by the control unit 110, and provides a voucher to the specialist who performs video consultation and diagnoses.

And the control unit 110 is equipped with an algorithm for qEEG analysis of the user's basic measurement data, emotion analysis through action, concentration, and facial expression through image information, and consists of a smart contract based on block chain network technology. It is to manage each user's and specialist's account, system access and data distribution, data source and storage of analysis results, etc.

In addition, the control unit 110 includes a video communication server function that mediates so that the user and the specialist can consult each other through an image.

The detailed configuration of the control unit 110 is as follows.

3 is a detailed configuration diagram of a control unit according to the present invention.

The control unit 110 includes a personal information storage unit 31 that stores personal information registered for mental health diagnosis and care using brain wave information and facial expression recognition technology, as shown in FIG. 3, and the user's brain waves and actions and facial expressions. The emotion analysis unit 32 that performs the primary quantitative analysis of the analysis, the account management unit 33 that manages the accounts of each user and the expert composed of a smart contract based on block chain network technology, and a diagnostic device ( 120) and the first analysis result in the emotion analysis unit 32, the data and analysis result storage unit 34 for storing the diagnosis result by the specialist, and the 1 analyzed by the emotion analysis unit 32 The primary analysis report providing unit 35 that provides the difference analysis report so that the specialist can check it through the remote diagnosis unit 130, and the image that supports the use of the remote image diagnosis function with the user according to the diagnosis result of the specialist and a communication support unit 36 .

And the detailed configuration of the application unit 140 is as follows.

4 is a detailed configuration diagram of the application unit.

The application unit 140 is a high-risk group according to the diagnosis result stored in the video consultation intermediary unit 41 and the data and analysis result storage unit 34, which mediate so that the user and the specialist can have video consultation with each other, as shown in FIG. 4 . Mental health diagnosis through the high-risk group tracking management unit 42 and the contribution compensation unit 150 that provides medical information including additional treatment by the high-risk group tracking management process by the control unit 110 to the user when it is determined as It includes a contribution compensation management unit 43 that supports to check and manage performance compensation results that contributed to the activation of the care service.

A method for mental health diagnosis and care using EEG information and expression recognition technology according to the present invention will be described in detail as follows.

5 is a flowchart illustrating a method for mental health diagnosis and care using brain wave information and facial expression recognition technology according to the present invention.

In case of user registration and mental health diagnosis and care service request through management of each user and specialist account made of a smart contract based on blockchain network technology, first, electrodes and analogs that collect the user's brain waves - A diagnostic device 120 having an EEG measuring means 21 composed of a wearable measuring device that performs digital conversion and an image information obtaining means having a camera device capable of photographing a user's behavior and face when measuring EEG information Collecting the user's brain wave information and the user image (S501) is performed.

Next, the remote diagnosis unit 130 provides the primary analysis report, EEG information, and facial expression recognition data so that the remote specialist can confirm the diagnosis by the specialist ( S502 ).

In addition, when remote image diagnosis with the user is required according to the diagnosis result of the specialist, a step S503 of mediating between the user and the specialist through the application unit 140 is performed so that the user and the specialist can have video consultation with each other (S503).

Next, a step (S504) of providing a diagnosis result to the user and providing a voucher to the user and the specialist using the diagnosis device 120 is performed in order to compensate the performance contributing to the activation of the mental health diagnosis and care service (S504).

The system and method for mental health diagnosis and care using EEG information and facial expression recognition technology according to the present invention described above is a system and method for a user to recognize and analyze a face by using a diagnostic device disposed in an individual's home or multi-user facility. It is possible to measure EEG along with video recording, and a specialist at a remote location analyzes the data to present a diagnostic opinion to the user.

It will be understood that the present invention is implemented in a modified form without departing from the essential characteristics of the present invention as described above.

Therefore, the specified embodiments are to be considered in an illustrative rather than a restrictive sense, the scope of the present invention is indicated in the claims rather than in the foregoing description, and all differences within the scope equivalent thereto are included in the present invention. will have to be interpreted.

110. Control Department
120. Diagnostics
130. Remote Diagnosis Department
140. Application Department
150. Contribution Compensation Department

Claims (9)

a diagnostic device that simultaneously collects the user's brain waves and image information through the camera;
a control unit that stores and manages a transaction book using a smart contract based on block chain technology for the user's personal information, analyzes the collected data provided from the diagnostic device, generates a primary analysis report, and provides it to a specialist;
a remote diagnosis unit that provides a primary analysis report provided by the control unit, EEG information, and facial expression recognition data so that a remote specialist can check it, and supports remote image diagnosis with a user if necessary;
an application unit for intermediary so that the user and the specialist can consult with each other;
EEG information and facial expression recognition technology, characterized in that it comprises; a contribution compensation unit that pays a voucher to the user's mobile phone in order to compensate the user using the diagnosis device for the achievement of contributing to the activation of mental health diagnosis and care service A system for mental health diagnosis and care using. The method of claim 1, wherein the diagnostic device is disposed in a user's home or a multi-user facility,
EEG information and expression recognition, characterized by measuring pulse waves to analyze the user's autonomic nerve age, stress, cumulative fatigue, and heart health and measuring brain waves to analyze brain health score, concentration, brain stress, and brain activity information A system for mental health diagnosis and care using technology. The system for mental health diagnosis and care using brain wave information and facial expression recognition technology according to claim 1, wherein the diagnosis device is a wearable device that a user can wear. The method of claim 1, 2, or 3, wherein the diagnostic device comprises:
An EEG measuring means comprising an electrode that collects the user's EEG and a wearable measuring device that performs analog-to-digital conversion;
An image information acquisition unit having a camera device capable of photographing a user's behavior and face when measuring EEG information;
Mental health diagnosis and care using brain wave information and facial expression recognition technology, characterized in that it includes a brain wave information and image information transmission unit that transmits the user's brain wave information and expression recognition data collected through the EEG measuring means and the image information acquisition unit to the control unit system for. According to claim 1, wherein the control unit,
A personal information storage unit for storing personal information registered for mental health diagnosis and care using brain wave information and facial expression recognition technology;
An emotion analysis unit that performs a primary quantitative analysis of the user's brain waves and behavior and facial expression analysis;
An account management unit that manages each user and professional account made of a smart contract based on blockchain network technology;
A data and analysis result storage unit for storing the original data provided from the diagnostic device, the primary analysis result in the emotion analysis unit, and the diagnosis result by a specialist;
A primary analysis report providing unit that provides the primary analysis report analyzed by the emotion analysis unit so that a specialist can check it through the remote diagnosis unit;
A system for mental health diagnosis and care using EEG information and facial expression recognition technology, characterized in that it includes a video communication support unit that supports to use a remote image diagnosis function with a user according to a diagnosis result of a specialist. According to claim 1, wherein the application unit,
A video consultation intermediary unit that mediates so that users and specialists can have video consultations with each other;
A high-risk group tracking management unit that provides the user with medical information including additional treatment by the high-risk group follow-up management process by the control unit when it is determined as a high-risk group according to the diagnosis result stored in the data and analysis result storage unit;
For mental health diagnosis and care using EEG information and expression recognition technology, characterized in that it includes a contribution compensation management unit that supports to check and manage the performance compensation results that contributed to the activation of mental health diagnosis and care services through the contribution compensation unit system. According to claim 1, wherein the user behavior through the image and facial expression analysis results are provided in parallel,
In a state where a specialist who will perform mental health diagnosis through the remote diagnosis unit is at a remote location, the measured and transmitted EEG data is diagnosed as to whether the user who performed self EEG measurement using a diagnostic device performed an accurate action according to the measurement guideline A system for mental health diagnosis and care using brain wave information and facial expression recognition technology, characterized in that it can be determined whether it is reliable data for The method of claim 1, wherein the contribution compensation unit,
A system for mental health diagnosis and care using EEG information and facial expression recognition technology, characterized in that a voucher is also provided to the specialist. When there is a request for user registration and mental health diagnosis and care service through the management of each user and specialist account made of a smart contract based on blockchain network technology,
collecting user's brain wave information and user image;
receiving a diagnosis by a specialist by providing the remote diagnosis unit with the primary analysis report, EEG information, and expression recognition data so that the remote specialist can check;
mediating a video consultation between the user and the specialist through an application unit when remote image diagnosis with the user is required according to the diagnosis result of the specialist;
Providing a diagnosis result to a user and providing a voucher to reward a user and a specialist using the diagnosis device for achievements contributed to the activation of mental health diagnosis and care services; EEG information and facial expression recognition technology comprising the methods for mental health diagnosis and care.

Images