KR20230161183A - Apparatus and method for determining mental health - Google Patents

Abstract

A mental health determination device and method are disclosed. A mental health determination device according to one aspect includes a camera that acquires a facial image of a subject; a parameter acquisition unit configured to acquire a plurality of heart rate variability parameters for the subject by analyzing the acquired facial image; and selecting at least one heart rate variability parameter among the obtained plurality of heart rate variability parameters according to the type of mental health to be determined, and using the selected at least one heart rate variability parameter and a machine learning-based mental health judgment model to determine the subject's mental health. a mental health judgment department that judges mental health; may include.

Description

Apparatus and method for determining mental health}

It is related to technology that determines mental health using heart rate variability parameters.

In modern times, people's physical health has improved, but on the contrary, their mental health has worsened, and the incidence of various mental diseases such as depression, anxiety, and insomnia is increasing.

Due to its nature, the treatment of these mental illnesses requires continuous management and treatment rather than short-term treatment. However, most people are unable to receive appropriate diagnosis and treatment due to reasons such as accessibility, cost issues, negative perceptions, or lack of awareness of mental illness. .

In addition, treatment and management of mental illness is limited to on-site consultations with specialists, and there is a problem in that changes in emotions and conditions in the daily lives of mentally ill patients are not identified and continuously diagnosed and/or treated in response.

Therefore, there is a need for a method to conveniently and accurately diagnose and manage mental health without receiving treatment from a specialist.

The purpose is to provide a device and method for determining mental health from heart rate variability parameters using machine learning.

A mental health determination device according to one aspect includes a camera that acquires a facial image of a subject; a parameter acquisition unit configured to acquire a plurality of heart rate variability parameters for the subject by analyzing the acquired facial image; and selecting at least one heart rate variability parameter among the obtained plurality of heart rate variability parameters according to the type of mental health to be determined, and using the selected at least one heart rate variability parameter and a machine learning-based mental health judgment model to determine the subject's mental health. a mental health judgment department that judges mental health; may include.

The plurality of heart rate variability parameters may include Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, InHF, LF(%), HF(%), and LF/HF ratio. there is.

The mental health includes depression, anxiety, insomnia, stress, and cognitive impairment, and the mental health judgment model may include a depression judgment model, an anxiety judgment model, an insomnia judgment model, a stress judgment model, and a cognitive disorder judgment model.

If the mental health to be judged is depression, the mental health determination unit determines Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, InHF, LF (%) among the plurality of heart rate variability parameters. ), HF (%) and LF/HF ratio can be selected.

If the mental health to be judged is anxiety, the mental health determination unit may select Mean HR, SDNN, RMSSD, pNN50, TP, LF, HF, InTP, InVLF, InLF, and InHF from among the plurality of heart rate variability parameters.

If the mental health to be judged is insomnia, the mental health determination unit may select Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, and InHF from among the plurality of heart rate variability parameters. .

If the mental health to be judged is stress, the mental health determination unit may select Mean HR, SDNN, RMSSD, TP, VLF, LF, HF, InTP, and InHF from the plurality of heart rate variability parameters.

If the mental health to be judged is cognitive impairment, the mental health determination unit can select Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, and InHF among the plurality of heart rate variability parameters. there is.

A mental health determination method according to another aspect includes obtaining a facial image of a subject; acquiring a plurality of heart rate variability parameters for the subject by analyzing the acquired facial image; Selecting at least one heart rate variability parameter among the obtained plurality of heart rate variability parameters according to the type of mental health to be determined; And determining the mental health of the subject using the selected at least one heart rate variability parameter and a machine learning-based mental health judgment model; may include.

The plurality of heart rate variability parameters may include Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, InHF, LF(%), HF(%), and LF/HF ratio. there is.

The mental health includes depression, anxiety, insomnia, stress, and cognitive impairment, and the mental health judgment model may include a depression judgment model, an anxiety judgment model, an insomnia judgment model, a stress judgment model, and a cognitive disorder judgment model.

In the step of selecting the at least one heart rate variability parameter, if the mental health to be determined is depression, Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, InHF, LF(%), HF(%) and LF/HF ratio can be selected.

In the step of selecting at least one heart rate variability parameter, if the mental health to be determined is anxiety, Mean HR, SDNN, RMSSD, pNN50, TP, LF, HF, InTP, InVLF, InLF and InHF can be selected.

In the step of selecting at least one heart rate variability parameter, if the mental health to be determined is insomnia, Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF and InHF can be selected.

In the step of selecting at least one heart rate variability parameter, if the mental health to be determined is stress, Mean HR, SDNN, RMSSD, TP, VLF, LF, HF, InTP, and InHF can be selected among the plurality of heart rate variability parameters. there is.

The step of selecting at least one heart rate variability parameter includes, if the mental health to be determined is cognitive impairment, Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, and InVLF among the plurality of heart rate variability parameters. , InLF and InHF can be selected.

Multiple heart rate variability parameters can be acquired from the subject's facial image, and the subject's mental health can be conveniently and accurately determined using the obtained multiple heart rate variability parameters and a machine learning-based mental health judgment model.

1 is a diagram illustrating a mental health determination system according to an exemplary embodiment.
FIG. 2 is a diagram illustrating an apparatus for generating a mental health determination model according to an exemplary embodiment.
Figure 3 is a diagram illustrating a mental health determination device according to an exemplary embodiment.
Figure 4 is a diagram illustrating a mental health determination device according to another embodiment.
Figure 5 is a diagram illustrating a method for generating a mental health judgment model according to an exemplary embodiment.
Figure 6 is a diagram illustrating a method for determining mental health according to an exemplary embodiment.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the attached drawings. When adding reference numerals to components in each drawing, it should be noted that identical components are given the same reference numerals as much as possible even if they are shown in different drawings. Additionally, in describing the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

Meanwhile, in each step, unless a specific order is clearly stated in the context, each step may occur in a different order from the specified order. That is, each step may be performed in the same order as specified, may be performed substantially simultaneously, or may be performed in the opposite order.

The terms described below are terms defined in consideration of functions in the present invention, and may vary depending on the intention or custom of the user or operator. Therefore, the definition should be made based on the contents throughout this specification.

Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. Terms are used only to distinguish one component from another. Singular expressions include plural expressions unless the context clearly indicates otherwise, and terms such as 'include' or 'have' refer to the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification. It is intended to specify that something exists, but it should be understood as not precluding the possibility of the existence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

In addition, the division of components in this specification is merely a division according to the main function each component is responsible for. That is, two or more components may be combined into one component, or one component may be divided into two or more components for more detailed functions. In addition to the main functions that each component is responsible for, each component may additionally perform some or all of the functions that other components are responsible for, and some of the main functions that each component is responsible for may be performed by other components. It may also be carried out. Each component may be implemented as hardware or software, or as a combination of hardware and software.

FIG. 1 is a diagram illustrating a mental health determination system according to an exemplary embodiment, FIG. 2 is a diagram illustrating a mental health determination model generating device according to an exemplary embodiment, and FIG. 3 is a diagram illustrating a mental health determination model according to an exemplary embodiment. This is a diagram showing a health determination device.

Referring to FIG. 1 , the mental health determination system 100 according to an exemplary embodiment may include a mental health determination model generating apparatus 110 and a mental health determination apparatus 120 .

The mental health judgment model generating device 110 may generate a mental health judgment model using a machine learning algorithm. At this time, the machine learning algorithm is a tree-based machine learning algorithm (e.g., Random Forest, XGboost (Extreme Gradient Boost), ADAboost (Adaptive Boost), Light GBM (Light Gradient Boost Machine), C5.0, ID3, C4.5, CART etc.), deep learning algorithm, K-nearest neighbor algorithm, Naive Bayes Classification algorithm, Neural Networks algorithm, Support Vector Machines, etc., but is not limited thereto. .

As shown in FIG. 2, the mental health judgment model generating device 110 may include a learning data collection unit 210 and a model generating unit 220.

The learning data collection unit 210 may collect learning data to be used to create a mental health judgment model.

According to an exemplary embodiment, the learning data collection unit 210 may collect a plurality of heart rate variability (HRV) parameters for various people and corresponding mental health diagnosis results from an external device as learning data. there is. Here, mental health includes depression, anxiety, insomnia, stress, and cognitive disorders, and multiple heart rate variability parameters include Mean HR (mean of heart rate), SDNN (the standard deviation of the NN interval), and RMSSD (the square root). of the mean squared differences of successive NN interval), pNN50(NN50/total number of all NN), TP(total power), VLF(power in very low frequency range), LF(power in low frequency range), HF(power) in high frequency range), InTP(log scale of total power), InVLF(log scale of VLF), InLF(log scale of LF), InHF(log scale of HF), LF(%), HF(%), LF /HF ratio, etc. may be included.

The learning data collection unit 210 may use various wired and wireless communication technologies to acquire learning data from an external device. The external device may be, but is not limited to, a device or server that stores electronic health records (eg, electronic health record (EHR) or electronic medical record (EMR)). In addition, wired communication can use twisted pair cables, coaxial cables, optical fiber cables, and ethernet cables, and wireless communication can use Bluetooth communication, BLE (Bluetooth Low Energy) communication, and short-range wireless. Communication (Near Field Communication, NFC), WLAN communication, Zigbee communication, Infrared Data Association (IrDA) communication, WFD (Wi-Fi Direct) communication, UWB (ultra-wideband) communication, Ant+ communication, WIFI communication , RFID (Radio Frequency Identification) communication, 3G communication, 4G communication, and 5G communication can be used, but are not limited to these.

If missing values exist in the collected heart rate variability parameters, the learning data collection unit 210 may process the missing values. According to one embodiment, the learning data collection unit 210 processes missing values using various methods, such as a method of deleting missing values, a method of imputing missing values, and a method of predicting missing values with a data set composed of variables without missing values. can do.

The model generator 220 may generate a mental health judgment model based on the collected learning data.

Specifically, the model generator 220 may generate a mental health judgment model by learning a machine learning model based on a plurality of heart rate variability parameters for various people and the corresponding mental health diagnosis results. At this time, the machine learning model is tree-based machine learning model (e.g., Random Forest, , neural networks, support vector machines, etc., but are not limited thereto.

According to an exemplary embodiment, the mental health judgment model may include a depression judgment model, an anxiety judgment model, an insomnia judgment model, a stress judgment model, and a cognitive disorder judgment model, and the model generator 220 may generate a mental health judgment model to be generated. Depending on the type of judgment model, all or part of a plurality of heart rate variability parameters can be selected and used.

For example, the model creation unit 220 is Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, InHF, LF(%), HF(%), LF/HF ratio A depression judgment model can be created by learning 15 heart rate variability parameters, including , and the corresponding depression diagnosis results.

For another example, the model generator 220 generates 11 heart rate variability parameters including Mean HR, SDNN, RMSSD, pNN50, TP, LF, HF, InTP, InVLF, InLF, and InHF, and anxiety diagnosis corresponding thereto. An anxiety judgment model can be created by learning the results.

For another example, the model generator 220 provides 12 heart rate variability parameters including Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, and InHF, and corresponding An insomnia judgment model can be created by learning the insomnia diagnosis results.

For another example, the model generator 220 learns nine heart rate variability parameters including Mean HR, SDNN, RMSSD, TP, VLF, LF, HF, InTP, and InHF, and the corresponding stress diagnosis results. You can create a stress judgment model by doing this.

For another example, the model generator 220 provides 12 heart rate variability parameters including Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, and InHF, and corresponding A mental health judgment model can be created by learning the cognitive disorder diagnosis results.

The mental health judgment model generated by the model generator 220 may be stored in the internal or external memory of the mental health judgment model generating device 110.

According to an exemplary embodiment, the mental health judgment model generating device 110 may update the mental health judgment model according to a user's command or periodically. For example, the learning data collection unit 210 may collect new learning data according to the user's command or periodically, and the model creation unit 220 may add a mental health judgment model using the collected new learning data. The mental health judgment model can be updated by learning.

The mental health determination device 120 may determine the mental health of the subject using a plurality of heart rate variability parameters of the subject and a mental health judgment model. Here, the mental health judgment model is a mental health judgment model generated by the mental health judgment model generating device 110 and may include a depression judgment model, an anxiety judgment model, an insomnia judgment model, a stress judgment model, and a cognitive disorder judgment model.

The mental health determination device 120 may be mounted on an electronic device or implemented as a separate device. Here, electronic devices may include cart-type devices and portable devices, and portable devices include mobile phones, smartphones, tablets, laptops, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), navigation devices, MP3 players, and digital devices. It may include cameras, wearable devices, etc. Wearable devices include glasses type, wrist watch type, wrist band type, ring type, earring type, belt type, necklace type, ankle band type, thigh band type, forearm band type, etc. can do. However, portable devices are not limited to the above-described examples, and wearable devices are also not limited to the above-described examples.

As shown in FIG. 3 , the mental health determination device 120 may include a camera 310, a parameter acquisition unit 320, and a mental health determination unit 330.

The camera 310 may obtain an image of the subject's face by photographing the subject's face. Here, the camera 310 may be a video camera.

The parameter acquisition unit 320 may acquire a plurality of heart rate variability parameters for the subject by analyzing the acquired facial image. Here, the plurality of heart rate variability parameters may include Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, InHF, LF(%), HF(%), LF/HF ratio, etc. You can.

According to an exemplary embodiment, the parameter acquisition unit 320 extracts a facial fine movement signal and/or facial color change signal due to heartbeat from the subject's facial image, and extracts the extracted facial fine movement signal and/or facial color change signal. Using this method, multiple heart rate variability parameters can be obtained for the subject. At this time, the parameter acquisition unit 320 may use various known techniques for acquiring heart rate variability parameters from the subject's facial image. For example, technologies such as registered patent 10-1706739, registered patent 10-2150635, and registered patent 10-2215557 can be used, but this is only an example and is not limited thereto.

The mental health determination unit 330 determines the type of mental health to be determined according to the user's command, selects at least one heart rate variability parameter among a plurality of heart rate variability parameters according to the determined mental health type, and determines the type of mental health to be determined according to the user's command. The mental health of a subject can be determined using the variability parameter and the mental health judgment model. At this time

For example, if the mental health to be judged is depression, the mental health determination unit 330 selects Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, among a plurality of heart rate variability parameters. Select 15 heart rate variability parameters including InHF, LF (%), HF (%), and LF/HF ratio, and determine whether the subject is depressed using the 15 selected heart rate variability parameters and the depression judgment model. can do.

For another example, if the mental health to be judged is anxiety, the mental health determination unit 330 selects Mean HR, SDNN, RMSSD, pNN50, TP, LF, HF, InTP, InVLF, InLF, and InHF among a plurality of heart rate variability parameters. You can select 11 heart rate variability parameters including and determine whether the subject has anxiety using the 11 selected heart rate variability parameters and the anxiety judgment model.

For another example, if the mental health to be judged is insomnia, the mental health determination unit 330 selects Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, and You can select 12 heart rate variability parameters, including InLF and InHF, and determine whether the subject has insomnia using the 12 selected heart rate variability parameters and the insomnia judgment model.

For another example, if the mental health to be judged is stress, the mental health determination unit 330 includes Mean HR, SDNN, RMSSD, TP, VLF, LF, HF, InTP, and InHF among a plurality of heart rate variability parameters. Nine heart rate variability parameters can be selected, and the presence or absence of stress for the subject can be determined using the selected nine heart rate variability parameters and the stress judgment model.

For another example, if the mental health to be judged is cognitive impairment, the mental health determination unit 330 selects Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, and InVLF among a plurality of heart rate variability parameters. , InLF, and InHF, 12 heart rate variability parameters can be selected, and the subject's cognitive impairment can be determined using the 12 selected heart rate variability parameters and the cognitive impairment determination model.

According to an exemplary embodiment, the mental health determination unit 330 may generate a graphic chart to explain the reason for the mental health determination result and output it through an output means. At this time, the graphic chart may express the position of the value of the subject's heart rate variability parameter in the heart rate variability parameter distribution or may express the role of each heart rate variability parameter in the judgment process of the mental health judgment model. For example, graphic charts may include gauge charts, radar charts, and Shapley Additive Explanations (SHAP) charts.

Figure 4 is a diagram illustrating a mental health determination device according to another embodiment. The mental health determination device 400 of FIG. 4 may be another embodiment of the mental health determination device 120 of FIG. 1 .

Referring to FIG. 4, the mental health determination device 400 includes a camera 310, a parameter acquisition unit 320, a mental health determination unit 330, an input unit 410, a storage unit 420, a communication unit 430, It may include an output unit 440 and a model update unit 450. Here, since the camera 310, the parameter acquisition unit 320, and the mental health determination unit 330 are the same as described above with reference to FIG. 3, their detailed description will be omitted.

The input unit 410 can receive various manipulation signals from the user. Additionally, the input unit 410 may receive input of the subject's actual mental health status as feedback.

According to one embodiment, the input unit 410 includes a key pad, dome switch, mouse, touch pad, jog wheel, jog switch, hardware or It may include software buttons, etc. In particular, when the touch pad forms a layered structure with the display, it can be called a touch screen.

The storage unit 420 may store programs or commands for operating the mental health determination device 400, and may store data input/output to the mental health determination device 400 and processed data. For example, the storage unit 420 uses the mental health judgment model generated through the mental health judgment model generation neglect 110, the subject's facial image data acquired through the camera 310, and the parameter acquisition unit 320. A plurality of heart rate variability parameters for the acquired subject, the mental health determination result of the subject determined through the mental health determination unit 330, etc. may be stored.

The storage unit 420 is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), and RAM. (Random Access Memory, RAM), SRAM (Static Random Access Memory), ROM (Read Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read Only Memory), PROM (Programmable Read Only Memory), magnetic memory, magnetic disk, optical disk. It may include at least one type of storage medium, etc. Additionally, the mental health determination device 400 may operate an external storage medium such as web storage that performs the storage function of the storage unit 420 on the Internet.

The communication unit 430 can communicate with an external device. For example, the communication unit 430 may transmit data input to the mental health determination device 400, stored data, and processed data to an external device, or receive various data to be used to determine the mental health of a subject from an external device. You can. At this time, the external device may include a medical server, medical device, portable device (eg, smartphone, tablet, wearable device, etc.), display device, etc.

The communication unit 430 can communicate with an external device using wired or wireless communication technology. At this time, wired communication can use twisted pair cable, coaxial cable, optical fiber cable, Ethernet cable, etc., and wireless communication can use Bluetooth communication, BLE communication, short-range wireless communication, WLAN communication, Zigbee communication, infrared communication, WFD communication, UWB communication, Ant+ communication, WIFI communication, RFID communication, 3G communication, 4G communication, and 5G communication can be used. However, these are only examples and are not limited thereto.

The output unit 440 may output data input to the mental health determination device 400, stored data, processed data, etc. According to one embodiment, the output unit 440 includes data about the mental health judgment model generated through the mental health judgment model generation neglect 110, facial image data of the subject acquired through the camera 310, and a parameter acquisition unit. A plurality of heart rate variability parameters for the subject acquired through 320, the mental health judgment result of the subject determined through the mental health determination unit 330, etc., using at least one of an auditory method, a visual method, and a tactile method. Can be printed. To this end, the output unit 440 may include a display, speaker, vibrator, etc.

If the mental health determination result of the mental health determination unit 330 and the actual mental health status of the subject input through the input unit 410 are different, the model update unit 450 The previously stored mental health judgment model can be updated based on health status. For example, the model update unit 450 uses at least one of the plurality of heart rate variability parameters acquired through the parameter acquisition unit 320 and the subject's actual mental health status input through the input unit 410 as new learning data, The previously stored mental health judgment model can be additionally trained. Accordingly, the judgment accuracy of the mental health judgment model can be improved.

Figure 5 is a diagram illustrating a method for generating a mental health judgment model according to an exemplary embodiment. The method for generating a mental health judgment model according to the exemplary embodiment of FIG. 5 may be performed by the mental health judgment model generating apparatus 110 of FIG. 1 .

Referring to FIG. 5, the mental health judgment model generating device may collect learning data to be used to create a mental health judgment model (510).

According to an exemplary embodiment, the mental health judgment model generating device may collect a plurality of heart rate variability parameters for various people and corresponding mental health diagnosis results as learning data from an external device.

If missing values exist in the collected heart rate variability parameters, the mental health judgment model generating device may process the missing values. According to one embodiment, the mental health judgment model generating device may process missing values using various methods, such as a method of deleting missing values, a method of imputing missing values, and a method of predicting missing values with a data set composed of variables without missing values. You can.

The mental health judgment model generating device may generate a mental health judgment model based on the collected learning data (520).

The mental health judgment model generating device can generate a mental health judgment model by learning a machine learning model based on a plurality of heart rate variability parameters for various people and the corresponding mental health diagnosis results.

According to an exemplary embodiment, the mental health judgment model may include a depression judgment model, an anxiety judgment model, an insomnia judgment model, a stress judgment model, and a cognitive disorder judgment model, and the mental health judgment model generating device may be configured to generate a mental health judgment model. Depending on the type of judgment model, all or part of a plurality of heart rate variability parameters can be selected and used.

For example, the mental health judgment model generation device includes Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, InHF, LF(%), HF(%), LF/HF ratio A depression judgment model can be created by learning 15 heart rate variability parameters, including , and the corresponding depression diagnosis results.

As another example, the mental health judgment model generation device uses 11 heart rate variability parameters including Mean HR, SDNN, RMSSD, pNN50, TP, LF, HF, InTP, InVLF, InLF, and InHF, and the corresponding anxiety diagnosis. An anxiety judgment model can be created by learning the results.

As another example, the mental health judgment model generation device has 12 heart rate variability parameters including Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, and InHF, and corresponding An insomnia judgment model can be created by learning the insomnia diagnosis results.

As another example, the mental health judgment model generation device learns nine heart rate variability parameters including Mean HR, SDNN, RMSSD, TP, VLF, LF, HF, InTP, and InHF, and the corresponding stress diagnosis results. You can create a stress judgment model by doing this.

As another example, the mental health judgment model generation device has 12 heart rate variability parameters including Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, and InHF, and corresponding A mental health judgment model can be created by learning the cognitive disorder diagnosis results.

According to an exemplary embodiment, the mental health judgment model generating device may update the mental health judgment model according to a user's command or periodically. For example, the mental health judgment model generating device can collect new learning data according to the user's command or periodically, and update the mental health judgment model by additionally training the mental health judgment model using the collected new learning data. can do.

Figure 6 is a diagram illustrating a method for determining mental health according to an exemplary embodiment. The mental health determination method according to the exemplary embodiment of FIG. 6 may be performed by the mental health determination device 120 of FIG. 1 .

Referring to FIG. 6, the mental health determination device may acquire a facial image of the subject by photographing the subject's face (610).

The mental health determination device may acquire a plurality of heart rate variability parameters for the subject by analyzing the acquired facial image (620). Here, the plurality of heart rate variability parameters may include Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, InHF, LF(%), HF(%), LF/HF ratio, etc. You can.

According to an exemplary embodiment, the mental health determination device extracts a facial fine movement signal and/or facial color change signal due to heartbeat from the subject's facial image, and uses the extracted facial fine motion signal and/or facial color change signal. Multiple heart rate variability parameters for a subject can be obtained.

The mental health determination device may select at least one heart rate variability parameter from a plurality of heart rate variability parameters according to the type of mental health to be determined (630).

For example, if the mental health to be judged is depression, the mental health judgment device selects Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, InHF, LF among multiple heart rate variability parameters. You can select 15 heart rate variability parameters including (%), HF (%), and LF/HF ratio.

For another example, if the mental health to be judged is anxiety, the mental health judgment device includes Mean HR, SDNN, RMSSD, pNN50, TP, LF, HF, InTP, InVLF, InLF, and InHF among a plurality of heart rate variability parameters. There are 11 heart rate variability parameters to choose from.

For another example, if the mental health to be judged is insomnia, the mental health judgment device selects Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, InHF among multiple heart rate variability parameters. You can select from 12 heart rate variability parameters, including .

For another example, if the mental health to be judged is stress, the mental health judgment device uses nine heart rate variability parameters including Mean HR, SDNN, RMSSD, TP, VLF, LF, HF, InTP, and InHF. Heart rate variability parameters can also be selected.

For another example, if the mental health to be judged is cognitive impairment, the mental health judgment device may select Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, among a plurality of heart rate variability parameters. There are 12 heart rate variability parameters to choose from, including InHF.

The mental health determination device may determine the mental health of the subject using at least one selected heart rate variability parameter and a mental health judgment model.

For example, the mental health judgment model may include a depression judgment model, an anxiety judgment model, an insomnia judgment model, a stress judgment model, and a cognitive impairment judgment model, and the mental health judgment device may include at least one selected heart rate variability parameter and a mental health judgment model. The mental health of the subject may be determined using one of a health judgment model, for example, a depression judgment model, an anxiety judgment model, an insomnia judgment model, a stress judgment model, and a cognitive disorder judgment model.

According to an exemplary embodiment, the mental health determination device may generate a graphic chart to explain the reason for the mental health determination result and output the graphic chart through an output means. At this time, the graphic chart may express the position of the value of the subject's heart rate variability parameter in the heart rate variability parameter distribution or may express the role of each heart rate variability parameter in the judgment process of the mental health judgment model. For example, graphic charts may include gauge charts, radar charts, and Shapley Additive Explanations (SHAP) charts.

According to an exemplary embodiment, the mental health determination device receives the subject's actual mental health status as feedback, and if the mental health determination result and the subject's actual mental health status are different, the mental health condition is stored based on the subject's actual mental health status. The health judgment model can be updated.

[Example]

A total of 2,101 sample data were collected and randomly divided. 1,681 sample data were used as a learning set to create a mental health judgment model, and 420 sample data were used as a validation set to verify the performance of the mental health judgment model.

Uses 15 heart rate variability parameters including Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, InHF, LF(%), HF(%), and LF/HF ratio Create a depression judgment model by learning a deep learning model, and perform deep learning using 11 heart rate variability parameters including Mean HR, SDNN, RMSSD, pNN50, TP, LF, HF, InTP, InVLF, InLF, and InHF. Create an anxiety judgment model by training the model, and create a deep learning model using 12 heart rate variability parameters including Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, and InHF. Create an insomnia judgment model by learning, and create a stress judgment model by learning a deep learning model using 9 heart rate variability parameters including Mean HR, SDNN, RMSSD, TP, VLF, LF, HF, InTP, and InHF. Create a cognitive impairment judgment model by learning a deep learning model using 12 heart rate variability parameters including Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, and InHF. As a result of creation and testing, Table 1 below was obtained.

Accuracy (%) F1 score Depression Judgment Model 84.0 0.789 Anxiety Judgment Model 85.3 0.803 Insomnia judgment model 83.2 0.746 stress judgment model 90.8 0.763 Cognitive impairment judgment model 82.4 0.736

Referring to Table 1, it can be seen that high performance is shown when generating a depression judgment model, anxiety judgment model, insomnia judgment model, stress judgment model, and cognitive impairment judgment model according to an embodiment of the present invention.

The above-described embodiments may be implemented as computer-readable code on a computer-readable recording medium. Computer-readable recording media may include all types of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media may include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, etc. Additionally, the computer-readable recording medium may be distributed over network-connected computer systems and written and executed as computer-readable code in a distributed manner.

So far, the present invention has been examined focusing on its preferred embodiments. A person skilled in the art to which the present invention pertains will understand that the present invention may be implemented in a modified form without departing from the essential characteristics of the present invention. Accordingly, the scope of the present invention is not limited to the above-described embodiments, but should be interpreted to include various embodiments within the scope equivalent to the content described in the patent claims.

100: Mental health judgment system
110: Mental health judgment model generation device
120, 400: Mental health judgment device
210: Learning data collection unit
220: Model creation unit
310: camera
320: Parameter acquisition unit
330: Mental Health Judgment Department
410: input unit
420: storage unit
430: Department of Communications
440: output unit
450: Model update unit

Claims (16)

A camera that acquires facial images of the subject;
a parameter acquisition unit configured to acquire a plurality of heart rate variability parameters for the subject by analyzing the acquired facial image; and
According to the type of mental health to be determined, at least one heart rate variability parameter is selected from among the plurality of heart rate variability parameters obtained, and the subject is assessed using the selected at least one heart rate variability parameter and a machine learning-based mental health judgment model. Mental Health Judgment Department, which judges the mental health of people; Including,
Mental health assessment device. According to paragraph 1,
The plurality of heart rate variability parameters include Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, InHF, LF(%), HF(%) and LF/HF ratio,
Mental health assessment device. According to paragraph 1,
The mental health includes depression, anxiety, insomnia, stress, and cognitive impairment, and the mental health judgment model includes a depression judgment model, an anxiety judgment model, an insomnia judgment model, a stress judgment model, and a cognitive disorder judgment model.
Mental health assessment device. According to paragraph 1,
The mental health judgment department,
If the mental health to be judged is depression, among the above multiple heart rate variability parameters, Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, InHF, LF(%), HF(%) and selecting the LF/HF ratio,
Mental health assessment device. According to paragraph 1,
The mental health judgment department,
If the mental health to be determined is anxiety, select Mean HR, SDNN, RMSSD, pNN50, TP, LF, HF, InTP, InVLF, InLF and InHF among the plurality of heart rate variability parameters.
Mental health assessment device. According to paragraph 1,
The mental health judgment department,
If the mental health to be determined is insomnia, select Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF and InHF among the plurality of heart rate variability parameters.
Mental health assessment device. According to paragraph 1,
The mental health judgment department,
If the mental health to be judged is stress, select Mean HR, SDNN, RMSSD, TP, VLF, LF, HF, InTP and InHF among the plurality of heart rate variability parameters.
Mental health assessment device. According to paragraph 1,
The mental health judgment department,
If the mental health to be determined is cognitive impairment, select Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF and InHF among the plurality of heart rate variability parameters.
Mental health assessment device. Obtaining a facial image of a subject;
acquiring a plurality of heart rate variability parameters for the subject by analyzing the acquired facial image;
Selecting at least one heart rate variability parameter among the obtained plurality of heart rate variability parameters according to the type of mental health to be determined; and
Determining mental health of the subject using the selected at least one heart rate variability parameter and a machine learning-based mental health judgment model; Including,
How to determine mental health. According to clause 9,
The plurality of heart rate variability parameters include Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, InHF, LF(%), HF(%) and LF/HF ratio,
How to determine mental health. According to clause 9,
The mental health includes depression, anxiety, insomnia, stress, and cognitive impairment, and the mental health judgment model includes a depression judgment model, an anxiety judgment model, an insomnia judgment model, a stress judgment model, and a cognitive disorder judgment model.
How to determine mental health. According to clause 9,
The step of selecting at least one heart rate variability parameter includes:
If the mental health to be judged is depression, among the above multiple heart rate variability parameters, Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF, InHF, LF(%), HF(%) and selecting the LF/HF ratio,
How to determine mental health. According to clause 9,
The step of selecting at least one heart rate variability parameter includes:
If the mental health to be determined is anxiety, select Mean HR, SDNN, RMSSD, pNN50, TP, LF, HF, InTP, InVLF, InLF and InHF among the plurality of heart rate variability parameters.
How to determine mental health. According to clause 9,
The step of selecting at least one heart rate variability parameter includes:
If the mental health to be determined is insomnia, select Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF and InHF among the plurality of heart rate variability parameters.
How to determine mental health. According to clause 9,
The step of selecting at least one heart rate variability parameter includes:
If the mental health to be judged is stress, select Mean HR, SDNN, RMSSD, TP, VLF, LF, HF, InTP and InHF among the plurality of heart rate variability parameters.
How to determine mental health. According to clause 9,
The step of selecting at least one heart rate variability parameter includes:
If the mental health to be determined is cognitive impairment, select Mean HR, SDNN, RMSSD, pNN50, TP, VLF, LF, HF, InTP, InVLF, InLF and InHF among the plurality of heart rate variability parameters.
How to determine mental health.