KR102517717B1 - Apparatus and method for recommending art therapy program through reinforcement learning - Google Patents

Abstract

The present invention relates to a mental health status classification device and method using a multimodal artificial neural network. The present invention relates to a method for recommending an art therapy program through reinforcement learning. The method is performed by an art therapy program recommendation module included in a device. The method comprises the steps of: receiving mental health status classification data of a test subject; inputting the mental health status classification data into an artificial neural network module included in the art therapy program recommendation module; outputting recommended art therapy program data by the artificial neural network module based on the mental health status classification data; and transmitting the output recommended art therapy program data to the test subject client.

Description

Apparatus and method for recommending art therapy program through reinforcement learning}

The present invention relates to an apparatus and method for classifying mental health conditions using a multimodal artificial neural network.

Depression is one of the most important topics in psychiatry, with 17% of the population experiencing it. Currently, the main method for diagnosing depression in psychiatry is to use a self-diagnosis questionnaire. However, in this case, there is a problem in that objectivity is low because the patient has no choice but to depend on the subjective response felt by the patient. It is extremely difficult to objectify the symptoms of depression. Patients' objectivity is further reduced as they pursue 'desirable sociality' that is conscious of the external gaze.

Even in the case of mental disorders other than depression, the current methodology had limitations. For example, people who have experienced a disaster, such as a natural disaster or a traffic accident, experience various physical and mental difficulties for many years after the disaster. Some of them recover function and return to daily life relatively early after the accident, while others complain of constant mental stress and difficulties in daily life. In order to discriminate people who progress to PTSD (Post-Traumatic Stress Disorder) at an early stage after disasters and traumas, various measurement tools (e.g., EEG, etc.) are used to select high-risk groups for PTSD. Methods have been proposed (US9792823B2). However, since this method requires the use of physical measurement tools, there are limitations in applying it to all people who have experienced large-scale disaster situations, and there are limitations in selecting high-risk groups among test subjects for mental disorder tests.

US Patent Publication US 2014-0046696 A1, Systems and Methods for Pharmacogenomic Decision Support in Psychiatry, AssureRx Health, Inc. US registered patent US 6317731 B1, Method for predicting the therapeutic outcome of a treatment, ADVANCED BIOLOGICAL LABORATORIES S.A. US registered patent US 9792823 B2, Multi-view learning in detection of psychological states, Raytheon BBN Technologies Corp.

Oh, J., Yun, K., Hwang, J. H., & Chae, J. H. (2017). classification of suicide attempts through a Machine learning algorithm Based on Multiple systemic Psychiatric scales., Frontiers in psychiatry,8, 192. Nuriel S. Mor, Kathryn L. Dardeck, (2018). Quantitative Forecasting of Risk for PTSD Using Ecological Factors: A Deep Learning Application, Journal of Social, Behavioral, and Health Sciences 2018, Volume 12, Issue 1, Pages 61-73. IR Galatzer-Levy, S Ma, A Statnikov, R Yehuda and A Y Shalev, (2017). Utilization of machine learning for prediction of post-traumatic stress: a re-examination of cortisol in the prediction and pathways to non-remitting PTSD, Transl Psychiatry (2017) 7, e1070. Barak-Corren Y, Castro VM, Javitt S, Hoffnagle AG, Dai Y, Perlis RH, et al. Predicting suicidal behavior from longitudinal electronic health records. Am J Psychiatry (2017) 174:154-62. Kessler RC, Warner CH, Ivany C, Petukhova MV, Rose S, Bromet EJ, et al. Predicting suicides after psychiatric hospitalization in US Army soldiers: the army study to assess risk and resilience in servicemembers (Army STARRS). JAMA Psychiatry (2015) 72:49-57.

Accordingly, an object of the present invention is to classify a patient's mental illness by using various scales commonly used in psychiatry, counseling data consisting of a patient's video/voice/text, and patient's activity data output from a wearable device as input data. An object of the present invention is to provide an apparatus and method for classifying mental health conditions using a modal artificial neural network.

Hereinafter, specific means for achieving the object of the present invention will be described.

An object of the present invention is to learn to classify the mental health condition of the test subject, who is not diagnosed for the mental health condition, through counseling data and activity data of the existing test subject, the test subject diagnosed for the mental health condition. A memory module for storing program codes of a mental health condition classification artificial neural network module, which is an artificial neural network; and a processing module for processing the program code of the mental health state classification artificial neural network module, wherein the program code of the mental health state classification artificial neural network module is used to input the counseling data and activity data of the test subject. input step; and an inference step of inferring and outputting mental health state classification data, which is data for the classification of the mental health state of the test subject, wherein the mental health state includes depression of the test subject. Characterized in, it can be achieved by providing a mental health condition classification device using a multimodal artificial neural network.

In addition, the mental health condition includes a plurality of mental diseases including depression of the test subject and a PTSD prognosis of the test subject, and the mental health condition classification artificial neural network module has a plurality of artificial intelligence learned for each of the plurality of mental diseases. A neural network may be included, and in the inference step, the mental health state classification data for each of the plurality of mental disorders of the test subject may be inferred and output.

Another object of the present invention is to learn to classify the mental health condition of the test subject who has not been diagnosed for the mental health condition through counseling data and activity data of an existing test subject who is a test subject diagnosed for the mental health condition. and a mental health state classification artificial neural network module, which is an artificial neural network, wherein the mental health state classification artificial neural network module inputs the counseling data and the activity data of the test subject, and is used to classify the mental health state of the test subject. It can be achieved by providing a mental health state classification device using a multimodal artificial neural network, characterized in that inferring and outputting mental health state classification data, which is data for the test, and that the mental health state includes depression of the test subject. .

Another object of the present invention is to learn to classify the mental health condition of the test subject who has not been diagnosed for the mental health condition through counseling data and activity data of an existing test subject who is a test subject diagnosed for the mental health condition. an input step of inputting the counseling data and the activity data of the subject to a mental health condition classification artificial neural network module, which is an artificial neural network; and an inference step in which the mental health state classification artificial neural network module infers and outputs mental health state classification data, which is data for classification of the mental health state of the test subject, and wherein the mental health state is the depression of the test subject. It can be achieved by providing a mental health state classification method using a multimodal artificial neural network, characterized in that it comprises.

Another object of the present invention is an artificial neural network trained to use counseling data of a test subject in the form of a video or audio and activity data of the test subject in the form of a signal as input data and output data of the test subject's mental health state classification data. A memory module for storing the program code of the mental health condition classification module including; and a processing module processing the program code of the mental health condition classification module. Including, the program code of the mental health state classification module, the input step of inputting the counseling data and the activity data of the test subject; and an inference step of inferring and outputting mental health state classification data, which is data for classification of the mental health state of the test subject, wherein the mental health state includes whether or not the test subject is depressed. Characterized in that, it can be achieved by providing a mental health state classification device using a multimodal artificial neural network.

In addition, the memory module further stores inference program codes and learning program codes of the multimodal encoder module, and the processing module further processes the inference program codes and learning program codes of the multimodal encoder module, and the multimodal encoder The reasoning program code of the module may include an input step of inputting the counseling data and the activity data of the test subject; and an inference step of outputting the multimodal encoding vector, wherein the learning program code of the multimodal encoder module includes a learning target model and an answer model having the same structure as the multimodal encoder module. A model construction step configured to do so; For each specific time section (time section step) of the consultation data and the activity data, combination data composed of a random concatenate is configured as input data, and one of the consultation data and the activity data is randomly sampled. a masking step of generating masked combination data by masking the masked data; an input data input step of inputting the masked combination data as input data of the learning target model and inputting the unmasked original combination data as input data of the correct answer model; an output data output step of obtaining a learning target multimodal encoding vector that is output data of the learning target model and a correct answer multimodal encoding vector that is output data of the correct answer model; and updating parameters of the learning target model in a direction of minimizing a loss value of a loss function based on a difference between the learning target multimodal encoding vector and the correct answer multimodal encoding vector, and adding EMA to the parameters of the learning target model. a parameter update step of updating parameters of the correct answer model by applying an exponentially moving average; Utilizing a multimodal artificial neural network configured to be executed on a computer, including, in the input step of the program code of the mental health condition classification module, the multimodal encoding vector is input as input data of the mental health condition classification module. This can be achieved by providing a mental health condition classification device.

In addition, the memory module further stores the program code of the neural network processing module, the processing module further processes the program code of the neural network processing module, and the program code of the neural network processing module is stored in the mental health condition classification module. a parameter updating step of updating parameters by processing the included learning session of the artificial neural network; an uploading step of uploading the updated parameters to a federated learning module that is a component of a main neural network server; and a download step of downloading a main neural network jointly learned by a plurality of examiner clients from a main neural network module, which is a component of the main neural network server, and replacing at least a part of the artificial neural network included in the mental health condition classification module. Is configured to be performed on a computer, wherein the main neural network server is configured to update the artificial neural network included in the mental health condition classification module by integrating the parameters and other parameters uploaded from other examiner clients. This can be achieved by providing a mental health condition classification device using a neural network.

As described above, the present invention has the following effects.

First, according to one embodiment of the present invention, by classifying mental disorders such as depression in real time, an effect capable of appropriately coping with a mental crisis situation is generated.

Second, according to one embodiment of the present invention, for patients who have visited a psychiatric outpatient clinic, etc., an effect that can discriminate mental disorders such as PTSD, depression, and suicide risk with relatively few resources occurs.

The following drawings attached to this specification illustrate preferred embodiments of the present invention, and together with the detailed description of the invention serve to further understand the technical idea of the present invention, the present invention is limited only to those described in the drawings. and should not be interpreted.
1 is a mental health state classification device using a multimodal artificial neural network that classifies a patient's mental illness using a patient's video/voice/text data and patient activity data output from a wearable device according to an embodiment of the present invention. A schematic diagram showing (1),
2 is a schematic diagram showing the configuration of an apparatus 1 for classifying mental health conditions using a multimodal artificial neural network according to an embodiment of the present invention;
3 is a schematic diagram showing a learning session of the artificial neural network module 12 for classifying mental health conditions according to an embodiment of the present invention;
4 is a schematic diagram showing the structure of a multilayer neural network model (deep learning or deep neural network model);
5 is a schematic diagram showing a calculation process at a node of the artificial neural network module 12 for classifying mental health conditions according to an embodiment of the present invention;
6 is a schematic diagram showing a drop-out method of the artificial neural network module 12 for classifying mental health conditions according to an embodiment of the present invention;
7 is a graph showing a ReLU activation function;
8 is a schematic diagram showing neural network transfer learning of the artificial neural network module 12 for classifying mental health conditions according to an embodiment of the present invention;
9 is a schematic diagram showing reinforcement learning of the artificial neural network module 12 for classifying mental health conditions according to another embodiment of the present invention;
10 is a flowchart illustrating a mental health state classification method using a multimodal artificial neural network according to an embodiment of the present invention;
11 is a schematic diagram showing the operational relationship of a mental health state classification device using a multimodal artificial neural network according to a modified example of the present invention;
12 is a schematic diagram showing the operational relationship according to each step of a mental health condition classification apparatus using a multimodal artificial neural network according to a modified example of the present invention and a main neural network server;
13 is a schematic diagram showing the configuration of an apparatus for classifying mental health conditions using a multimodal artificial neural network according to a modified example of the present invention;
14 is a schematic diagram showing the specific operating relationship of a mental health state classification device using a multimodal artificial neural network according to a modified example of the present invention;
15 is a schematic diagram showing a multimodal encoder module according to a modified example of the present invention;
16 is a schematic diagram showing a learning session of a multimodal encoder module according to a modified example of the present invention;
17 is a schematic diagram showing a mental health condition classification module according to a modified example of the present invention;
18 is a schematic diagram showing a learning session of a mental health condition classification module according to an embodiment of the present invention;
19 is a schematic diagram showing an art therapy program recommendation module according to a modified example of the present invention;
20 is a schematic diagram illustrating reinforcement learning of an art therapy program recommendation module according to a modified example of the present invention.

Hereinafter, an embodiment in which a person skilled in the art can easily practice the present invention will be described in detail with reference to the accompanying drawings. However, in the detailed description of the operating principle of the preferred embodiment of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted.

In addition, the same reference numerals are used for parts having similar functions and actions throughout the drawings. Throughout the specification, when a specific part is said to be connected to another part, this includes not only the case where it is directly connected but also the case where it is indirectly connected with another element interposed therebetween. In addition, including a specific component does not exclude other components unless otherwise stated, but means that other components may be further included.

Mental health condition classification device using multimodal artificial neural network

1 shows mental health using a multimodal artificial neural network that classifies a patient's mental illness using consultation data composed of a patient's video/voice/text and activity data output from a wearable device according to an embodiment of the present invention. It is a schematic diagram showing the state classification device 1. As shown in FIG. 1, the mental health condition classification apparatus 1 using the multimodal artificial neural network of the training session includes counseling data of an existing test subject who has been diagnosed for a mental health condition (a, patient's Psychiatric counseling session data composed of video/voice/text), activity data (b, sensor data including activity/sleep/stress data input from a wearable device given to the patient), and survey item data (c, Ground Truth) are input The mental health condition classification device (1) using a multimodal artificial neural network is learned, and the mental health condition classification device (1) using a multimodal artificial neural network in an inference session is used for mental health conditions that have not been diagnosed. Depressive disorder classification data (A), PTSD (posttraumatic stress disorder) prognosis classification data (B), suicide attempt possibility classification data, anxiety disorder (Anxiety disorder) classification data such as mental health status classification data is output. Hereinafter, for convenience of description, the content of the present invention will be described based on depression classification data (A) and PTSD prognosis classification data (B) among mental health status classification data, and the scope of the present invention may not be limited thereto.

An apparatus for classifying mental health conditions using a multimodal artificial neural network according to an embodiment of the present invention (1) is a processing module of a computing device such as a specific web server, a virtual server such as a cloud server, a smartphone, a tablet PC, and a desktop PC. It may be processed by, and configured to be stored in a memory module of each device.

Counseling data (a) refers to data about a counseling session of a test subject that is input in the form of video, audio, or text through a digital device. Counseling data (a) may be input from a smartphone, tablet, or PC of the examinee or from a smartphone, tablet, or PC of the doctor in charge.

Activity data (b) refers to activity signal data about sleep, respiration, stress, etc. output by sensor data of a wearable device provided to the test subject. The activity data (b) may be input from a smart phone, tablet, PC, etc. connected to the wearable device of the examinee, or may be input from the smart phone, tablet, PC, etc. of the doctor in charge.

Questionnaire data (c) includes text data on psychiatric diagnoses such as the Depression Questionnaire, which is a questionnaire on depression (Depressive Disorder), the Suicide Attempt Questionnaire, which is a questionnaire on suicide attempts, and the PTSD Questionnaire, which is a questionnaire on PTSD (posttraumatic stress disorder). it means. The suicide attempt questionnaire according to an embodiment of the present invention is "Have you ever attempted suicide during your lifetime?, ② Have you ever attempted suicide within the last year?, ③ Have you ever attempted suicide within the last month?" , and each questionnaire can be used as the ground truth for the output value in the learning session of the mental health state classification device 1 using a multimodal artificial neural network. Questionnaire data (c) may be input from a smartphone, tablet, or PC of the examinee or from a smartphone, tablet, or PC of a doctor in charge.

Depression classification data (A) is based on the actual depression as Ground Truth based on the questionnaire data (c) of the depression questionnaire in the learning session. It means the possibility of depression of the test subject.

* PTSD prognosis classification data (B) is based on the actual PSTD prognosis based on the PTSD questionnaire survey item data (c) in the learning session as Ground Truth, and is classified in the inference session of the artificial neural network module (12). It means the possibility of prognosis of PTSD of the test subject to be tested.

The mental health state classification apparatus 1 using a multimodal artificial neural network according to an embodiment of the present invention can be configured in the form of supervised learning, linear / logistic regression analysis (Regression), support vector machine (Support Vector Machine) ), multi-layer perceptron, Naive-Bayesian Classification, Random Forest Classification, and Neural Network. For convenience of description, the mental health state classification apparatus 1 using the multimodal artificial neural network according to an embodiment of the present invention will be described as an example configured with the artificial neural network.

2 is a schematic diagram showing the configuration of an apparatus 1 for classifying mental health conditions using a multimodal artificial neural network according to an embodiment of the present invention. As shown in FIG. 2, the mental health state classification apparatus 1 using the multimodal artificial neural network according to an embodiment of the present invention includes a consultation data encoder 10, an activity data encoder 11, and mental health state classification. An artificial neural network module 12 may be included.

The counseling data encoder 10 is learned using existing counseling data (a) and activity data (b) of the test subject as input data, and an artificial neural network module that infers a counseling encoding vector based on the counseling data (a) of the test subject. it means.

The activity data encoder 11 is trained using existing counseling data (a) and activity data (b) of the test subject as input data, and an artificial neural network module that infers an activity encoding vector based on the activity data (b) of the test subject. it means.

The mental health condition classification artificial neural network module 12 takes a combination vector obtained by combining a counseling encoding vector and an activity encoding vector as input data, and classifies mental health conditions such as depression classification data (A) and PTSD prognosis classification data (B). It refers to an artificial neural network module that uses data as output data. The mental health state classification artificial neural network module 12 may be trained using the questionnaire item data (c) as the ground truth.

The neural network model of the counseling data encoder 10, activity data encoder 11, and mental health condition classification artificial neural network module 12 is a supervised learning algorithm derived from the structure of neurons in biology. The basic operating principle of a neural network model is to classify an optimal output value for an input value by interconnecting several neurons. From a statistical point of view, a neural network model can be viewed as a projective tracking regression that takes a non-linear function over a linear combination of input variables. Hereinafter, for convenience of description, the mental health condition classification artificial neural network module 12 will be mainly described.

3 is a schematic diagram showing a learning session of the artificial neural network module 12 for classifying mental health conditions according to an embodiment of the present invention. As shown in FIG. 3, the counseling encoding vector of the existing test subject, the activity encoding vector, and the history vector (a counseling history vector that is a combination of counseling encoding vectors for counseling before the current time step and activities for activities before the current time step) The activity history vector, which is a combination of encoding vectors, is a concatenated vector) is input to each node of the input layer such as x1, x2, and x3 as input data, and based on weights such as w1, h1, h2, Depression classification data (A) classified based on an activation function such as softmax or ReLU after computing a hidden layer such as h3 is output from the output layer of y1. Hidden layer in the direction of reducing the error (error, -Sigma(y _i log _pi )) between the actual depression (Ground Truth) determined by the output depression classification data (A) and the actual depression questionnaire data (c) Back propagation can be performed to update the weight of

Regarding the specific structure of the mental health condition classification artificial neural network module 12 according to an embodiment of the present invention, FIG. 4 is a schematic diagram showing the structure of a multi-layer neural network model (deep learning or deep neural network model). As shown in FIG. 4, the multilayer neural network model of the counseling data encoder 10, activity data encoder 11, and mental health condition classification artificial neural network module 12 according to an embodiment of the present invention is an input layer , a hidden layer, and an output layer. The input layer is composed of nodes corresponding to each input variable, and the number of nodes is equal to the number of input variables. The hidden layer serves to process the linear combination of variable values transmitted from the input layer with a non-linear function such as the sigmoid function and pass it to the output layer or other hidden layer (recently, by applying the chain rule in Back propagation, the error is Since the problem of vanishing gradient is generated in , ReLU is generally used instead of the sigmoid function). The output layer is a node corresponding to an output variable, and output nodes are generated as many as the number of classes in the classification model.

5 is a schematic diagram showing a calculation process in a node of the artificial neural network module 12 for classifying mental health conditions according to an embodiment of the present invention. As shown in FIG. 5 , calculation actually occurs at each node, and this calculation process is mathematically designed to simulate a process occurring in neurons constituting a human neural network. A node responds when it receives a stimulus of a certain size or more, and the size of the response is approximately proportional to the value obtained by multiplying the input value and the node's coefficient (or weights), excluding the bias value. In general, a node receives multiple inputs and has as many coefficients as the number of inputs. Therefore, by adjusting this coefficient, different weights can be given to different inputs. Finally, the multiplied values are all added up, and the sum goes into the input of the activation function. The result of the activation function corresponds to the output of the node, and this output value is ultimately used for classification or regression analysis. Each layer of the neural network model is composed of at least one node, and activation/deactivation of each node is determined according to an input value. The input data becomes the input of the first layer (input layer), and then the output of each layer becomes the input of the next layer. All the coefficients keep changing slightly during the learning process, which in turn reflects which inputs each node considers important. And 'training' of the neural network model is the process of updating these coefficients.

The most problematic problem in multilayer neural network models, that is, deep learning, is overfitting. Overfitting occurs when the complexity of the model is high compared to the amount of data given. Unfortunately, the complexity of the model increases exponentially as the neural network gets deeper. Therefore, although everyone knows that a deep network that can learn almost infinite phenotypes is good, overfitting occurs so severely that neural network research was stopped by Professor Marvin Minsky's 1969 Perceptrons theory. However, around 2007~2008, new initialization methods to prevent overfitting, such as RBM (Restricted Boltzmann Machine) and CNN (Convolutional Neural Network), were proposed, and deep learning research was actively conducted again.

In particular, RBM is used as a component of DBN (Deep Belief Network), and overfitting by effectively pre-training each layer of the feedforward neural network to be trained through an unsupervised RBM (restricted Boltzmann machine) We set an initialize point at a level that can prevent this, and proceed with learning in the form of using supervised back propagation again.

However, unsupervised pretraining such as RBM and DBN, which was proposed in two papers, the Greedy layer-wise training of deep networks by Professor Bengio's research team published in NIPS 2006 and A fast learning algorithm for deep belief nets by Professor Hinton's research team published in NIPS 2007 methods are not used in current practice. This is because it is now known that random initialization performs much better than initializing the weights in this way when there is enough data. As part of random initialization, the concept of drop-out was introduced, and most of them use the drop-out method recently.

6 is a schematic diagram showing a drop-out method of the artificial neural network module 12 for classifying mental health conditions according to an embodiment of the present invention. As shown in FIG. 6, the drop-out method of the mental health condition classification artificial neural network module 12 according to an embodiment of the present invention uses about 50% of neurons in the hidden layer instead of using all neurons in each learning. use. It is known that there is an effect of ensemble of several small neural networks in one deep learning, and the ensemble greatly reduces overfitting. In addition, neurons with similar weights are reduced, so neurons making redundant judgments are reduced, which has the advantage of being able to use neurons efficiently.

In addition, recently, new activation functions such as Rectified Linear Unit (ReLU) have been used to solve problems such as slow learning time and overfitting. 7 is a graph showing a ReLU activation function. As shown in FIG. 7, the existing sigmoid function has a problem that errors disappear whenever gradient descent is performed in several layers. As you go through several layers and reach the limit, the gradient of the sigmoid function becomes smaller, causing a problem that the weights are not updated. However, when using the ReLU function as an activation function, when the gradient is learned as 0 or 1, the error is propagated to 100%, which solves this problem. Since the ReLU function is not limited to [0,1] like sigmoid and its range is unlimited, it can be seen that it has more reliable expressiveness. In addition, there are many unnecessary values among the output values of each node. In this case, when using the sigmoid function, it is necessary to calculate all values, but the ReLU function can reduce a large part of the amount of calculation, resulting in an effect of improving the computing speed. Regularization can be improved by the ReLU function.

In the learning session of the mental health condition classification artificial neural network module 12 according to an embodiment of the present invention, the data on suicide attempts are disproportionate compared to general mental disorders such as depression (Depressive disorder) or anxiety disorder ( Among the learning data, there is an imbalance between data with suicide attempts and data without suicide attempts) and a small amount. In order to solve this problem, the mental health condition classification artificial neural network module 12 according to an embodiment of the present invention is a general mental condition that is easy to learn by the artificial neural network due to the large amount of data such as depression (Depressive disorder). It can be configured as a method of transferring the previously learned artificial nerve of the artificial neural network module for the disease.

Regarding neural network transfer, FIG. 8 is a schematic diagram showing neural network transfer learning of the artificial neural network module 12 for classifying mental health conditions according to an embodiment of the present invention. As shown in FIG. 8, some layers of the artificial neural network module 12_1 for classifying mental health conditions for depression are part of layers of the artificial neural network module 12_2 for classifying mental health conditions for other mental disorders. The remaining layers of the artificial neural network module 12_2, which are generated by neural network transition and classify mental health conditions for different mental disorders, are generated as initialization layers in which initial weight w and initial bias b, which are parameters, are randomly initialized. Afterwards, the counseling encoding vector and activity encoding vector of the test subject for other mental disorders are used as input data, and whether or not other mental illnesses determined based on the questionnaire data (c) is used as the ground truth for the output data. Parameters of the mental health condition classification artificial neural network module 12_2 for other mental disorders may be calibrated in detail by retraining the health state classification artificial neural network module 12_2. At this time, the input data of the mental health condition classification artificial neural network module 12_1 for depression classification is the test subject's consultation encoding vector and activity encoding vector (or, further includes a history vector) for depression, and the output data is depression classification data, The input data of the mental health condition classification artificial neural network module 12_2 for classification of other mental disorders is the counseling encoding vector and the activity encoding vector (or further includes a history vector) of the test subject for other mental disorders, and the output data is other mental disorders. Classification data (eg, PTSD prognostic classification data, suicide attempt probability classification data, etc.).

In particular, neural network transition according to an embodiment of the present invention may be performed to configure only the output layer or the second half layer after a specific layer as an initialization layer. According to this, even if the amount of data of the subject for each mental health condition, such as the possibility of suicide attempt and the prognosis of PTSD, is very insufficient to train the artificial neural network, the artificial neural network can predict the pattern of the subject's mental illness for general mental disorders such as depression in advance. By considering it, the effect of being able to smoothly perform mental health condition classification occurs. In addition, without inserting dummy data into the mental health condition classification artificial neural network module 12 or performing data processing such as data cloning, the classification of mental diseases with insufficient data such as the possibility of suicide attempt is performed smoothly. The effect of being able to do that happens.

In addition, re-learning of the mental health state classification artificial neural network module 12 according to an embodiment of the present invention is an input layer of the mental health state classification artificial neural network module 12_2 for other mental disorders generated by neural network transition For hidden layers other than the initialization layer, parameters can be fixed so that they are not changed by relearning. In certain frameworks you can use trainableParameter=0 or freeze=1. According to this, the effect of learning only the output layer or the latter layer by storing the active value of the fixed layer is generated.

In addition, the re-learning of the artificial neural network module 12_2 for classifying mental health conditions for other mental disorders according to an embodiment of the present invention is performed by the upper layer artificial neural network to classify mental health conditions for other mental disorders. The artificial neural network module 12_2 ) may be configured to be relearned by adjusting the hyperparameters. According to an embodiment of the present invention, the hyperparameters of the mental health condition classification artificial neural network module 12 adjusted by the upper layer artificial neural network are Parameter Scale, Gradient, Learning rate (learning rate), Learning rate decay (learning rate decomposition), Regularization strength (normalized strength), and the like.

The learning rate adjusted during re-learning according to an embodiment of the present invention may be applied to parameter update during re-learning as shown in Equation 1 below.

In Equation 1 above, W is the initial weight, b is the initial bias, η is the learning rate, L is the loss, and ∂L/∂W and ∂L/∂b are the gradients of the weight and bias, respectively. do.

After all, the upper-layer artificial neural network refers to a higher-layer artificial neural network that performs automatic tuning for each mental disorder by adjusting and outputting hyperparameters during relearning of the mental health condition classification artificial neural network module 12 in detail. parameters such as the counseling data encoder 10, the activity data encoder 11, the mental health status classification artificial neural network module 12, the output depression classification data (A), and the PTSD prognosis classification data (B) It can be configured to output hyperparameters based on The output depression classification data (A) can be used to calculate the gradient, weight, loss, etc. of the learned mental health condition classification artificial neural network module 12 . Including the output depression classification data (A), the gradient, weight, loss, learning rate, and amount of data of the mental health condition classification artificial neural network module 12 can be used to train the upper-layer artificial neural network. there is.

In relation to another embodiment of the present invention, reinforcement learning may be used for mental health state classification of the mental health state classification artificial neural network module 12 . 9 is a schematic diagram showing reinforcement learning of the artificial neural network module 12 for classifying mental health conditions according to another embodiment of the present invention. As shown in FIG. 9, in the case of reinforcement learning of the mental health state classification artificial neural network module 12, the counseling encoding vector, activity encoding vector, and history vector of the test subject are Environment, and the mental health state classification artificial neural network module 12 is Agent, Mental Health Status Classification The mental health status classification of the artificial neural network module (12) constitutes reinforcement learning, in which the similarity between the mental health status classification diagnosed by Action, the questionnaire data (c) and the mental health status classification consists of Reward. The mental health condition classification artificial neural network module 12 may be updated. According to reinforcement learning according to an embodiment of the present invention, the accuracy of the mental health state classification artificial neural network module 12 can be improved even when the environment is non-linear, such as counseling data (a) and activity data (b) of the test subject. effect occurs.

10 is a flowchart illustrating a mental health condition classification method using a multimodal artificial neural network according to an embodiment of the present invention. As shown in FIG. 10, the mental health state classification method using a multimodal artificial neural network according to an embodiment of the present invention is a learning step of learning the mental health state classification device 1 using a multimodal artificial neural network ( S10), a new data input step of inputting the test subject's data (S11), and an inference step of classifying the test subject's mental illness (S12).

The learning step (S10) is a step of learning the mental health state classification device 1 using the multimodal artificial neural network using the existing counseling data, activity data, and questionnaire data (used as ground truth) of the test subject. At this time, separate artificial neural network modules are configured for mental disorders such as suicide attempts, PTSD, depression, and anxiety disorders of existing test subjects, and each artificial neural network module is separately learned.

The new data input step (S11) is a step of inputting the counseling data and activity data of the test subject into the mental health state classification apparatus 1 using the multimodal artificial neural network learned in S10.

In the inference step (S12), mental health state classification data (depression classification data, PTSD prognosis classification data, etc.) This step is to infer (classify) and output.

[modified example]

The mental health condition classification device 1 using a multimodal artificial neural network according to a modified example of the present invention can be configured with a self-supervised structure that can be applied integrally regardless of the modality of data. Regarding the specific configuration and operational relationship of the mental health condition classification apparatus 1 using a multimodal artificial neural network according to a modified example of the present invention, FIG. 11 is a mental health condition using a multimodal artificial neural network according to a modified example of the present invention 12 is a schematic diagram showing the operation relationship of a mental health status classification device using a multimodal artificial neural network according to a modified example of the present invention and a main neural network server according to each step, 13 is a schematic diagram showing the configuration of a mental health condition classification device using a multimodal artificial neural network according to a modified example of the present invention, and FIG. 14 is a mental health condition using a multimodal artificial neural network according to a modified example of the present invention. It is a schematic diagram showing the specific operating relationship of the classification device. As shown in FIG. 11, the mental health state classification apparatus 1 using a multimodal artificial neural network may be configured in the examiner client 100, and the testee for the mental health state utilizes the testee client 101 Mental health counseling is conducted with the examiner through a wired/wireless network connection with the examiner client 100 connected through the live counseling streaming server, counseling data received through the mental health examination application module configured in the examiner client 100, activity data, Questionnaire data (used as ground truth) is input into the mental health condition classification device 1 using the multimodal artificial neural network according to the modified example of the present invention included in the examiner client 100, and the multimodal artificial neural network is used. A mental health state classification device (1) using a multimodal artificial neural network is configured to transmit parameters of the artificial neural network included in the device to the main neural network server (200) in a mental health state classification device (1).

In addition, as shown in FIG. 12, the apparatus for classifying mental health conditions using a multimodal artificial neural network according to a modified example of the present invention is configured in a plurality of examiner clients 100, respectively, and the main neural network server 200 and the wired/wireless network It can be connected to and configured to perform a client learning step, a parameter update step, and a main neural network download step. In addition, as shown in FIGS. 13 and 14, the mental health state classification apparatus 1 using the multimodal artificial neural network according to the modified example of the present invention uses the counseling data and activity data of the test subject as input data, and the test subject's The test subject's mental health status classification data corresponding to the counseling data, activity data, and questionnaire data is used as output data. module, a parameter upload module, and a main neural network download module. The parameter upload module transmits the changed parameters and corrected change information to the federated learning module 220 of the main neural network server 200, and the federated learning module 220 transmits the parameters and corrected changed information received from the plurality of tester clients 100. It can be configured to generate a pre-learned main neural network by performing federated learning based on. The pre-learned main neural network may be configured to update the artificial neural network module of the multimodal encoder module and the mental health state classification module through the main neural network module 210 . According to this, a multimodal encoder module for performing mental health condition classification without sharing personal information included in counseling data, activity data, and questionnaire data with other examiner clients 100 or testee clients 101 and mental health The effect of being able to learn the artificial neural network of the state classification module is generated.

Regarding the multimodal encoder module, FIG. 15 is a schematic diagram showing a multimodal encoder module according to a modified example of the present invention. As shown in FIG. 15, the multimodal encoder module may be composed of an artificial neural network having consultation data and activity data as input data and a multi-modal encoding vector as output data, a CNN layer, It can be configured to include a Transformer layer.

CNN layers can be configured to include layer normalization and GELU activation functions, and to include temporal convolution networks (TCNs). Temporal convolutional network (TCN) is a framework that includes temporality and a large receptive field according to sequential data using temporal convolution and dilation. The CNN layer may be configured to output a dimensionally reduced latent vector using a specific time step of consultation data (video, audio) and activity data as input data. The CNN layer according to an embodiment of the present invention may take, as input data, combination data configured by random concatenation for a specific time section step of consultation data (video, audio) and activity data. According to this, an effect of enabling overall feature extraction with reduced pairing of counseling data and activity data with a temporal relationship is generated by the configuration of randomly combined input data.

The Transformer layer may be configured to include a Multi-head Self Attention Layer - FFNN - softmax. The Multi-head Self Attention Layer of the multimodal encoder module receives latent vectors as input data and performs Positional encoding that combines location information, and performs multiple Self Attention operations (e.g., scaled It refers to a layer that generates attention information by concatenating the attention value matrix generated by performing dot product attention). A feed forward neural network (FFNN) of the multimodal encoder module may mean a position-wise FFNN or a fully-connected FFNN. The input data of the FFNN of the multimodal encoder module may be composed of a layer normalized value of a latent vector and a matrix obtained by residual connection of the attention information. The softmax (e.g., ReLU, GELU, etc.) of the multimodal encoder module takes the value obtained by layer normalization of the residual connection matrix of the FFNN output matrix as input data and mental health status classification data It may mean an activation layer used as output data. According to this, the characteristics of the temporal order of counseling data also affect mental health state classification by positional encoding, and multiple attention calculations are performed in parallel by the multi-head self attention layer, so attention of various values is utilized. This results in a more sophisticated classification of mental health conditions. In addition, according to this, there is an effect that it is possible to classify mental health conditions with one artificial neural network even for multimodal without the need to separately construct different artificial neural networks according to modalities.

Regarding the learning session of the multimodal encoder module, FIG. 16 is a schematic diagram showing a learning session of the multimodal encoder module according to a modified example of the present invention. As shown in FIG. 16, in the learning session of the multimodal encoder module, the multimodal encoder module is configured to include a learning target model and a correct answer model having the same structure, and a mental health state classification device using a multimodal artificial neural network ( In 1), the learning session of the multimodal encoder module can be configured to be performed in the following steps. According to this, an effect of enabling self-learning is generated even if separate labeling data is not obtained for the multimodal encoder module.

① Regarding the masking step, when composing data composed of random concatenation for each specific time section step of consultation data (video, audio) and activity data (signal) as input data, video data, audio data, signal data Masking by randomly sampling one of them - block-wise masking that masks adjacent blocks together for video data, and masking spans of multimodal encoding vectors for audio data and signal data -

② Regarding the input data input step, input masked combination data as input data of the multimodal encoder module, which is the learning target model, and input unmasked original combination data as input data of the multimodal encoder module, which is the correct answer model.

③ In relation to the output data output step, the learning target multimodal encoding vector, which is the output data of the multimodal encoder module, which is the learning target model, and the correct answer multimodal encoding vector, which is the output data of the multimodal encoder module, which is the correct answer model, are obtained - at this time, the correct answer model The output data of the multimodal encoder module can be configured to use the Top K layer among all n transformer layers -

④ Regarding the parameter update step, the parameters of the multimodal encoder module, which is the learning target model, are minimized in the direction of minimizing the loss value of the loss function based on the difference (loss) between the learning target multimodal encoding vector and the correct multimodal encoding vector. and update the parameters of the multimodal encoder module, which is the correct answer model, by applying EMA (exponentially moving average) to the parameters of the multimodal encoder module, which is the learning target model.

In the parameter update step, the loss function of the multimodal encoder module, which is the learning target model, may be configured as in the following equation.

In the above equation, L is the loss function, y _t is the correct answer multimodal encoding vector, f _t (x) is the learning target multimodal encoding vector, and β is the smoothing adjustment variable. The degree of smoothing in the learning session can be adjusted by β configured in the loss function of the multimodal encoder module. According to this, overfitting of the multimodal encoder module, which is a learning target model, is prevented and labels are smoothed.

In addition, in the parameter update step, learning of the multimodal encoder model, which is the correct answer model, may be performed as in the following equation.

In the above equation, Δ is the weight of the multimodal encoder model, which is the correct answer model, θ is the weight of the multimodal encoder model, which is the learning target model, and τ is an adjustment variable. When τ is 1, only the weight of the correct answer model is used and τ is 0 It can be configured to perform learning of the multimodal encoder model, which is the correct answer model, using only the weight of the learning target model. According to this, a noise removal effect is generated for the learning of the correct answer model, so that the learning of the learning target model is possible more stably.

Regarding the mental health condition classification module, FIG. 17 is a schematic diagram showing a mental health condition classification module according to a modified example of the present invention. As shown in FIG. 17, the mental health condition classification module according to a modified example of the present invention uses a concatenated vector obtained by concatenating a plurality of multimodal encoding vectors generated according to a time step as input data, and uses the test subject's mental state as input data. It can be composed of an artificial neural network module that outputs mental health condition classification data including health status classification class (eg, depression classification data, PTSD prognosis classification data, suicide attempt possibility classification data, etc.) and confidence.

Regarding the specific configuration of the mental health state classification module, the mental health state classification module according to an embodiment of the present invention may be configured to include a Multi-head Self Attention Layer - FFNN - softmax. The Multi-head Self Attention Layer of the mental health condition classification module receives a concatenated vector obtained by concatenating a plurality of multimodal encoding vectors generated according to the time step as input data and performs positional encoding to combine location information, It refers to a layer that generates attention information by concatenating an attention value matrix generated by performing a plurality of self-attention operations (eg, scaled dot product attention) in parallel on a positionally encoded matrix. The feed forward neural network (FFNN) of the mental health condition classification module may mean a position-wise FFNN or a fully-connected FFNN. The input data of the FFNN of the mental health condition classification module is layer normalization of a combination vector obtained by concatenating a plurality of multimodal encoding vectors generated according to time step and a matrix in which the attention information is residually connected. ) can be configured with one value. The softmax of the mental health status classification module means an activation layer that has as input data the value obtained by layer normalization of the matrix in which the output matrix of FFNN is residual connection and mental health status classification data as output data. can According to this, the feature of each object location of the concatenated vector of a plurality of multimodal encoding vectors generated according to the time step by positional encoding also affects mental health state classification, and multi-head self attention Since a plurality of attention calculations are performed in parallel by the layer, attention of various values is utilized, resulting in the effect of enabling more sophisticated classification of mental health conditions.

At this time, the mental health status classification data of the examinee client 100 generated by the mental health status classification module of the examinee client 100 may be output on the display of the examinee client 100, and the examinee may determine the examinee's counseling data, activity data, and questionnaire items. Based on the data, a diagnosis result is generated by diagnosing the test subject, and the generated diagnosis result is compared with the test subject's mental health status classification data output from the mental health status classification module. Correction information reflecting the examiner's correction may be generated in the learning module.

The client learning module, when the examiner client 100 generates correction information reflecting the examiner's correction to the testee's mental health state classification data, multimodal mental health status of the examinee client 100 by using the corrected information as the ground truth A learning session of the mental health condition classification module may be performed to update parameters of an artificial neural network included in the mental health condition classification module, which is a component of the classification device 1 . At this time, the parameters of the multimodal encoder module can be separately self-supervised learning, and can be frozen in the learning session of the mental health condition classification module.

Regarding the learning session for the multimodal encoder module of the client learning module, the parameters of the multimodal encoder module may be separately self-supervised learning as described above.

Regarding the learning session for the mental health condition classification module of the client learning module, FIG. 18 is a schematic diagram showing the learning session of the mental health condition classification module according to an embodiment of the present invention. As shown in FIG. 18, the learning session of the mental health condition classification module of the client learning module may be configured to be performed when correction information is generated, and the time step output data of the multimodal encoder module to the mental health condition classification module. A concatenated vector obtained by concatenating a plurality of multimodal encoding vectors generated according to the above is input as input data, mental health status classification data is used as output data, and the mental health status classification data and correction information are modified. Parameters of the mental health condition classification module may be updated in a direction in which the loss of the ground truth is reduced (or in a direction in which the degree of similarity increases).

After the learning session of the mental health status classification module by the client learning module, the parameter upload module transfers the changed parameters of the multimodal encoder module and the changed parameters of the mental health status classification module to the federated learning module 220 of the main neural network server 200. This module is uploaded to . The parameter may include the gradient (g) of the multimodal encoder module and the mental health state classification module or the weight (w) of the artificial neural network after the learning session of the mental health state classification module. Parameter upload for the multimodal encoder module and the mental health condition classification module of the parameter upload module is configured to be performed when the correction information is generated.

In addition, the parameter upload module may be configured to apply noise (ε) to the changed parameters of the multimodal encoder module and the mental health condition classification module and upload them to the main neural network server 200. For example, if the parameter is the gradient (g), it can be configured to be uploaded as g+ε, and if the parameter is the weight (w), it can be configured to be uploaded as w+ε. In this case, the noise ε may be configured to have a positive value and a negative value randomly assigned so that the effect of the noise is minimally applied during federated learning in the federated learning module 220 . According to this, since noise is applied to the parameters in the examiner client 100 and uploaded to the main neural network server, even if a third party acquires the parameters, it is impossible to acquire the counseling data, activity data, and questionnaire data of the examinee.

The main neural network download module downloads the multimodal encoder main neural network and the mental health state classification main neural network pre-learned by the federated learning module 220 of the main neural network server 200 to generate the multimodal encoder module and the mental health state classification module. It is a module that replaces (replaces, transitions) at least some networks.

The network download of the multimodal encoder module downloads the multimodal encoder main neural network pretrained in the main neural network module 210 of the main neural network server 200, excluding the rear layers (last n layers) of the multimodal encoder module. It is configured to replace (replace, transfer) the rest to the downloaded multimodal encoder main neural network. According to this, since the neural network of the multimodal encoder module of each examiner client 100 is not completely replaced with the main neural network, the multimodal encoder module is continuously updated, while the effect of personalizing mental health status classification according to the counseling style of the testee is possible occurs.

The network download of the mental health state classification main neural network downloads the mental health state classification main neural network pre-learned in the main neural network module 210 of the main neural network server 200, and classifies the mental health state downloaded from the mental health state classification module. It is configured to substitute (replace, transfer) to the main neural network.

The main neural network server 200 may include a main neural network module 210 and a federated learning module 220, and parameters of a multimodal encoder module and a mental health condition classification module uploaded from a plurality of examiner clients 100 After collecting and updating the main neural network module 210, the server is configured to redistribute the pre-learned main neural network to the inspector client 100.

The main neural network module 210 may include a multimodal encoder main neural network and a mental health state classification main neural network, and the associated learning module 220 may update parameters with a specific gradient (g) or a specific weight (w). can be configured. The multimodal encoder main neural network refers to a main neural network corresponding to the multimodal encoder module, and the mental health condition classification main neural network refers to a main neural network corresponding to the mental health condition classification module.

The federated learning module 220 collects the parameters of the multimodal encoder main neural network and the mental health state classification main neural network uploaded from the plurality of testee clients 100, and then uses the collected parameters to perform the main neural network module 210. It is a module configured to update. At this time, in the method of updating the main neural network module 210 using the collected parameters, the correction change information, which is the difference between the correction information generated as follows and the information before correction (mental health status classification data), is used as the weight of the parameter, and the parameter The parameter may be averaged based on the modified change information by dividing the sum of the product of the modified change information and the modified change information by the sum of the modified change information.

When the parameter is the gradient (g), the federated learning module 220 may be configured to update the parameter of the main neural network module 210 as shown in the following equation.

In the above equation, w _new is the weight after updating the main neural network module 210, w _old is the weight before updating the main neural network module 210, α is the learning rate, and s _n is the correction uploaded from the tester client 100 n Change information g _n may be configured to mean a gradient uploaded from the inspector client 100 n.

When the parameter is weight (w), the federated learning module 220 may be configured to update the parameter of the main neural network module 210 as shown in the following equation.

In the above equation, w _new is the post-update weight of the main neural network module 210, s _n is the correction change information uploaded from the inspector client 100 n, and w _n means the weight uploaded from the inspector client 100 n can be configured.

According to this, during joint learning in the main neural network module 210, correction change information is used as a weight of a parameter, thereby generating a mini-batch effect. In addition, during federated learning in the main neural network module 210, correction change information is used as a weight of a parameter, thereby reducing network topology and asynchronous communication problems. In addition, when the difference between the corrected information and the pre-corrected information is large, an effect of having a greater influence on the update of the main neural network module occurs.

Regarding the recommendation of the art therapy program, the apparatus 1 for classifying mental health conditions using a multimodal artificial neural network according to a modified embodiment of the present invention may further include an art therapy program recommendation module. 19 is a schematic diagram showing an art therapy program recommendation module according to a modified example of the present invention. As shown in FIG. 19, the art therapy program recommendation module according to the modified example of the present invention may be configured in the examiner client 100, and may be connected to the art therapy program server and the art therapy program DB through a wired or wireless network, and the test subject For this, the mental health condition classification data output from the mental health condition classification module can be used as input data, and the recommended art therapy program data (art therapy program category, difficulty, color, etc.) can be used as output data, and the output recommended art therapy program It may be configured to transmit data to the art therapy application module of the test subject client (101).

In the art therapy program recommendation module according to a modified example of the present invention, reinforcement learning is applied to the recommendation of an art therapy program for the test subject based on the mental health state classification data output from the mental health state classification module for the test subject. can 20 is a schematic diagram showing reinforcement learning of an art therapy program recommendation module according to a modified example of the present invention. As shown in FIG. 20, in the case of reinforcement learning of the art therapy program recommendation module, the counseling data and activity data of the test subject are Environment, the art therapy program recommendation module is Agent, and the recommended art therapy program selection of the art therapy program recommendation module is Action. , The art therapy program recommendation module can be updated by configuring reinforcement learning in which the mental health status classification data is composed of State, and the difference (improvement degree) of mental health classification data before and after the art therapy program is composed of Reward. According to the reinforcement learning according to the modified example of the present invention, an effect of recommending an art therapy program that is expected to have the highest possibility of art therapy for the subject is generated according to the output of the subject's mental health status classification data.

As described above, those skilled in the art to which the present invention pertains will be able to understand that the present invention can be embodied in other specific forms without changing its technical spirit or essential features. Therefore, the above-described embodiments should be understood as illustrative in all respects and not restrictive. The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts should be construed as being included in the scope of the present invention.

The features and advantages described in this specification are not all inclusive, and many additional features and advantages will become apparent to those skilled in the art, particularly from consideration of the drawings, specification, and claims. Moreover, it should be noted that the language used herein has been chosen primarily for readability and instructional purposes, and may not have been chosen to delineate or limit the subject matter of the invention.

The foregoing description of embodiments of the present invention has been presented for purposes of illustration. It is not intended to limit the invention to the precise form disclosed or to make it without omission. Those skilled in the art can appreciate that many modifications and variations are possible in light of the above disclosure.

Therefore, the scope of the present invention is not limited by the detailed description, but by any claims of the application based thereon. Accordingly, the disclosure of embodiments of the invention is illustrative and not limiting of the scope of the invention set forth in the claims below.

1: Mental health condition classification device using multimodal artificial neural network
10: Activity Data Encoder
11: consultation data encoder
12: Mental health condition classification artificial neural network module
100: inspector client
101: test subject client
200: main neural network server
210: main neural network module
220: federated learning module
a: consultation data
b: activity data
c: Questionnaire data
A: Depression classification data
B: PTSD prognostic classification data

Claims (5)

A method for recommending an art therapy program, the method being performed by an art therapy program recommendation module included in a device, the method comprising:
Receiving mental health condition classification data for the test subject;
inputting the mental health state classification data into an artificial neural network module included in the art therapy program recommendation module;
outputting recommended art therapy program data by the artificial neural network module based on the mental health state classification data;
Transmitting the outputted recommended art therapy program data to the test subject client
including,
The recommended art therapy program data includes at least one of the category, difficulty, and color of the recommended art therapy program;
The artificial neural network module is updated by performing reinforcement learning,
In the reinforcement learning, the test subject's activity data composed of the test subject's counseling data and signal form is Environment, the art therapy program recommendation module is Agent, and the recommended art therapy program output from the art therapy program recommendation module is Action, and the mind The health status classification data is State, and the difference between the mental health classification data before and after applying the recommended art therapy program as art therapy to the test subject is composed of Reward.
How to recommend an art therapy program.
According to claim 1,
The mental health condition classification data is received from the mental health condition classification module,
The mental health condition classification module is trained to use the counseling data of the test subject in the form of a video or audio and the activity data of the test subject in the form of a signal as input data and output the mental health state classification data of the test subject as output data including artificial neural networks
How to recommend an art therapy program.
delete delete An apparatus including an art therapy program recommendation module,
The device includes a memory module for storing the program code of the art therapy program recommendation module; and
A processing module for processing the program code of the art therapy program recommendation module
including,
The program code of the art therapy program recommendation module is executed by the processing module,
Receiving mental health condition classification data for the test subject;
Inputting the mental health state classification data to an artificial neural network module included in the art therapy program recommendation module,
Outputting recommended art therapy program data by the artificial neural network module based on the mental health state classification data;
Operating to transmit the outputted recommended art therapy program data to a test subject client;
The recommended art therapy program data includes at least one of the category, difficulty, and color of the recommended art therapy program;
The artificial neural network module is updated by performing reinforcement learning,
In the reinforcement learning, the test subject's activity data composed of the test subject's counseling data and signal form is Environment, the art therapy program recommendation module is Agent, and the recommended art therapy program output from the art therapy program recommendation module is Action, and the mind The health status classification data is State, and the difference between the mental health classification data before and after applying the recommended art therapy program as art therapy to the test subject is composed of Reward.
A device including an art therapy program recommendation module.

Images