KR20200031265A - Method and apparatus for early diagnosis of depression - Google Patents

Abstract

Provided are a method for early diagnosing depression and an apparatus thereof. According to the present invention, the method for early diagnosing depression comprises the steps of: generating a behavioral indicator for early diagnosing depression by collecting behavioral data while a user is performing a decision-making task based on reinforcement learning; analyzing a task performance strategy through a calculation model by using the behavioral data of a user that is collected while performing the task and generating a model-based indicator for early diagnosing depression; analyzing a brain image acquired while performing the task by using the model-based indicator and generating a brain area information amount indicator for early diagnosing depression; analyzing brain connectivity by using the brain area information amount indicator and generating a brain connectivity indicator for early diagnosing depression; and learning the indicators based on machine learning to generate a predictive model for early diagnosing depression.

Description

Method and apparatus for early diagnosis of depression

The present invention grasps an individual's task execution strategy during execution of reinforcement learning-based decision making tasks using a computational model, and analyzes an individual's internal learning strategy, learning signal, and brain image signal based on the analysis to prematurely depress depression. It relates to a method and apparatus for diagnosis.

Dopamine is essential for both model-free learning and model-based learning. Many previous studies report the role of dopamine in RPE induction (Schultz W et al., 1997; Hollerman et al., 1998), and recent studies suggest an essential role of dopamine in associative learning (Sharpe MJ et al., 2017). This means dopamine's involvement in SPE expression. However, depression is characterized by reduced dopamine levels (Dunlop and Nemeroff 2007; Malhi and Berk 2007), and thus will be a barrier to learning both model-free systems and model-based systems. You can. Indeed, in many studies, dull RPE was found with depression under various experimental conditions (Pavlovian learning: Kumar et al., 2008, instrumental learning: Gradin et al., 2011; Dombrovski et al., 2013; Bettina et al., 2015; Rothkirch et al., 2017; Kumar Et al., 2018). Consistent with previous RPE deficiencies, the present invention further suggests a depressive effect on SPE expression.

One of the brain regions associated with the intervention process, FPC (Lee et al., 2014), shows that the response to the maximum reliability signal gradually decreases as people become depressed. This indicates that depressed people lose the ability to compare the reliability calculated in a model-based system and a model-free system.

It also confirms that with depression, the intervention system becomes sensitive to the model-free system. One of the model parameters that control the rate of update of model-free reliability is positively correlated with depression levels, which means that the reliability of the model-free system changes dramatically based on current observations. This is one indicator that learning in a model-free system becomes more sensitive to depression when compared to learning in a model-based system. In addition, the MVPA analysis suggests that the depressed group better predicts changes in model-free reliability signals in bilateral ilPFC and FPC. Consistent with the model parameter study results, these results indicate that the brain regions involved in the intervention process are sensitive to the model-free system.

The technical task to be achieved by the present invention is to identify an individual's task execution strategy during execution of a reinforcement learning-based decision task using a calculation model, and analyze an individual's internal learning strategy, learning signal, and brain image signal based on this. Therefore, I would like to propose a system that can diagnose depression early.

In one aspect, the method for early diagnosis of depression proposed in the present invention comprises: generating behavioral indicators for early diagnosis of depression by collecting behavioral data while a user performs a decision task based on reinforcement learning; Analyzing a task performance strategy through a calculation model using the user's behavior data collected during the task performance, and generating a model-based indicator for early diagnosis of depression; Analyzing brain images obtained during the task using the model-based indicator and generating a brain region information amount indicator for early diagnosis of depression; Analyzing brain connectivity using the brain region information amount indicator and generating a brain connectivity indicator for early diagnosis of depression; And learning the indicators based on machine learning to generate a predictive model for early diagnosis of depression.

Analyzing the task performance strategy through a computational model using the user's behavior data collected during the task performance, and generating a model-based indicator for early diagnosis of depression include model-based impact and model-free for decision making It is based on a model that distinguishes the effects of, and describes the integration of a model-based system with a model-free system.

In the evaluation phase, the model-based system and the model-free system learn behavioral values in a given state, and the model-based system uses the FORWARD learning algorithm to state the state prediction error (SPE). It learns the probability of transition and calculates the action value using the backward algorithm.

In the evaluation phase, the model-based system and the model-free system learn the behavior values in a given state, and the model-free system uses the SARSA algorithm to learn the state-behavior values with the Reward Prediction Error (RPE). do.

In the mediation process, the model-based system uses the history of the SPE to update the reliability to implement a layered empirical Bayes method.

In the arbitration process, the model-free system updates reliability using a Peer-Hall association rule based on an unsigned RPE.

In the action selection step, the action is probabilistically selected using softmax rules.

Analyzing the brain image obtained during the task using the model-based indicator, and generating a brain region information amount indicator for early diagnosis of depression, collecting brain images to reduce T1 equilibrium effects In order to remove the first two volumes, the EPI image is modified for slice timing and motion movement, and partially normalized to the standard template image provided by the SPM software.

Brain images are analyzed using general linear model analysis (GLM), subject-specific value signals and mediation signals are calculated as mediation models, and signals are compared to voxel-related signals in epi images, and regressors The order of (Regressor) is the prediction error of the model-based system and the prediction error of the model-free system, the reliability comparison signal, the selected value of the model-based system, the selected value of the model-free system, the selected integrated value and the unselected integration. The difference between the values and the order of likelihood of choosing a selected action is followed, and the ligase is sequentially non-orthogonalized in the GLM analysis to eliminate the effect of the ligencer sequence when interpreting the results.

Analyzing brain images using Multivoxel Pattern Analysis (MVPA), three things known to be involved in the mediation process, performed to understand various mediation signal processing in a region of interest (ROI) Inferior lateral prefrontal cortex (ilPFC) and Frontopolar Prefrontal Cortex (FPC), which are the ROIs of, are selected, and the masks of each brain region are functionally defined through GLM analysis. Reliability signals related to the mediation signal are updated when subjects experience prediction errors, and it is assumed that all epi scans taken from the current stimulus representation to the next stimulus and compensation representation appear to encode the reliability signal from the current representation, and each reliability signal In, the highest active signal of the highest percentage of each subject and the lowest signal value of the lowest percentage of each subject are used for analysis, and a binary-assisted vector machine classifier is applied to learn the characteristics of voxels, so that all voxels in the mask are used for learning. .

Analyzing brain connectivity using the brain region information level indicator and generating brain connectivity indicators for early diagnosis of depression include using a dynamic causal modeling (DCM) method to perform the brains of subjects during task performance Changes in connectivity between regions were observed. The brain regions used for the analysis are bilateral ilPFC, FPC, vmPFC, mPFC, and putamen (Lee et al., 2014).

In another aspect, the apparatus for early diagnosis of depression proposed in the present invention includes a behavior indicator generating unit that collects behavior data while a user performs a decision task based on reinforcement learning to generate behavior indicators for early diagnosis of depression; A model-based index generator that analyzes a task performance strategy through a computational model using the user's behavior data collected during the task performance and generates a model-based index for early diagnosis of depression; A brain region information amount indicator generating unit for analyzing brain images obtained during the task using the model-based indicator and generating a brain region information amount indicator for early diagnosis of depression; A brain connectivity index generator for analyzing brain connectivity using the brain region information level indicator and generating a brain connectivity indicator for early diagnosis of depression; And a depression diagnosis modeling unit for learning the indicators based on machine learning to generate a predictive model for early diagnosis of depression.

According to embodiments of the present invention, variables related to depression are identified based on an individual's behavioral tendency revealed as a result of task performance and an individual's learning signal analyzed by a calculation model, and a brain image obtained during task execution is modeled Based on the analysis, it is possible to find brain image indicators and connectivity indicators for early diagnosis of depression, and additionally, brain images can be analyzed by machine learning to find brain signal processing indicators for early diagnosis of depression. The discovered indicators can be learned based on machine learning to generate a depression prediction model.

1 is a flowchart illustrating a method for early diagnosis of depression according to an embodiment of the present invention.
2 is a view for explaining the subject structure according to an embodiment of the present invention.
3 is a graph showing behavioral results according to an embodiment of the present invention.
4 is a diagram for explaining model selection according to an embodiment of the present invention.
5 is a diagram for explaining a neural correlator for a calculated signal according to an embodiment of the present invention.
6 is a view for explaining information transfer connectivity between brain regions according to an embodiment of the present invention.
7 is a view showing the structure of an apparatus for early diagnosis of depression according to an embodiment of the present invention.
8 is a view showing the effect of depression on the prediction error expression according to an embodiment of the present invention.
9 is a view showing the effect of depression on the intervention system according to an embodiment of the present invention.

In the effect of depression on the prediction error representation, it was found that two types of prediction errors, SPE and RPE, which are distinguished according to an embodiment of the present invention, appear abnormally in a depressed brain. As a result of the neuroanalysis, the presence of depression weakened the SPE signal in the left insula and decreased RPE in the bilateral caudate. In addition, the degree of blunted signal expression correlates significantly with depressive symptoms.

In the effect of depression on the arbitration system, it implies maladaptive control of the intervention system. FPC (Lee et al., 2014), one of the brain regions involved in the intervention process, shows that the response to the maximum reliability signal gradually decreases as people become depressed. This indicates that depressed people lose the ability to compare the reliability calculated in a model-based system and a model-free system.

In terms of how depression affects judgment-action translation, depression entails sub-optimal action selection. The computational model shows that model parameters that determine exploitation in MB strategy are inversely related to depression levels, which makes exploratory decisions particularly for depressed people, especially when biased toward model-free systems. Indicates a tendency to fall. This explains the behavioral tendency that the consistency of choice decreases in the presence of depression.

In the presence of depression, two opposite choice coherence behaviors can be explained (Beever et al., 2013; Blanco et al., 2013). Beever et al. (2013) found that there was no significant difference in the search pattern in the situation of maximizing compensation between the general and depressed groups. On the other hand, Blanco et al. (2013) suggest that depressed groups tend to explore more. I think these conflicts originated from different task structures. Beever's study used a relatively static optimal policy to enable people to rely on a model-free system to accomplish their tasks. However, in the compensation structure used by Blanco, changes to the optimal policy occur more frequently, so people must use both model-based systems and model-free systems to make appropriate decisions. In terms of model parameters, Blanco's task requires more model-based systems, so the contribution of utilization of the MB plan in the behavior selection process is higher, resulting in lower utilization of the depressed group, resulting in more exploratory behavior choices. cause

The findings of the nervous system that the depression has a higher STG response also support the findings. Previous studies have suggested that STG responses increase and increase when people change their behavior rather than maintaining their behavior (Paulus et al., 2005). Depressed people tend to make irregular choices because they change their behavior due to smaller utilization parameters, which leads to a higher STG response.

Reduced utilization is also consistent with reduced compensation sensitivity in the event of depression. When considering utilization parameters in terms of compensation sensitivity (Huys et al., 2013), small utilization parameters give similar probabilities for all actions, indicating that the sensitivity to compensation is lower. Previous studies have reported a reduction in compensatory sensitivity in the compensation process in the presence of depression (Steele et al. 2007; Chen et al. 2015; Alloy et al. 2016), and further suggest that this tendency occurs, especially when model-based systems are very active do.

The system and method for early diagnosis of depression proposed in the present invention contributes to discovering an extended intervention model that can explain the dynamic search-utilization liver tendency in human decision making. Based on the proposed model, the effects of depression on different stages of the reinforcement learning process are characterized. In the numerical calculation process, the model-based system and the model-free system show that the prediction error signal expression in the neurocorrelates, the insula and the caudate, respectively, decreases. During the intervention process, the comparison signals of the model-based system and the model-free system appeared abnormally in the FPC region. In addition, the areas involved in the intervention process, bilateral ilPFC and FPC, became more sensitive to model-free systems in the presence of depression. These results imply that when there is depression, the proper balance between the two systems is broken. Finally, in the behavioral selection phase, depressed people have a hard time choosing the best policy because internal models drive exploratory decision making, especially when model-based systems dominate. This exploratory decision is related to increased activation in the STG.

The results of this study can explain both model-based disruption and model-free division in the presence of depression. This paradigm can also be applied to other mental disorders that show an improper balance between strengthening systems, such as addiction (Lucantonio et al., 2014) and impulse (Voon et al., 2014).

The current study is applicable to a variety of clinical applications. First, this result provides a basis for why conventional deep brain stimulation or repeated transcranial magnetic stimulation is effective in treating depression. Second, considering that the onset of MDD is approximately 25 to 45 years old (Kessler et al., 2007), because the average age of study participants in this study is relatively young (22.8 years old), it is possible to diagnose depression early. Showed the possibility. Lastly, the results of this study suggest that it can alleviate the symptoms of depression as a treatment that can alleviate increased compensatory predictive signals.

1 is a flowchart illustrating a method for early diagnosis of depression according to an embodiment of the present invention.

The proposed method for early diagnosis of depression collects behavior data while the user performs a decision task based on reinforcement learning to generate an action indicator for early diagnosis of depression (110), and collects user behavior data collected during the task execution. Analyzing the task performance strategy through the computational model using, generating a model-based indicator for early diagnosis of depression (120), analyzing the brain image obtained during the task using the model-based index, and early depression Generating a brain area information amount indicator for diagnosis (130), analyzing brain connectivity using the brain area information amount indicator, generating a brain connectivity index for early diagnosis of depression (140), and machine learning the indicators Learning based on, and generating a predictive model for early diagnosis of depression (150).

In step 110, the user collects behavioral data while performing a reinforcement learning-based decision-making task to generate behavioral indicators for early diagnosis of depression.

In step 120, the task performance strategy is analyzed through a computational model using the user's behavior data collected during the task performance, and a model-based indicator for early diagnosis of depression is generated.

It is based on a model that distinguishes model-based and model-free impacts on decision-making and describes the integration of model-based and model-free systems, which include evaluation, intervention, and behavioral selection. It consists of a process.

In the evaluation phase, the model-based system and the model-free system learn behavioral values in a given state, and the model-based system uses the FORWARD learning algorithm to state the state prediction error (SPE). It learns the probability of transition and calculates the action value using the backward algorithm. In addition, in the evaluation step, the model-based system and the model-free system learn behavioral values in a given state, and the model-free system uses the SARSA algorithm to calculate the state-behavior value as a Reward Prediction Error (RPE). To learn.

In the mediation process, the model-based system implements a layered empirical Bayes method using SPE's history to update reliability, and the model-free system is a Pearce-Hall based on an unsigned RPE. Reliability is updated using an association rule.

In the action selection step, the action is probabilistically selected using softmax rules.

In step 130, the brain image obtained during the task is analyzed using the model-based indicator, and an amount of brain region information for early diagnosis of depression is generated. Brain images are collected to remove the first two volumes to reduce T1 equilibrium effects. EPI images are modified for slice timing and motion movement, and partially normalized with standard template images provided by SPM software.

At this time, brain images are analyzed using general linear model analysis (GLM). The subject-specific value signal and the mediation signal are calculated as an arbitration model, and the signal is compared with the voxel-related signal in the epi image, and the order of the regressor is the prediction error of the model-based system and the prediction error of the model-free system. , The reliability comparison signal, the model-based system's selected value, the model-free system's selected value, the difference between the selected integrated value and the unselected integrated value, and the likelihood of choosing the selected action. The ligercers are sequentially non-orthogonalized in the GLM analysis to eliminate the effect of the ligersor order in interpreting the results.

In addition, brain images may be analyzed using Multivoxel Pattern Analysis (MVPA). It is performed to understand the processing of various mediation signals in a region of interest (ROI), and the three ROIs known to be involved in the mediation process, the inferior lateral prefrontal cortex (ilPFC) and the frontal region The frontal prefrontal cortex (FPC) is selected, and the masks of each brain region are functionally defined through GLM analysis. It is assumed that the reliability-related signals are updated when subjects experience a prediction error, and all epi scans taken from the current stimulus representation to the next stimulus and compensation representation appear to encode the reliability signal from the current representation. In each reliability signal, the highest active signal of the highest percentage of each subject and the lowest signal value of the lowest percentage of each subject are used for analysis, and a binary-assisted vector machine classifier is applied to learn the characteristics of voxels so that all voxels in the mask are trained. Used for

In step 140, brain connectivity is analyzed using the brain region information amount indicator, and brain connectivity indicators for early diagnosis of depression are generated. In the step of generating a brain connectivity index for early diagnosis of depression, the dynamic causal modeling (DCM) method was used to observe changes in connectivity between brain regions during subjects' task performance. The brain regions used for the analysis are bilateral ilPFC, FPC, vmPFC, mPFC, and putamen (Lee et al., 2014).

In step 150, the indicators are learned based on machine learning to generate a predictive model for early diagnosis of depression.

2 is a view for explaining the subject structure according to an embodiment of the present invention.

According to an embodiment of the present invention, 65 right-handed Koreans (28 women, average age 22.8 ± 3.8 years) participated in the study. Participants were recruited from the community through online announcements. Only 28 subjects underwent functional magnetic resonance imaging (fMRI) scans during the assignment. Two subjects whose total cumulative compensation was below the guess-level (average value of compensation using 10,000 random simulations) were excluded from the analysis. Thus, a total of 63 behavioral data and 28 fMRI data remained for analysis. To obtain an individual's level of depression, people were instructed to complete the Center for Epidemiologic Studies Depression (CES-D) questionnaire (Radloff, 1977) prior to the experiment (for the distribution of depression severity among participants). No subject had a history of neuropsychiatric disorders. All subjects provided written consent to the experimental rules approved by the Institutional Review Board (IRB) of the Korea Advanced Institute of Science and Technology (KAIST).

As shown in FIG. 2A, a sequential two-stage Markov decision task (Lee et al. 2014) was proposed to separate the model-based plan from the model-free plan. In this task, the subjects make a binary selection (either left or right) and then proceed to another step with a certain probability. When the next step appears on the screen, the participants make another judgment. After making two sequential judgments, the final stage of coin delivery is reached. Subjects performed 100 trials in the pre-learning session to learn the structure of the task. On average, four main sessions with 80 exams follow the pre-learning session. Participants were encouraged to take as many coins as possible in the main session.

As shown in FIG. 2B, this task is divided into a specific-goal condition and a flexible-goal condition. In a specific-target state, a box with a specific color (red, blue, yellow) appears, and subjects are monetarily compensated if the coin color is the same as that of the specific box. Otherwise, no compensation is given. On the other hand, in the flexible-target state, a white box appears at the beginning of the test and participants can get all kinds of coins.

Two types of state-transition probabilities are used to control the uncertainty of the environment to prevent continued dependence of one system on other systems. The state-transition probability is (0.5, 0.5) in high uncertainty environments and (0.9, 0.1) in low uncertainty situations.

In summary, (i) the specific-target state of the environment with high uncertainty, (ii) the specific-target state of the environment with low uncertainty, (iii) the flexible-target state of the environment with high uncertainty, and (iv) the environment with low uncertainty. Four blocks appear: flexible-target state. Four types of blocks appear in random order, with each block lasting for 4-6 trials. At the beginning of each test, participants can tell whether they are in a specific-goal or flexible-going state by the color of the box. However, no information on environmental uncertainty is provided.

The computational assumptions adopted in the present invention distinguish between model-based and model-free influences on decision-making, and are based on previous models (Lee et al., 2014) explaining the integration of the two systems. The model consists of three processes: valuation, arbitration, and action selectino.

In the evaluation phase, the model-based system and the model-free system learn action values in a given state. The model-based system uses the FORWARD learning algorithm (Glascher et al., 2010) to learn the state transition probability with the State Prediction Error (= 1- expected change probability, SPE), and calculate the behavior value with the BACKWARD algorithm do. On the other hand, the model-free system uses the SARSA algorithm (Sutton and Barto, 1998) to calculate the state-action value with the Reward Prediction Error (= actual value-expected value, RPE). To learn.

In the mediation process, the model-based system uses the history of the SPE to update the reliability and implements a hierarchical empirical Bayes method, while the model-free system uses unsigned RPE (unsigned RPE). Reliability is updated using the Pearce-Hall association rule based. This reliability signal competes with the push-pull mechanism to assign a weight to each system.

Finally, in the behavior selection phase, the model selects behavior stochastically using softmax rules (Daw et al., 2006, Glascher et al., 2010, Wilson et al., 2014).

In the present invention, two types of modified models have been proposed: a separate learning arbitration model and a dynamic exploitation arbitration model. The separate learning intervention model assumes different learning speeds of the model-based system and the model-free system. The dynamic utilization mediation model advances the segregation learning model using the additional assumption that the degree of exploitation, an indicator of RL agent policy optimization, is dependent on the dependence on both systems. Three types of equations, logistics, linear and weighted linear, have been proposed to explain changes in utilization.

First, fMRI data is acquired. Functional imaging was performed with a 3T Siemens (Magnetom) Verio scanner located at the KAIST Brain Imaging Center. 42 axial slices covering the entire brain are acquired in an interleaved-ascending sequence with a resolution of 3 mm x 3 mm x 3 mm. A single-shot echo-planar imaging pulse sequence was used (TR = 2800 ms, TE = 30 ms, FOV = 192 mm, flip angle = 90 °). High-resolution structural images were also acquired at a resolution of 0.7 mm x 0.7 mm x 0.7 mm for each subject.

Next, fMRI data is preprocessed. Images were processed and analyzed using SPM12 software (Braincommunication Lab Wellcome Division, London, UK). To reduce T1 equilibrium effects, the first two volumes were removed. EPI images were modified for slice timing and motion movement, and partially normalized to standard template images provided by SPM software.

In general linear model analysis (GLM), normalized images were smoothed using a 6mm FWHM Gaussian kernel and a high pass filter (128s cut-off) was applied to remove noise.

In multivoxel pattern analysis (MVPA), unsmoothed epi image data was used. De-trending and z-scoring were performed to remove linear trends and to match the signal range.

In General Linear Model Analysis, the subject-specific value signal and the intervention signal are calculated as an intervention model and the signal is compared to the voxel-related signal in the epi image. The order of the regressor is the prediction error (SPE) of the model-based system and the prediction error (RPE) of the model-free system, the reliability comparison signal (= max (RelMB, RelMF), maximum reliability), and the model-based system. Follow the order of the selected value (Qfwd) of the model, the selected value of the model-free system (Qsarsa), the difference between the selected integrated value and the unselected integrated value (Qarb), and the probability of choosing the selected action (Pchosen action). The regressors were sequentially non-orthogonalized in the GLM analysis to eliminate the effect of the regressor order in interpreting the results. MARSBAR software (http://marsbar.sourceforge.net) was used to extract parameter estimates from regions of interest (Brett M et al., 2002).

In Multivoxel Pattern Analysis, MVPA was performed to understand various interventional signal processing in a specific region of interest (ROI). Three ROIs known to be involved in the intervention process were selected: the inferior lateral prefrontal cortex (ilPFC) and the frontopolar prefrontal cortex (FPC). Masks in each brain region were functionally defined through GLM analysis. Clusters that respond to the maximum reliability signal remaining after full-brain correction (p <0.001, uncorrelated) were used as a mask for each ROI.

Reliability-related signals (MB reliability, MF reliability, Max reliability) are updated when subjects experience prediction errors (ie, when status and compensation appear). Assume that all epi scans taken from the current stimulus representation to the next stimulus / compensation representation appear to encode the reliability signal from the current representation, taking into account the BOLD response time of 4-6 seconds. For each reliability signal, the upper 33% highest active signal and the lower 33% lowest signal value of each subject were used for the analysis and classified into 'high value group' and 'low value group'.

A binary-assisted vector machine classifier was applied to learn the features of voxels. All voxels in the mask were used for learning. The dimensions of the data are 196, 79 and 294 for the left ilPFC, right ilPFC and FPC respectively. Training was conducted individually, and the average number of data for each subject was 349.9 in MB reliability, 548.5 in MF reliability, and 542.5 in MAX reliability. The data were randomly divided into 30 blocks, each block contained two groups of the same number, the classifier was trained using 29 blocks, and the other block was used for testing purposes. The generalized prediction performance was calculated by averaging 30 prediction accuracy. The whole process was implemented based on the Princeton Multi-Voxel Pattern Analysis toolbox (Detre et al., 2006). Finally, ANOVA analysis was used to compare the signal prediction accuracy between the general and depressive groups divided based on CES-D's standard cutoff criteria of 16 points (Comstock et al., 1997, Weissman et al., 1977).

Referring back to FIG. 2 (a), in the present invention, a sequential two-stage Markov decision task designed by Lee et al. (2014) was used. Each time a participant makes a binary selection (left or right), they move to the next state according to a particular state-transition probability p. The probability of transition corresponds to (0.5, 0.5) in an environment with high certainty, and (0.9, 0.1) in an environment with low certainty.

2 (b), the task is divided into two states: a specific target state and a flexible target state. Under a specific goal condition, subjects are allowed to earn coins only when the coin color matches the color of the token box (red (40), blue (20), yellow (10)). On the other hand, in a flexible goal condition, all types of tokens (red (40), blue (20), yellow (10)) are redeemable.

3 is a graph showing behavioral results according to an embodiment of the present invention.

3 (a) shows the relationship (n = 63) between the depression level (CES-D) and the accumulated reward. As your depression level increases, your ability to work decreases. Fig. 3 (b) shows the relationship between the depression level and the ratio of optimal selection (n = 63). The ratio of optimal choices is inversely related to the symptoms of depression. Fig. 3 (c) shows the relationship (n = 63) between depression level and selection consistency in the first state (S1). The rate of decrease in selection consistency is inversely correlated with depression.

4 is a diagram for explaining model selection according to an embodiment of the present invention.

FIG. 4 (a) is a computational assumption explaining the integration of a model-based (MB) learning system and a model-free (MF) system. The two separate systems calculate behavioral values and reliability using prediction errors (SPE and RPE, respectively, valuation). The model then determines the integrity of the two systems by comparing the reliability of the two systems (arbitration, arbitration). After integrating the two systems using estimated weights calculated during the intervention process, the behavior is selected based on the softmax rule. The model structure follows (Lee et al., 2014).

4 (b) is a model comparison result. In the present invention, Bayesian Information Criteria (BIC) was selected as a means for selecting a model that best describes human behavior data.

Referring to FIG. 4 (c), Arb refers to a calculation model proposed by (Lee et al., 2014). Additional versions of mediation consider separate learning and dynamic exploitation. Arbα assigns separate learning parameters to two different systems (ie, αMB, αMF instead of α). Arbα, τ is the progress of Arbα as an additional assumption that utilization varies depending on how much each system is used. The best version of the model uses the cumulative integration of model-based utilization (τMB) and model-free utilization (τMF) parameters with model-select probability (PMB) as utilization (τ) (i.e., τ = PMB * τMB + (1-PMB) * τMF). The asterisk (*) represents a significant difference at the 0.05 level through pair t-test comparison. The error bar represents the standard error mean.

5 is a diagram for explaining a neural correlator for a calculated signal according to an embodiment of the present invention.

The proposed model values and arbitration signals are shown in the above area. Neural correlates are consistent with (Lee et al., 2014). The SPE and RPE represent state prediction error and reward prediction error, respectively. The maximum reliability represents the maximum reliability of the model-based system and the model-free system (= max (RelMB, RelMF)). QMB and QMF represent values selected from model-based systems and model-free systems. Qarb represents the difference between the selected action value and the unselected action value. The Pchosen action represents the probability assigned to the selected action.

6 is a diagram for explaining information transfer connectivity between brain regions according to an embodiment of the present invention.

FIG. 6 (a) is a diagram for explaining information transfer connectivity between brain regions. In the process of generating brain connectivity indicators for early diagnosis of depression, the dynamic causal modeling (DCM) method was used to observe changes in connectivity between brain regions during subjects' tasks. Brain regions used for the analysis are bilateral ilPFC (inferior lateral PFC), FPC, vmPFC (Ventromedial PFC), mPFC (medial PFC), pPut (posterior Putamen), and ACC (anterior Cingulate Cortex) (Lee et al., 2014). The arrow indicates that the connectivity between the two regions is controlled by the PMF signal. PMF is a signal indicating how much the subject is currently using a model-free signal.

6 (b) is a graph showing the influence of PMF on the connection of mPFC from ilPFC.

7 is a view showing the structure of an apparatus for early diagnosis of depression according to an embodiment of the present invention.

The proposed early diagnosis device for depression 700 includes a behavioral indicator generator 710, a model-based indicator generator 720, a brain region information amount indicator generator 730, a brain connectivity index generator 740, and a depression diagnosis modeling unit. (750).

The behavioral indicator generator 710 collects behavioral data while the user performs a decision-based task based on reinforcement learning to generate behavioral indicators for early diagnosis of depression.

The model-based index generation unit 720 analyzes a task performance strategy through a computational model using the user's behavior data collected during the task performance, and generates a model-based index for early diagnosis of depression.

The brain region information amount indicator generation unit 730 analyzes the brain image acquired during the task using the model-based indicator and generates a brain region information amount indicator for early diagnosis of depression.

The brain connectivity index generator 740 analyzes brain connectivity using the brain region information amount indicator and generates a brain connectivity indicator for early diagnosis of depression.

The depression diagnosis modeling unit 750 learns the indicators based on machine learning to generate a predictive model for early diagnosis of depression.

The model-based index generation unit 720 is based on a model that distinguishes model-based influence and model-free influence on decision-making, and describes the integration of the model-based system and the model-free system, and the model Consists of a process that includes assessment, intervention, and behavioral choice.

In the evaluation step, the model-based index generation unit 720 learns behavior values in a given state in the model-based system and the model-free system, and the model-based system uses a forward learning algorithm to predict the state error. The state transition probability is learned with (State Prediction Error; SPE), and the action value is calculated with a backward algorithm. In addition, in the evaluation step, the model-based system and the model-free system learn behavioral values in a given state, and the model-free system uses the SARSA algorithm to compensate for the state-behavior with a reward prediction error (RPE). To learn.

The model-based index generation unit 720 implements a hierarchical empirical Bayes method using the history of the SPE in order to update the reliability of the model-based system in the mediation process. In addition, in the arbitration process, the model-free system updates reliability using a Peer-Hall association rule based on an unsigned RPE.

The model-based index generation unit 720 selects behavior stochastically using a softmax rule in the behavior selection step.

The brain region information amount index generation unit 730 collects brain images and removes the first two volumes to reduce T1 equilibrium effects. EPI images are modified for slice timing and motion movement, and partially normalized with standard template images provided by SPM software.

The brain region information amount index generation unit 730 analyzes the brain image using general linear model analysis (GLM), and the subject-specific value signal and the mediation signal are calculated as the mediation model, and the signal is calculated from the epi image. Compared to voxel-related signals. The order of the regressor is the prediction error of the model-based system and the prediction error of the model-free system, the reliability comparison signal, the selected value of the model-based system, the selected value of the model-free system, and the selected integrated value and not selected. The order between the differences between unintegrated values and the likelihood of choosing a selected action is followed, and the ligase is sequentially non-orthogonalized in the GLM analysis to eliminate the effect of the ligencer order when interpreting the results.

The brain region information amount index generation unit 730 analyzes the brain image by using Multivoxel Pattern Analysis (MVPA) and performs various intervention signal processing in a region of interest (ROI). do. Three ROIs known to be involved in the intervention process are selected: the left / right inferior lateral prefrontal cortex (ilPFC) and the frontopolar prefrontal cortex (FPC), and the masks in each brain region are analyzed by GLM. Is defined functionally. It is assumed that the reliability-related signals are updated when subjects experience a prediction error, and all epi scans taken from the current stimulus representation to the next stimulus and compensation representation appear to encode the reliability signal from the current representation. In each reliability signal, the highest active signal of the highest percentage of each subject and the lowest signal value of the lowest percentage of each subject are used for analysis, and a binary-assisted vector machine classifier is applied to learn the characteristics of voxels so that all voxels in the mask are trained. Used for

The brain connectivity index generating unit 740 observed a change in connectivity between brain regions during the task of the subjects using the dynamic causal modeling (DCM) method. Brain regions used for the analysis were bilateral left / right inferior lateral prefrontal cortex (ilPFC), frontal prefrontal cortex (FPC), ventraldial prefrontal cortex (vmPFC), medial These are the medial prefrontal cortex (mPFC) and the putamen (Lee et al., 2014).

8 is a view showing the effect of depression on the prediction error expression according to an embodiment of the present invention.

8 (a) shows the depression effect (n = 28) on the SPE response. The left insula ([-36, 20, -4], z-score = 4.49) is related to SPE expression. The estimated effect size between brain lobe activity and SPE is inversely related to depression levels. 8 (b) and 8 (c) show the depression effect (n = 28) on the RPE response. The estimated influence values between the bilateral caudates (left: [-4, 6, -4], z-score = 4.50, right: [4, 8 -4], z-score = 5.30), and RPE is It is inversely related to the symptoms of depression.

9 is a view showing the effect of depression on the intervention system according to an embodiment of the present invention.

9 (a) shows the effect of depression on model parameters (n = 63). The left side of the border line in FIG. 9 (a) shows the relationship between depression values and model-free reliability learning parameters. Depressed people show a higher learning rate, indicating that they tend to rely more on current compensation prediction errors in the model-free reliability learning process. The right side of the border in FIG. 9 (a) is an example of the effect of learning speed on reliability. The general group has a fixed reliability signal for both systems. On the other hand, the depressed group showed a rapid change in MF reliability due to the high learning rate. Fig. 9 (b) shows the relationship between depression levels and brain regions representing the intervention process (GLM analysis) (n = 28). The response of the frontal prefrontal cortex (FPC, coordinates [8,44,40], z-score = 3.83) decreases with increasing depression levels. FIG. 9 (c) is a comparison (MVPA analysis) of model-free reliability prediction ability in the general group (n = 15) and the depressed group (n = 13). Masks of the three brain regions associated with the mediation region, left / right ilPFC and FPC, are obtained through GLM analysis. MVPA analysis based on these regions showed that the predictive ability of the model-free reliability signal was significantly higher in all regions in the depressed group (p = 0.042, p = 0.047, p = 0.036, one-way ANOVA) ).

The device described above may be implemented with hardware components, software components, and / or combinations of hardware components and software components. For example, the devices and components described in the embodiments include, for example, a processor, controller, arithmetic logic unit (ALU), digital signal processor (micro signal processor), microcomputer, field programmable array (FPA), It may be implemented using one or more general purpose computers or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may run an operating system (OS) and one or more software applications running on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of understanding, a processing device may be described as one being used, but a person having ordinary skill in the art, the processing device may include a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that may include. For example, the processing device may include a plurality of processors or a processor and a controller. In addition, other processing configurations, such as parallel processors, are possible.

The software may include a computer program, code, instruction, or a combination of one or more of these, and configure the processing device to operate as desired, or process independently or collectively You can command the device. Software and / or data may be interpreted by a processing device, or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. Can be embodied in The software may be distributed on networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, or the like alone or in combination. The program instructions recorded on the medium may be specially designed and constructed for the embodiments or may be known and usable by those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and magnetic media such as floptical disks. -Hardware devices specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language code that can be executed by a computer using an interpreter, etc., as well as machine language codes produced by a compiler.

As described above, although the embodiments have been described by a limited embodiment and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques are performed in a different order than the described method, and / or the components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or other components Alternatively, even if replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (24)

Generating behavioral indicators for early diagnosis of depression by collecting behavioral data while a user performs a decision-making task based on reinforcement learning;
Analyzing a task performance strategy through a computational model using the user's behavior data collected during the task performance and generating a model-based indicator for early diagnosis of depression;
Analyzing brain images obtained during the task using the model-based indicator and generating a brain region information amount indicator for early diagnosis of depression;
Analyzing brain connectivity using the brain region information amount indicator and generating a brain connectivity indicator for early diagnosis of depression; And
Learning the indicators based on machine learning to generate a predictive model for early diagnosis of depression
Depression early diagnosis method comprising a. According to claim 1,
The step of analyzing the task performance strategy through the calculation model using the behavioral data of the user collected during the task execution, and generating a model-based indicator for early diagnosis of depression,
It is based on a model that distinguishes model-based and model-free impacts on decision-making and describes the integration of model-based and model-free systems, which include evaluation, intervention, and behavioral selection. Which consists of
How to diagnose depression early. According to claim 2,
In the evaluation phase, the model-based system and the model-free system learn behavioral values in a given state, and the model-based system uses the FORWARD learning algorithm to state the state prediction error (SPE). Learning the probability of transition, and calculating the behavioral value with the backward algorithm
How to diagnose depression early. According to claim 2,
In the evaluation phase, the model-based system and the model-free system learn the behavior values in a given state, and the model-free system uses the SARSA algorithm to learn the state-behavior values with the Reward Prediction Error (RPE). doing
How to diagnose depression early. According to claim 2,
In the mediation process, the model-based system uses the SPE's history to update the reliability to implement a layered empirical Bayes method.
How to diagnose depression early. According to claim 2,
In the arbitration process, the model-free system uses the Peer-Hall association rule based on unsigned RPE to update reliability.
How to diagnose depression early. According to claim 2,
In the action selection phase, we use the softmax rule to stochastically select actions
How to diagnose depression early. According to claim 1,
Analyzing the brain image obtained during the task using the model-based indicator, and generating a brain region information amount indicator for early diagnosis of depression,
Collect brain images to remove the first two volumes to reduce T1 equilibrium effects, EPI images are modified for slice timing and motion movement, and provided by SPM software Partially normalized to one standard template image
How to diagnose depression early. The method of claim 8,
Brain images are analyzed using general linear model analysis (GLM), subject-specific value signals and mediation signals are calculated as mediation models, and signals are compared to voxel-related signals in epi images, and regressors The order of (Regressor) is prediction error of model-based system and prediction error of model-free system, reliability comparison signal, selected value of model-based system, selected value of model-free system, selected integrated value and unselected integration The difference between the values and the order of likelihood of choosing the selected action is followed, and the ligase is sequentially non-orthogonalized in the GLM analysis to eliminate the effect of the ligencer order when interpreting the results.
How to diagnose depression early. The method of claim 8,
Analyzing brain images using Multivoxel Pattern Analysis (MVPA), three things known to be involved in the mediation process, performed to understand various mediation signal processing in a region of interest (ROI) Inferior lateral prefrontal cortex (ilPFC) and Frontopolar Prefrontal Cortex (FPC), which are the ROIs of, are selected, and the masks of each brain region are functionally defined through GLM analysis.
How to diagnose depression early. The method of claim 8,
Reliability-related signals are updated when subjects experience prediction errors, and it is assumed that all epi scans taken from the current stimulus representation to the next stimulus and compensation representation appear to encode the reliability signal from the current representation. The highest active signal of the highest constant ratio and the lowest signal value of the lower constant ratio are used for analysis, and the binary-assisted vector machine classifier is applied to learn the characteristics of the voxels, so that all voxels in the mask are used for learning.
How to diagnose depression early. According to claim 1,
Analyzing brain connectivity using the brain region information level indicator, and generating a brain connectivity indicator for early diagnosis of depression,
Dynamic Causal Modeling (DCM) method is used to observe changes in connectivity between brain regions during task execution.
How to diagnose depression early. A behavioral indicator generator that collects behavioral data while a user performs a decision-making task based on reinforcement learning and generates behavioral indicators for early diagnosis of depression;
A model-based index generator that analyzes a task performance strategy through a computational model using the user's behavior data collected during the task performance and generates a model-based index for early diagnosis of depression;
A brain region information amount indicator generating unit for analyzing brain images obtained during the task using the model-based indicator and generating a brain region information amount indicator for early diagnosis of depression;
A brain connectivity index generator for analyzing brain connectivity using the brain region information level indicator and generating a brain connectivity indicator for early diagnosis of depression; And
Depression diagnosis modeling unit that learns the indicators based on machine learning to generate a predictive model for early diagnosis of depression
Depression early diagnosis device comprising a. The method of claim 13,
The model-based index generation unit,
It is based on a model that distinguishes model-based and model-free impacts on decision-making and describes the integration of model-based and model-free systems, which include evaluation, intervention, and behavioral selection. Which consists of
Depression early diagnosis device. The method of claim 14,
The model-based index generation unit,
In the evaluation phase, the model-based system and the model-free system learn behavioral values in a given state, and the model-based system uses the FORWARD learning algorithm to state the state prediction error (SPE). Learning the probability of transition, and calculating the behavioral value with the backward algorithm
Depression early diagnosis device. The method of claim 14,
The model-based index generation unit,
In the evaluation phase, the model-based system and the model-free system learn the behavior values in a given state, and the model-free system uses the SARSA algorithm to learn the state-behavior values with the Reward Prediction Error (RPE). doing
Depression early diagnosis device. The method of claim 14,
The model-based index generation unit,
In the mediation process, the model-based system uses the SPE's history to update the reliability to implement a layered empirical Bayes method.
Depression early diagnosis device. The method of claim 14,
The model-based index generation unit,
In the arbitration process, the model-free system uses the Peer-Hall association rule based on unsigned RPE to update reliability.
Depression early diagnosis device. The method of claim 14,
The model-based index generation unit,
In the action selection phase, we use the softmax rule to stochastically select actions
Depression early diagnosis device. The method of claim 13,
The brain region information amount indicator generation unit,
Collect brain images to remove the first two volumes to reduce T1 equilibrium effects, EPI images are modified for slice timing and motion movement, and provided by SPM software Partially normalized to one standard template image
Depression early diagnosis device. The method of claim 20,
The brain region information amount indicator generation unit,
Brain images are analyzed using general linear model analysis (GLM), subject-specific value signals and mediation signals are calculated as mediation models, and signals are compared to voxel-related signals in epi images, and regressors The order of (Regressor) is the prediction error of the model-based system and the prediction error of the model-free system, the reliability comparison signal, the selected value of the model-based system, the selected value of the model-free system, the selected integrated value and the unselected integration. The difference between the values and the order of likelihood of choosing the selected action is followed, and the ligase is sequentially non-orthogonalized in the GLM analysis to eliminate the effect of the ligencer order when interpreting the results.
Depression early diagnosis device. The method of claim 20,
The brain region information amount indicator generation unit,
Analyzing brain images using Multivoxel Pattern Analysis (MVPA), three things known to be involved in the mediation process, performed to understand various mediation signal processing in a region of interest (ROI) Inferior lateral prefrontal cortex (ilPFC) and Frontopolar Prefrontal Cortex (FPC), which are the ROIs of, are selected, and the masks of each brain region are functionally defined through GLM analysis.
Depression early diagnosis device. The method of claim 20,
The brain region information amount indicator generation unit,
Reliability-related signals are updated when subjects experience prediction errors, and it is assumed that all epi scans taken from the current stimulus representation to the next stimulus and compensation representation appear to encode the reliability signal from the current representation. The highest active signal of the highest constant ratio and the lowest signal value of the lower constant ratio are used for analysis, and the binary-assisted vector machine classifier is applied to learn the characteristics of the voxels, so that all voxels in the mask are used for learning.
Depression early diagnosis device. The method of claim 13,
The brain connection index generation unit,
Dynamic Causal Modeling (DCM) method is used to observe changes in connectivity between brain regions during task execution.
Depression early diagnosis device.

Images