WO2023173538A1 - 阿尔兹海默症评估方法、系统、设备及存储介质 - Google Patents

阿尔兹海默症评估方法、系统、设备及存储介质 Download PDF

Info

Publication number
WO2023173538A1
WO2023173538A1 PCT/CN2022/089556 CN2022089556W WO2023173538A1 WO 2023173538 A1 WO2023173538 A1 WO 2023173538A1 CN 2022089556 W CN2022089556 W CN 2022089556W WO 2023173538 A1 WO2023173538 A1 WO 2023173538A1
Authority
WO
WIPO (PCT)
Prior art keywords
modal
data
interest
representation vector
degree
Prior art date
Application number
PCT/CN2022/089556
Other languages
English (en)
French (fr)
Inventor
赵婷婷
孙行智
徐卓扬
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2023173538A1 publication Critical patent/WO2023173538A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/30—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/168—Evaluating attention deficit, hyperactivity
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/40—Detecting, measuring or recording for evaluating the nervous system
    • A61B5/4076—Diagnosing or monitoring particular conditions of the nervous system
    • A61B5/4088—Diagnosing of monitoring cognitive diseases, e.g. Alzheimer, prion diseases or dementia
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
    • G16H10/60—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for patient-specific data, e.g. for electronic patient records
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS

Definitions

  • This application relates to the field of digital medical technology, which is mainly used in disease risk assessment, and in particular relates to an Alzheimer's disease assessment method, system, equipment and storage medium.
  • AD Alzheimer's disease
  • senile dementia a clinical syndrome characterized by progressive deterioration of memory and cognitive functions. It mostly occurs in the elderly, and its cause is still unknown. Those who develop the disease before the age of 65 are called Alzheimer's disease; those who develop the disease after the age of 65 are called senile dementia.
  • the clinical causes of AD mainly include family history, physical disease induction, head trauma, immune function and other factors; the clinical manifestations are divided into three stages, including mild dementia, memory loss, moderate dementia, severe memory loss, and severe dementia. Dementia, severe memory loss.
  • MMSE Mini-Mental Examination
  • ADL Assessment of Daily Living Activities
  • BPSD Behavior and Mental State Assessment
  • MMSE Mini-Mental Examination
  • ADL Assessment of Daily Living Activities
  • BPSD Behavior and Mental State Assessment
  • Blood and cerebrospinal fluid examinations are used to screen for organic causes
  • neuroimaging examinations are used to screen for neurological organic lesions and trauma
  • electroencephalogram spectrum examination genetic screening, etc.
  • problems with current examination methods are that the neuropsychological scale test is a subjective questionnaire test with low accuracy and reliability; genetic examination is expensive and difficult to implement, and it is also difficult for people who carry disease-prone genes.
  • AD Alzheimer's disease
  • This application provides an Alzheimer's disease assessment method, system, equipment and storage medium. Its main purpose is to mine the data correlation between different modalities of the target object, effectively improve the accuracy of feature expression, and improve the understanding of Alzheimer's disease. Accuracy of assessment of mutism.
  • embodiments of the present application provide an Alzheimer's disease assessment method, including:
  • multi-modal condition description data of the target object where the multi-modal condition description data includes condition observation data of the target object from different aspects;
  • the multi-modal features are input into the neural network evaluation model to evaluate whether the target object is in the early high-risk stage of Alzheimer's disease.
  • embodiments of the present application provide an Alzheimer's disease assessment system, including:
  • An acquisition module used to obtain multi-modal condition description data of the target object, where the multi-modal condition description data includes condition observation data of the target object from different aspects;
  • the fusion module is used to obtain the fusion features between any two aspects of the disease observation data in the multi-modal disease description data based on the multi-modal attention mechanism, and splice all the fusion features to obtain the multi-modal features. ;
  • An evaluation module is used to input the multi-modal features into a neural network evaluation model to evaluate whether the target object is in a high-risk stage for the early onset of Alzheimer's disease.
  • embodiments of the present application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor.
  • the processor executes the computer program, Following steps:
  • multi-modal condition description data of the target object where the multi-modal condition description data includes condition observation data of the target object from different aspects;
  • the multi-modal features are input into the neural network evaluation model to evaluate whether the target object is in the early high-risk stage of Alzheimer's disease.
  • inventions of the present application provide a computer storage medium.
  • the computer storage medium stores a computer program.
  • the computer program is executed by a processor, the following steps are implemented:
  • multi-modal condition description data of the target object where the multi-modal condition description data includes condition observation data of the target object from different aspects;
  • the multi-modal features are input into the neural network evaluation model to evaluate whether the target object is in the early high-risk stage of Alzheimer's disease.
  • the Alzheimer's disease assessment method, system, equipment and storage medium proposed in the embodiments of this application extract disease observation data from different aspects of the target object and integrate the disease observation data from different aspects through a multi-modal attention mechanism. Internal connections between different aspects of disease observation data. Since different aspects of disease observation data describe the condition of the same patient, the internal connection between different aspects of disease observation data can be mined through the multi-modal attention mechanism, thereby highlighting some of the important features.
  • the multi-modal features finally obtained can represent the condition of the target object more accurately and comprehensively, and combined with the neural network evaluation model, the evaluation ability and accuracy of the model are improved, so that the final evaluation of whether the target object is in Alzheimer's disease The results are more accurate in the early and high-risk stages of the disease.
  • Alzheimer's disease by identifying whether you are in a high-risk stage for the early onset of Alzheimer's disease, you can identify the early signs of Alzheimer's disease, provide early warning of the high probability of Alzheimer's disease, and effectively mediate early intervention. , no manual screening is required, reducing labor costs and improving screening efficiency.
  • Figure 1 is a schematic scene diagram of an Alzheimer's disease assessment method provided by an embodiment of the present application.
  • Figure 2 is a flow chart of an Alzheimer's disease assessment method provided by an embodiment of the present application.
  • Figure 3 is a flow chart of a fusion feature acquisition method provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of the attention mechanism in the embodiment of the present application.
  • Figure 5 is a schematic structural diagram of an Alzheimer's disease assessment system provided by an embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of a computer device provided in an embodiment of the present application.
  • AD Alzheimer's disease
  • a single modality of data is used, the characteristics of AD patients cannot be comprehensively represented, because the modalities are often highly correlated, but this correlation is between the feature layer and the data. Layer extraction is very difficult.
  • Traditional multi-modal feature fusion only considers integrating different types of features, and does not consider more about the connection between modalities.
  • the core idea of the attention mechanism (Attention) is to find the correlation between the original data and then highlight some of its important features. Therefore, this application can use the attention mechanism to find the correlation between multi-modalities, so as to consider the relationship between modalities more comprehensively.
  • the embodiment of this application proposes an Alzheimer's disease assessment method based on a multi-modal attention mechanism. Considering the complex clinical characteristics of Alzheimer's disease, using multi-modal data will make it easier to evaluate Alzheimer's disease. Patients with the disease have a more comprehensive characterization, because it is more likely to obtain an assessment model with higher accuracy. There are often internal correlations between different modal data of the same patient.
  • the embodiment of this application "based on the multi-modal attention mechanism, brain imaging data, mental and psychological assessment data , language data and electronic medical record data, and splice all the fusion features to obtain multi-modal features.”
  • each modality will " Observe” another modal feature, find the interesting part (ie, weight), and retain this information.
  • the final representation of the patient is obtained, and this representation is used as the input of the subsequent evaluation model to obtain the probability of Alzheimer's disease.
  • Figure 1 is a schematic scene diagram of an Alzheimer's disease assessment method provided by an embodiment of the present application.
  • the user inputs the multi-modal condition description data of the target object on the client, and the client extracts the target
  • the multi-modal condition description data is sent to the server.
  • the server executes the Alzheimer's disease assessment method and finally evaluates The probability that the target subject has Alzheimer's disease.
  • the server can be implemented as an independent server or a server cluster composed of multiple servers.
  • the client can be a smartphone, tablet, laptop, desktop computer, etc., but is not limited to this.
  • the client and the server can be connected through Bluetooth, USB (Universal Serial Bus, Universal Serial Bus) or other communication connection methods, and the embodiments of this application are not limited here.
  • Figure 2 is a flow chart of an Alzheimer's disease assessment method provided by an embodiment of the present application. As shown in Figure 2, the method includes:
  • the multi-modal condition description data includes condition observation data of the target object from different aspects
  • the multimodal condition description data of the target object is obtained.
  • the target object generally refers to the patient.
  • the multimodal condition description data includes observation and diagnosis data of the patient's condition from different aspects, such as the patient's brain imaging data, the patient's Psychological assessment data, patients’ daily language data, patients’ electronic medical record data, and patient-related indicator data, etc.
  • Brain imaging data refers to the patient's brain CT image data.
  • the brain CT image data of patients with different stages of cognitive impairment are different.
  • the image data is preprocessed first. Preprocessing operations include skull removal, registration, white matter segmentation, grayscale normalization and other operations. Registration is divided into multiple areas using the Brainnetome brain area template. Specifically, it can be divided into 246 areas, of which 36 areas are subcutaneous areas. , in the specific embodiment, only the upper layer of skin indicators can be selected, that is, 210 areas are selected for research.
  • FreeSurfer software is used to calculate multiple index characteristics of each brain region, which can include 8 index characteristics, including surface area, gray matter volume, average thickness, thickness standard deviation, integral corrected average curvature, integral corrected Gaussian curvature, folding index and Intrinsic curvature index, etc.
  • the patient's brain CT image data is granulated through these indicators, and the key indicators of the patient's brain CT image data are extracted. These key indicators are the disease observation data of the brain CT image.
  • the patient's mental psychological assessment data refers to the assessment of patients using relevant national or internationally recognized assessment standards.
  • Commonly used diagnostic standards include the Chinese Mental Disease Classification Scheme and Diagnostic Criteria, etc., allowing patients to answer relevant questions in the diagnostic standards, and to Each question is quantified and relevant quantitative data is extracted.
  • the quantitative data is the observation data of the patient's condition in terms of psychological assessment.
  • the daily language data of patients refers to the fact that patients with Alzheimer's disease generally have varying degrees of cognitive impairment, memory impairment, impairment of use, etc., and may have symptoms such as agnosia, aphasia, and apraxia. Different levels of symptoms are represented by different scores. , and its scores are used as disease observation data in the patient's daily language data.
  • the patient's electronic medical record data includes the patient's medical history, the patient's family history, the patient's psychological intelligence, routine examinations of the patient, patient lumbar puncture, patient MRI, patient PET and patient gene expression, etc.
  • the specific content can be determined according to the actual situation.
  • the embodiments of the present application are not specifically limited here.
  • the patient's medical history refers to the diseases the patient has suffered from before, especially including some serious diseases; the patient's family medical history refers to the diseases suffered by the patient's previous generations, and similarly, refers to some serious diseases; the patient's mental intelligence It is measured through the psychological intelligence calculation scale; routine examinations of patients include blood routine, thyroid function, homocysteine, etc.; patient lumbar puncture refers to the patient's lumbar puncture; patient MRI refers to the patient's When performing an MRI, a coronal view should be added to see if there is atrophy in the hippocampus and medial temporal lobe; PET should be checked on the patient to understand the brain metabolism; it is necessary to check gene expression on the patient to see if there is any association with Alzheimer's disease gene fragments.
  • Patient-related indicator data refers to the use of some indicators to mark the disease-related conditions of Alzheimer's disease, such as growth factor indicators, microglial inflammatory glia, astrocyte loss markers, etc. Specific indicators The data situation can be determined according to the actual situation, and is not specifically limited in the embodiments of this application.
  • S220 Based on the multi-modal attention mechanism, obtain the fusion features between any two aspects of the disease observation data in the multi-modal disease description data, and splice all the fusion features to obtain multi-modal features;
  • the obtained multi-modal condition description data includes observation data of the condition from all aspects, in the initial state, the observation data of the condition from all aspects are independent of each other and cannot be obtained. Knowing the correlation between observation data from different aspects, it is easy to understand that since the observation data from various aspects describe the same condition of the same patient, there must be some inevitable connection between the observation data from different aspects, and The traditional Alzheimer's disease assessment method does not take into account the intrinsic correlation between different aspects of observation data, which makes the subsequent feature expression incomplete and affects the accuracy of the assessment results.
  • a multi-modal attention mechanism is used to obtain the fusion characteristics between any two aspects of the condition observation data in the multi-modal condition description data.
  • the modal attention mechanism refers to mining the mutually interesting parts of observation data from different aspects in the multi-modal condition description data.
  • the multi-modal condition description data includes the patient's brain image data, the patient's mental state
  • Five aspects of data including psychological assessment data, patients' daily language data, patients' electronic medical record data, and patients' related indicator data, need to obtain the fusion characteristics between any two aspects of observation data, and use the patient's brain image to Data and the patient's mental and psychological assessment data are taken as an example to illustrate.
  • the interesting part of the patient's brain impact data on the patient's mental and psychological assessment data is called a fusion feature
  • the part of the patient's mental and psychological assessment data that affects the patient's brain data is called a fusion feature.
  • the part of interest is also called a fusion feature, but the content of the two fusion features is different.
  • this method the fusion features between any two aspects of observation data are obtained, and all the fusion features are spliced to obtain the multi-mode state characteristics.
  • this multi-modal feature also includes mutually interesting parts between different aspects of disease observation data, so that its feature expression is more accurate and comprehensive, and can improve the understanding of disease conditions. Precision and accuracy in Alzheimer's disease assessment.
  • the embodiments of this application use a multi-modal attention mechanism to fully explore the intrinsic connections between multiple different aspects of disease observation data, and take this connection into consideration in the assessment of Alzheimer's disease to obtain a more comprehensive view of the characteristics. representation, thereby improving the evaluation ability of the model.
  • S230 Input the multi-modal features into a neural network evaluation model to evaluate whether the target object is in a high-risk stage for the early onset of Alzheimer's disease.
  • the multimodal features are input into the neural network evaluation model to obtain the probability that the target object has Alzheimer's disease.
  • the neural network evaluation model is a machine learning model. Before use, it needs to be trained with samples and labels.
  • the samples are multi-modal obtained by other patients according to the same method.
  • the label indicates whether the patient is in the early high-risk stage of Alzheimer's disease, and the initial neural network evaluation model is trained using the samples and labels to obtain the trained neural network evaluation model.
  • the label is obtained through the following method: using gene sequencing assessment method for Alzheimer's disease patients, divided into levels 0 to 5, which respectively represent the incidence probability of Alzheimer's disease 0%, 20%, 40%, 60 %, 80% and 100%.
  • 60%, 80% and 100% are defined as the high-risk stage of disease. According to this definition, the sample is marked to see whether the sample is in the high-risk stage of disease.
  • the neural network evaluation model in the embodiment of this application belongs to a kind of neural network. Before using the neural network evaluation model, it also needs to be trained or updated. The neural network evaluation model is trained through the obtained samples and labels. train.
  • the training process of the neural network evaluation model can be divided into three steps: defining the structure of the neural network evaluation model and the output results of forward propagation; defining the loss function and back propagation optimization algorithm; finally generating sessions and iterating on the training data Run the backpropagation optimization algorithm.
  • neuron is the smallest unit that constitutes a neural network.
  • a neuron can have multiple inputs and one output.
  • the input of each neuron can be the output of other neurons or the input of the entire neural network.
  • the output of the neural network is the weighted sum of the inputs of all neurons, and the weights of different inputs are the neuron parameters.
  • the optimization process of the neural network is the process of optimizing the values of the neuron parameters.
  • the effect of the neural network and the optimization goal are defined through the loss function.
  • the loss function gives the calculation formula of the difference between the output result of the neural network and the real label.
  • Supervised learning is a way of training the neural network. The idea is On the labeled data set with known answers, the results given by the neural network should be as close as possible to the real answer (i.e. label). By adjusting the parameters in the neural network to fit the training data, the neural network provides evaluation capabilities for unknown samples.
  • the back propagation algorithm implements an iterative process. At the beginning of each iteration, a part of the training data is first taken, and the evaluation results of the neural network are obtained through the forward propagation algorithm. Because the training data all have correct answers, the gap between the evaluation results and the correct answers can be calculated. Based on this gap, the backpropagation algorithm will update the values of the neural network parameters accordingly, making it closer to the real answer.
  • the trained neural network evaluation model can be used for application.
  • An Alzheimer's disease assessment method proposed in the embodiment of this application extracts disease observation data from different aspects of the target object, and uses a multi-modal attention mechanism to fuse the internal connections between the disease observation data from different aspects.
  • Different aspects of disease observation data describe the condition of the same patient.
  • the multi-modal attention mechanism can mine the intrinsic connection between different aspects of disease observation data, thereby highlighting some of the important features, so that the final multi-modal Features can represent the condition of the target object more accurately and comprehensively, and combined with the neural network evaluation model, the evaluation capability and accuracy of the model can be improved, allowing the final assessment of whether the target object is in the early high-risk stage of Alzheimer's disease. The results are more accurate.
  • this method not only takes into account the unique clinical characteristics of Alzheimer's disease, but also solves the difficulty of multi-modal data fusion, and multi-modal features represent and characterize patients more comprehensively and accurately, further improving the Feature representation accuracy for multimodal data fusion.
  • Alzheimer's disease by identifying whether you are in a high-risk stage for the early onset of Alzheimer's disease, you can identify the early signs of Alzheimer's disease, provide early warning of the high probability of Alzheimer's disease, and effectively mediate early intervention. , no manual screening is required, reducing labor costs and improving screening efficiency.
  • obtaining the fusion features between any two aspects of disease observation data in the multi-modal disease description data includes:
  • a fusion feature between the condition observation data of one aspect and the condition observation data of the other aspect is obtained.
  • Figure 3 is a flow chart of a fusion feature acquisition method provided by an embodiment of the present application.
  • the fusion features between any two aspects of disease observation data in the multi-modal disease description data are obtained in the following manner, Take the patient's brain imaging data and the patient's mental and psychological evaluation data as an example.
  • the patient's brain imaging data and the patient's mental and psychological evaluation data represent the observation results of the patient from two different perspectives, because they are both from the same patient. characteristics, so there is a certain correlation between the two observation data.
  • the patient's brain imaging data and the patient's mental psychological assessment data are just discrete data, so they need to be represented as vectors.
  • the essential idea of the attention mechanism is to obtain the weight coefficient of the value corresponding to each key by calculating the similarity or correlation between the query and each key, and then perform a weighted sum of the values to obtain the final Attention value:
  • the function f(query,key) can be a similarity function, or other functions that can express the relationship between query and key.
  • the two features a′ and b′ are spliced together as the fusion feature of the patient, and the fusion feature is represented by x.
  • the step of obtaining the first degree of interest of the first representation vector to the second representation vector is to obtain the second degree of interest of the second representation vector to the first representation vector.
  • the degree of interest is obtained through the following formula:
  • S ab represents the first degree of interest
  • S ba represents the second degree of interest
  • a represents the first representation vector
  • b represents the second representation vector
  • f(a,b) represents a The degree of interest of each element in b to each element in b
  • f(b,a) represents the degree of interest of each element in b to each element in a.
  • f represents the similarity function.
  • f(a,b) represents the similarity of each element in a to each element in b
  • f(b,a) represents the similarity of each element in b to each element in a
  • the first cross-modal feature is obtained according to the first degree of interest and the second representation vector
  • the first cross-modal feature is obtained according to the second degree of interest and the first representation vector.
  • Represent a vector to obtain the second cross-modal feature which is obtained through the following formula:
  • a′ represents the first cross-modal feature
  • b′ represents the second cross-modal feature
  • S ab represents the first degree of interest
  • S ba represents the second degree of interest
  • a represents The first representation vector
  • b represents the second representation vector.
  • the expression formula of the neural network evaluation model is as follows:
  • W represents the probability of suffering from Alzheimer's disease
  • W represents the weight coefficient of the fully connected layer in the neural network evaluation model
  • p represents the bias
  • x represents the input data of the neural network evaluation model.
  • the multi-modal condition description data includes brain imaging data, mental psychological assessment data, language data and electronic medical record data.
  • the multi-modal condition description data of the target object also includes:
  • the brain imaging data, the mental psychological assessment data, the language data and the electronic medical record data are granulated to obtain the multi-modal condition description data.
  • the acquired multi-modal condition description data includes condition description data from all aspects, it is necessary to perform granular processing on these data to select the most representative data. , used to describe the multi-modal condition description data.
  • the embodiment of this application proposes an Alzheimer's disease assessment method based on a multi-modal attention mechanism. Considering the complex clinical characteristics of Alzheimer's disease, using multi-modal data will make it easier to evaluate Alzheimer's disease. Patients with the disease have a more comprehensive characterization, because it is more likely to obtain an assessment model with higher accuracy. There are often internal correlations between different modal data of the same patient.
  • the embodiment of this application "based on the multi-modal attention mechanism, brain imaging data, mental and psychological assessment data , language data and electronic medical record data, and splice all the fusion features to obtain multi-modal features.”
  • each modality will " Observe” another modal feature, find the interesting part (ie, weight), and retain this information.
  • the final representation of the patient is obtained, and this representation is used as the input of the subsequent evaluation model to obtain the probability of Alzheimer's disease.
  • Figure 5 is a schematic structural diagram of an Alzheimer's disease evaluation system provided by an embodiment of the present application. As shown in Figure 5, the system includes an acquisition module 510, a fusion module 520 and an evaluation module 530, where:
  • the acquisition module 510 is used to obtain multi-modal condition description data of the target object, where the multi-modal condition description data includes condition observation data of the target object from different aspects;
  • the fusion module 520 is used to obtain the fusion features between any two aspects of the disease observation data in the multi-modal disease description data based on the multi-modal attention mechanism, and splice all the fusion features to obtain the multi-modal features. ;
  • the evaluation module 530 is used to input the multi-modal features into the neural network evaluation model to evaluate whether the target subject is in the early high-risk stage of Alzheimer's disease.
  • This embodiment is a system embodiment corresponding to the above method embodiment, and its specific implementation process is the same as the above method embodiment. For details, please refer to the above method embodiment, and this system embodiment will not be described again here.
  • the fusion module includes a representation unit, an interest unit, a transmembrane state unit and a fusion unit, where:
  • the representation unit is used to obtain the first representation vector of the condition observation data in one aspect, and obtain the second representation vector of the condition observation data in the other aspect;
  • the interest unit is used to obtain a first degree of interest of the first representation vector on the second representation vector, and obtain a second degree of interest of the second representation vector on the first representation vector;
  • the transmembrane state unit is used to obtain the first cross-modal feature according to the first degree of interest and the second representation vector, and obtain the first cross-modal feature according to the second degree of interest and the first representation vector.
  • Two cross-modal features are used to obtain the first cross-modal feature according to the first degree of interest and the second representation vector, and obtain the first cross-modal feature according to the second degree of interest and the first representation vector.
  • the fusion unit is configured to obtain fusion features between the condition observation data of one aspect and the condition observation data of another aspect according to the first cross-modal feature and the second cross-modality feature.
  • the unit of interest is obtained by the following formula:
  • S ab represents the first degree of interest
  • S ba represents the second degree of interest
  • a represents the first representation vector
  • b represents the second representation vector
  • f(a,b) represents a The degree of interest of each element in b to each element in b
  • f(b,a) represents the degree of interest of each element in b to each element in a.
  • f represents the similarity function.
  • the transmembrane state unit is obtained by the following formula:
  • a′ represents the first cross-modal feature
  • b′ represents the second cross-modal feature
  • S ab represents the first degree of interest
  • S ba represents the second degree of interest
  • a represents The first representation vector
  • b represents the second representation vector.
  • the expression formula of the neural network evaluation model is as follows:
  • W represents the probability of suffering from Alzheimer's disease
  • W represents the weight coefficient of the fully connected layer in the neural network evaluation model
  • p represents the bias
  • x represents the input data of the neural network evaluation model.
  • the multi-modal condition description data includes brain imaging data, mental psychological assessment data, language data and electronic medical record data.
  • the acquisition module also includes an acquisition unit and a granulation unit. ,in:
  • the acquisition unit is used to acquire the brain imaging data, mental psychological assessment data, language data and electronic medical record data of the target object;
  • the granulation unit is used to granulate the brain imaging data, the mental psychological assessment data, the language data and the electronic medical record data to obtain the multi-modal condition description data.
  • Each module in the above-mentioned Alzheimer's disease assessment system can be implemented in whole or in part by software, hardware and combinations thereof.
  • Each of the above modules may be embedded in or independent of the processor of the computer device in the form of hardware, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
  • FIG. 6 is a schematic structural diagram of a computer device provided in an embodiment of the present application.
  • the computer device may be a server, and its internal structure diagram may be as shown in FIG. 6 .
  • the computer device includes a processor, memory, network interface, and database connected through a system bus. Wherein, the processor of the computer device is used to provide computing and control capabilities.
  • the memory of the computer device includes computer storage media and internal memory.
  • the computer storage medium stores operating systems, computer programs and databases. This internal memory provides an environment for the execution of operating systems and computer programs in computer storage media.
  • the database of the computer device is used to store data generated or obtained during the execution of the Alzheimer's disease assessment method, such as multi-modal disease description data, multi-modal features, etc.
  • the network interface of the computer device is used to communicate with external terminals through a network connection.
  • the computer program when executed by the processor implements an Alzheimer's disease assessment method.
  • a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor.
  • the processor executes the computer program, the Alzheimer's disease in the above embodiment is realized. Steps in the disease assessment method.
  • the processor executes the computer program, the functions of each module/unit in this embodiment of the Alzheimer's disease assessment system are implemented.
  • a computer storage medium is provided, and a computer program is stored on the computer storage medium.
  • the storage medium may be non-volatile or volatile.
  • the computer program implements the steps of the Alzheimer's disease assessment method in the above embodiment when executed by the processor.
  • the computer program implements the functions of each module/unit in the above embodiment of the Alzheimer's disease assessment system.
  • Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory may include random access memory (RAM) or external cache memory.
  • RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Synchlink DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
  • SRAM static RAM
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • DDRSDRAM double data rate SDRAM
  • ESDRAM enhanced SDRAM
  • SLDRAM synchronous chain Synchlink DRAM
  • Rambus direct RAM
  • DRAM direct memory bus dynamic RAM
  • RDRAM memory bus dynamic RAM
  • Module completion means dividing the internal structure of the device into different functional units or modules to complete all or part of the functions described above.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Public Health (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Pathology (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Developmental Disabilities (AREA)
  • Psychiatry (AREA)
  • Evolutionary Computation (AREA)
  • Primary Health Care (AREA)
  • Hospice & Palliative Care (AREA)
  • Psychology (AREA)
  • General Physics & Mathematics (AREA)
  • Child & Adolescent Psychology (AREA)
  • Epidemiology (AREA)
  • Artificial Intelligence (AREA)
  • Heart & Thoracic Surgery (AREA)
  • General Engineering & Computer Science (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Neurology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Educational Technology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Social Psychology (AREA)
  • Physiology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)

Abstract

一种阿尔兹海默症评估方法、系统、设备及存储介质，该方法包括：获取目标对象的多模态病情描述数据（S210）；基于多模态注意力机制，获取多模态病情描述数据中任意两个方面的病情观测数据之间的融合特征，并将所有融合特征进行拼接，得到多模态特征（S220）；将多模态特征输入到神经网络评估模型中，评估出目标对象是否处于阿尔兹海默症早期发病高风险阶段（S230）。所述方法通过提取目标对象不同方面的病情观测数据，通过多模态注意力机制来获取不同方面的病情观测数据之间的内部联系，提高该评估方法的准确度，并通过判别是否处于阿尔兹海默症早期发病高风险阶段，能提前对阿尔兹海默症高概率发病进行预警，无需人工筛查，降低人工成本，提高筛查效率。

Description

阿尔兹海默症评估方法、系统、设备及存储介质
本申请要求于2022年03月16日提交中国专利局、申请号为202210259559.0，发明名称为“阿尔兹海默症评估方法、系统、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
【技术领域】
本申请涉及数字医疗技术领域，主要应用于疾病风险评估，尤其涉及一种阿尔兹海默症评估方法、系统、设备及存储介质。
【背景技术】
阿尔兹海默症(Alzheimer Disease，简称AD)是一种以记忆功能和认知功能进行性退化为特征的临床综合征，多发于老年期，病因迄今未明。65岁以前发病者，称早老性痴呆；65岁以后发病者称老年性痴呆。AD的临床病因主要有家族史、躯体疾病诱发、头部外伤和免疫功能等其他因素；临床表现分为三个阶段，分别表现为轻度痴呆、记忆力减退，中度痴呆、记忆力严重减退，重度痴呆、严重记忆力丧失。
当前对阿尔兹海默症临床检查方法有：神经心理学测验，主要包括记忆力水平的简易精神量表(MMSE)、日常生活能力评估(ADL)、行为和精神状态评估(BPSD)量表等；血液和脑脊液检查，用以筛查器质性病因；神经影像学检查，用以筛查神经器质性病变和外伤；脑电图频谱检查；基因筛查等。当前检查方法存在的问题是，神经心理学量表测验是主观的问卷测试，其准确度和可信度较低；基因检查成本较高，且不易实施，对于携带易发病基因的人群，其也不必然形成AD早期的诊断因素，因此可能引导不必要的早期干预；神经心理学量表测验、血液和脑脊液检查、神经影像学检查、脑电图频谱检查这些检查方法主要的共同缺点在于，仅能应用在AD已发病阶段，即已经体现出临床症状时，用以确定发病程度，而无法做到早期筛查，并介导早期治疗。较晚的检查无法体现早期介入干预的筛查意义，使得病人错过了最优的早期干预期。
综上，上述已有方式中，发明人意识到无论是哪一种，均难以同时做到可靠的、易于实施的、更够在早期进行的筛查，需要人工来进行筛查，需要大量人力成本，费时长，效率低。因此如何降低人工成本，提高筛查效率是亟待解决的问题。
【发明内容】
本申请提供一种阿尔兹海默症评估方法、系统、设备及存储介质，其主要目的在于挖掘目标对象不同模态之间的数据关联关系，有效提高特征表达的准确性，提高对阿尔兹海默症的评估准确度。
第一方面，本申请实施例提供一种阿尔兹海默症评估方法，包括：
获取目标对象的多模态病情描述数据，所述多模态病情描述数据包括所述目标对象从不同方面的病情观测数据；
基于多模态注意力机制，获取所述多模态病情描述数据中任意两个方面的病情观测数据之间的融合特征，并将所有融合特征进行拼接，得到多模态特征；
将所述多模态特征输入到神经网络评估模型中，评估出所述目标对象是否处于阿尔兹海默症早期发病高风险阶段。
第二方面，本申请实施例提供一种阿尔兹海默症评估系统，包括：
获取模块，用于获取目标对象的多模态病情描述数据，所述多模态病情描述数据包括所述目标对象从不同方面的病情观测数据；
融合模块，用于基于多模态注意力机制，获取所述多模态病情描述数据中任意两个方面的病情观测数据之间的融合特征，并将所有融合特征进行拼接，得到多模态特征；
评估模块，用于将所述多模态特征输入到神经网络评估模型中，评估出所述目标对象是否处于阿尔兹海默症早期发病高风险阶段。
第三方面，本申请实施例提供一种计算机设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现以下步骤：
获取目标对象的多模态病情描述数据，所述多模态病情描述数据包括所述目标对象从不同方面的病情观测数据；
基于多模态注意力机制，获取所述多模态病情描述数据中任意两个方面的病情观测数据之间的融合特征，并将所有融合特征进行拼接，得到多模态特征；
将所述多模态特征输入到神经网络评估模型中，评估出所述目标对象是否处于阿尔兹海默症早期发病高风险阶段。
第四方面，本申请实施例提供一种计算机存储介质，所述计算机存储介质存储有计算机程序，所述计算机程序被处理器执行时实现以下步骤：
获取目标对象的多模态病情描述数据，所述多模态病情描述数据包括所述目标对象从不同方面的病情观测数据；
基于多模态注意力机制，获取所述多模态病情描述数据中任意两个方面的病情观测数据之间的融合特征，并将所有融合特征进行拼接，得到多模态特征；
将所述多模态特征输入到神经网络评估模型中，评估出所述目标对象是否处于阿尔兹海默症早期发病高风险阶段。
本申请实施例提出的一种阿尔兹海默症评估方法、系统、设备及存储介质，通过提取目标对象不同方面的病情观测数据，并通过多模态注意力机制来融合不同方面的病情观测数据之间的内部联系，由于不同方面的病情观测数据描述的是同一患者的病情，通过多模态注意力机制可以挖掘不同方面的病情观测数据之间的内在联系，从而突出其中某些重要特征，使得最终得到的多模态特征能更加准确、更加全面的表示目标对象的病情，并结合神经网络评估模型，提升模型的评估能力和准确度，使得最后评估出的目标对象是否处于阿尔兹海默症早期发病高风险阶段的结果更加准确。另外，通过判别是否处于阿尔兹海默 症早期发病高风险阶段，从而实现对阿尔兹海默症早期征兆进行判别，能提前对阿尔兹海默症高概率发病进行预警，并有效介导早期干预，无需人工筛查，实现降低人工成本，提高筛查效率。
【附图说明】
图1为本申请实施例提供的一种阿尔兹海默症评估方法的场景示意图；
图2为本申请实施例提供的一种阿尔兹海默症评估方法的流程图；
图3为本申请实施例提供的一种融合特征获取方法的流程图；
图4为本申请实施例中的注意力机制的图解示意图；
图5为本申请实施例提供的一种阿尔兹海默症评估系统的结构示意图；
图6为本申请实施例中提供的一种计算机设备的结构示意图。
本申请目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
【具体实施方式】
应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
AD是老年人中常见的一种神经性疾病，其主要影响老年人脑部的中枢神经，使其发生退行性变化，当患者出现明显的阿尔兹海默病临床症状时，往往已经到了晚期，在对AD疾病进行诊断的过程中，如果只使用单一模态的数据，那么不能全面的表示AD患者的特征，由于模态之间往往是高度相关的，但这种相关性在特征层和数据层提取难度都很大。传统的多模态特征融合仅仅考虑把不同类型的特征整合在一起，并没有更多的考虑模态之间的联系。注意力机制(Attention)的核心思想就是基于原有的数据找到其之间的关联性，然后突出其某些重要特征。因此本申请可以使用注意力机制来寻找多模态之间的关联性，从而更加全面的考虑模态之间的关系。
本申请实施例提出的一种基于多模态注意力机制的阿尔兹海默症评估方法，考虑到阿尔兹海默症疾病复杂的临床特点，使用多模态的数据会使得对阿尔兹海默症患者有更全面的特征刻画，因为更可能得到精度更高的评估模型。同一患者的不同模态数据之间往往存在着内部关联关系，为了让模型学习到这种内部关联关系，本申请实施例“基于多模态注意力机制，对脑部影像数据、精神心理测评数据、语言数据和电子病历数据中任意两者之间的融合特征，并将所有融合特征进行拼接，得到多模态特征”，使用交叉的注意力机制来获取这种内部联系，各模态会“观察”另一模态特征，发现感兴趣的部分(即权重)，将这种信息保留下来。最后通过拼接多模态特征，得到患者的最终表示，将此表示作为后续评估模型的输入，从而得到阿尔兹海默症的概率。通过这样的方法，不仅考虑了阿尔兹海默症疾病的独特临床特点，也解决了多模态数据融合的难点。
图1为本申请实施例提供的一种阿尔兹海默症评估方法的场景示意图，如图1所示，首先用户在客户端输入该目标对象的多模态病情描述数据，客户端提取到目标对象的多模态病情描述数据后，将该多模态病情描述数据发送给服务端，服务端接收到该多模态病情描述数据后，执行该一种阿尔兹海默症评估方法，最后评估出目标对象患有阿尔兹海默症 的概率。
需要说明的是，服务端可以用独立的服务器或者是多个服务器组成的服务器集群来实现。客户端可为智能手机、平板电脑、笔记本电脑、台式计算机等，但并不局限于此。客户端和服务端可以通过蓝牙、USB(Universal Serial Bus，通用串行总线)或者其他通讯连接方式进行连接，本申请实施例在此不做限制。
图2为本申请实施例提供的一种阿尔兹海默症评估方法的流程图，如图2所示，该方法包括：
S210，获取目标对象的多模态病情描述数据，所述多模态病情描述数据包括所述目标对象从不同方面的病情观测数据；
首先获取目标对象的多模态病情描述数据，该目标对象一般是指患者，多模态病情描述数据是包括从不同方面对该患者的病情观测诊断数据，比如患者的脑部影像数据、患者的精神心理测评数据、患者的日常语言数据、患者的电子病历数据和患者的相关指标数据等。
脑部影像数据是指患者的脑部CT影像数据，不同认知障碍阶段的患者脑部CT影像数据是不同的，获取到患者脑部CT影像数据后，先对该影像数据进行预处理，该预处理操作包括去头骨、配准、白质分割、灰度归一化等操作，配准是用Brainnetome脑区模板划分成多个区域，具体可以划分为246个区域，其中36个区域为皮下区域，在具体实施例中可以只选取皮上层指标，即选取210个区域用于研究。然后采用FreeSurfer软件计算每个脑区的多种指标特征，具体可以包括8种指标特征，包括表面面积、灰质体积、平均厚度、厚度标准差、积分矫正平均曲率、积分矫正高斯曲率、折叠指数和内在曲率指数等等，通过这些指标将患者脑部CT影像数据进行颗粒化，提取出患者脑部CT影像数据的关键指标，通过这些关键指标即为脑部CT影像方面的病情观测数据。
患者的精神心理测评数据是指利用相关国家或者国际公认的测评标准对患者进行测评，常用的诊断标准有中国精神疾病分类方案与诊断标准等，使得患者回答该诊断标准中的相关问题，并对每个问题进行量化，提取出相关量化数据，该量化数据即为心里测评方面该患者的病情观测数据。
患者的日常语言数据是指阿尔兹海默症患者一般存在不同程度的认知障碍、记忆障碍、受用障碍等，会存在失认、失语和失用等症状，不同程度的症状用不同的打分表示，将其打分作为患者的日常语言数据方面的病情观测数据。
患者的电子病历数据包换患者病史、患者家族病史、患者心理智力、对患者的常规检查、患者腰穿、患者核磁、患者PET和患者基因表达等等，具体包含内容可以根据实际情况进行确定，本申请实施例在此不做具体的限定。具体地，患者病史是指患者以前所患的病，尤其是包括一些严重的疾病；患者家族病史是指患者上下一代所患有的病，同样地，是指一些严重的疾病；患者心理智力是通过心理智力计算量表进行测评得到的；对患者的常规检查包括血常规、甲状腺功能、同型半胱氨酸等的检查；患者腰穿是指对患者进行腰 部穿刺；患者核磁是指对患者进行核磁共振检查，要加做冠状位，看海马，颞叶内侧是否有萎缩；对患者查PET，了解脑代谢情况；对患者有必要进行查基因表达，看是否有与阿尔兹海默症相关的基因片段。
患者的相关指标数据是指采用一些指标来标志阿尔兹海默症的相关患病情况，比如生长因子指标、小胶质细胞炎症胶质物、星型胶质细胞损失标志物等等，具体指标数据情况可以根据实际情况进行确定，本申请实施例在此不做具体的限定。
S220，基于多模态注意力机制，获取所述多模态病情描述数据中任意两个方面的病情观测数据之间的融合特征，并将所有融合特征进行拼接，得到多模态特征；
上一步中获取到多模态病情描述数据后，由于获取到的多模态病情描述数据包括各个方面对病情的观测数据，初始状态时，各个方面对病情的观测数据是相互独立的，无法得知不同方面观测数据之间的关联性，容易理解的是，由于各个方面对病情的观测数据是描述同一个患者的同一病情的，不同方面的观测数据之间必然存在某种必然的联系，而传统的阿尔兹海默症评估方法并没有考虑到不同方面的观测数据之间内在关联，从而使得后续的特征表达不全面，影响评估结果的准确性。
不同模态数据之间往往是高度相关的，但这种相关性在特征层和数据层提取难度都很大。传统的多模态特征融合仅仅考虑把不同类型的特征整合在一起，并没有更多的考虑模态之间的联系。注意力机制(Attention)的核心思想就是基于原有的数据找到其之间的关联性，然后突出其某些重要特征。因此我们可以使用注意力机制来寻找多模态之间的关联性，从而更加全面的考虑模态之间的关系。
本申请实施例中获取到多模态病情描述数据之后，采用多模态注意力机制，获取多模态病情描述数据中任意两个方面的病情观测数据之间的融合特征，此处，多模态注意力机制是指挖掘多模态病情描述数据中不同方面的观测数据相互感兴趣的部分，举例地，本申请实施例中多模态病情描述数据包括患者的脑部影像数据、患者的精神心理测评数据、患者的日常语言数据、患者的电子病历数据和患者的相关指标数据等五个方面的数据，需要获取其中任意两个方面的观测数据之间的融合特征，以患者的脑部影像数据和患者的精神心理测评数据为例进行说明，患者的脑部影响数据对患者的精神心理测评数据的感兴趣部分称之为融合特征，而患者的精神心理测评数据对患者脑部影响数据的感兴趣部分也称之为融合特征，但是这两个融合特征的内容是不同的，按照这个方法获取到任意两个方面观测数据之间的融合特征，并对所有融合特征进行拼接，得到多模态特征。该多模态特征除了包含多模态病情描述数据中不同方面的病情观测数据外，还包括不同方面的病情观测数据之间的相互感兴趣部分，从而其特征表达更加准确和全面，能够提高对阿尔兹海默症的评估精确度和准确度。
本申请实施例通过多模态注意力机制，充分挖掘多种不同方面的病情观测数据之间的内在联系，并将这种联系考虑到阿尔兹海默症的评估中，获得对特征的更全面的表示，从而提升模型的评估能力。
S230，将所述多模态特征输入到神经网络评估模型中，评估出所述目标对象是否处于阿尔兹海默症早期发病高风险阶段。
最后将多模态特征输入到神经网络评估模型中，得到该目标对象患有阿尔兹海默症的概率。
本申请实施例中，神经网络评估模型是一种机器学习模型，在使用之前，需要先利用样本和标签对其进行训练，本申请实施例中，样本为其它患者按照相同方法获取的多模态特征，标签为该患者是否处于阿尔兹海默症早期发病高风险阶段，利用样本和标签对初始的神经网络评估模型进行训练，即可得到训练之后的神经网络评估模型。
其中，标签是通过如下方法获得：对阿尔兹海默症患者采用基因测序评估的方法，分为0～5级，分别表征阿尔兹海默症的发病概率0％、20％、40％、60％、80％和100％，其中，本申请实施例中60％、80％和100％被定义为发病高风险阶段，按照该定义对样本进行标注，看样本是否处于发病高风险阶段。
本申请实施例中的神经网络评估模型属于神经网络中的一种，在使用该神经网络评估模型前，也需要对其进行训练或者更新训练，通过取得的样本和标签，对神经网络评估模型进行训练。该神经网络评估模型的训练过程可以分为三个步骤：定义神经网络评估模型的结构和前向传播的输出结果；定义损失函数以及反向传播优化的算法；最后生成会话并在训练数据上反复运行反向传播优化算法。
其中，神经元是构成神经网络的最小单位，一个神经元可以有多个输入和一个输出，每个神经元的输入既可以是其它神经元的输出，也可以是整个神经网络的输入。该神经网络的输出即是所有神经元的输入加权和，不同输入的权重就是神经元参数，神经网络的优化过程就是优化神经元参数取值的过程。
神经网络的效果及优化的目标是通过损失函数来定义的，损失函数给出了神经网络的输出结果与真实标签之间差距的计算公式，监督学习为神经网络训练的一种方式，其思想就是在已知答案的标注数据集上，该神经网络给出的结果要尽量接近真实的答案(即标签)。通过调整神经网络中的参数对训练数据进行拟合，使得神经网络对未知的样本提供评估能力。
反向传播算法实现了一个迭代的过程，每次迭代开始的时候，先取一部分训练数据，通过前向传播算法得到神经网络的评估结果。因为训练数据都有正确的答案，所以可以计算出评估结果和正确答案之间的差距。基于这个差距，反向传播算法会相应的更新神经网络参数的取值，使得和真实答案更加接近。
通过上述方法完成训练过程后，即可利用完成训练后的神经网络评估模型进行应用。
本申请实施例提出的一种阿尔兹海默症评估方法，通过提取目标对象不同方面的病情观测数据，并通过多模态注意力机制来融合不同方面的病情观测数据之间的内部联系，由于不同方面的病情观测数据描述的是同一患者的病情，通过多模态注意力机制可以挖掘不同方面的病情观测数据之间的内在联系，从而突出其中某些重要特征，使得最终得到的多 模态特征能更加准确、更加全面的表示目标对象的病情，并结合神经网络评估模型，提升模型的评估能力和准确度，使得最后评估出的目标对象是否处于阿尔兹海默症早期发病高风险阶段的结果更加准确。
通过这样的方法，不仅考虑了阿尔兹海默症疾病的独特临床特点，也解决了多模态数据融合的难点，并且多模态特征对患者的表示、刻画更加全面和准确，也进一步提升了多模态数据融合的特征表示准确性。
另外，通过判别是否处于阿尔兹海默症早期发病高风险阶段，从而实现对阿尔兹海默症早期征兆进行判别，能提前对阿尔兹海默症高概率发病进行预警，并有效介导早期干预，无需人工筛查，实现降低人工成本，提高筛查效率。
在上述实施例的基础上，优选地，所述获取所述多模态病情描述数据中任意两个方面的病情观测数据之间的融合特征，包括：
获取其中一个方面的病情观测数据的第一表示向量，获取另一个方面的病情观测数据的第二表示向量；
获取所述第一表示向量对所述第二表示向量的第一感兴趣程度，获取所述第二表示向量对所述第一表示向量的第二感兴趣程度；
根据所述第一感兴趣程度和所述第二表示向量，获取第一跨模态特征，根据所述第二感兴趣程度和所述第一表示向量，获取第二跨模态特征；
根据所述第一跨模态特征和所述第二跨模态特征，获取其中一个方面的病情观测数据和另一个方面的病情观测数据之间的融合特征。
图3为本申请实施例提供的一种融合特征获取方法的流程图，如图3所示，多模态病情描述数据中任意两个方面的病情观测数据之间的融合特征通过如下方式获取，以患者的脑部影像数据和患者的精神心理测评数据为例进行说明，患者的脑部影像数据和患者的精神心理测评数据代表从两个不同视角对患者的观测结果，因为都是同一患者的特征，因此这两种观测数据之间具有一定的关联关系。
刚开始患者的脑部影像数据和患者的精神心理测评数据只是一些离散的数据，因此需要将其表示为向量。
将患者的脑部影像数据经过一个表示层得到该模态的向量表示，即第一表示向量，用a表示，同理，将患者的精神心理测评数据经过一个表示层得到该模态的向量表示，即第二表示向量，用b表示。
注意力机制的本质思想即通过计算query和各个key的相似性或者相关性，得到每个key对应value的权重系数，然后对value进行加权求和，即得到了最终的Attention数值：
Attention＝f(query,key)*value，
其中函数f(query,key)可以是相似度函数，或者其它可以表示query和key关系的函数。
其中，a′＝S ab*b，b′＝S ba*a。
最后将a′和b′两部分特征拼接在一起，作为患者的融合特征，该融合特征用x表示。
在上述实施例的基础上，优选地，所述获取所述第一表示向量对所述第二表示向量的第一感兴趣程度，获取所述第二表示向量对所述第一表示向量的第二感兴趣程度，通过如下公式获得：
S ab＝f(a,b)；
S ba＝f(b,a)；
其中，S ab表示所述第一感兴趣程度，S ba表示所述第二感兴趣程度，a表示所述第一表示向量，b表示所述第二表示向量，f(a,b)表示a中每一元素对b中每一元素的感兴趣程度，f(b,a)表示b中每一元素对a中每一元素的感兴趣程度。
在上述实施例的基础上，优选地，f表示相似度函数。
相应地，f(a,b)表示a中每一元素对b中每一元素的相似度，f(b,a)表示b中每一元素对a中每一元素的相似度。
在上述公式中，计算第一表示向量对第二表示向量的感兴趣部分时，是通过相似度函数进行计算的，常见的相似度函数有Jaccard相关系数、余弦相似度、皮尔森相关系数、欧几里德距离等等。
在上述实施例的基础上，优选地，所述根据所述第一感兴趣程度和所述第二表示向量，获取第一跨模态特征，根据所述第二感兴趣程度和所述第一表示向量，获取第二跨模态特征，通过如下公式获得：
a′＝S ab*b；
b′＝S ba*a；
其中，a′表示所述第一跨模态特征，b′表示所述第二跨模态特征，S ab表示所述第一感兴趣程度，S ba表示所述第二感兴趣程度，a表示所述第一表示向量，b表示所述第二表示向量。
在上述实施例的基础上，优选地，所述神经网络评估模型的表示公式如下：
Figure PCTCN2022089556-appb-000001
其中，
Figure PCTCN2022089556-appb-000002
表示患有阿尔兹海默症的概率，W表示所述神经网络评估模型中全连接层的权重系数，p表示偏置，x表示所述神经网络评估模型的输入数据。
在上述实施例的基础上，优选地，所述多模态病情描述数据包括脑部影像数据、精神心理测评数据、语言数据和电子病历数据，所述获取目标对象的多模态病情描述数据之前还包括：
获取所述目标对象的脑部影像数据、精神心理测评数据、语言数据和电子病历数据；
对所述脑部影像数据、所述精神心理测评数据、所述语言数据和所述电子病历数据进行颗粒化处理，获取所述多模态病情描述数据。
本申请实施例中获取到多模态病情描述数据后，由于获取的多模态病情描述数据包括各个方面的病情描述数据，还需要对这些数据进行颗粒化处理，选出最具有代表性的数据，用来描述该多模态病情描述数据。
本申请实施例提出的一种基于多模态注意力机制的阿尔兹海默症评估方法，考虑到阿尔兹海默症疾病复杂的临床特点，使用多模态的数据会使得对阿尔兹海默症患者有更全面的特征刻画，因为更可能得到精度更高的评估模型。同一患者的不同模态数据之间往往存在着内部关联关系，为了让模型学习到这种内部关联关系，本申请实施例“基于多模态注意力机制，对脑部影像数据、精神心理测评数据、语言数据和电子病历数据中任意两者之间的融合特征，并将所有融合特征进行拼接，得到多模态特征”，使用交叉的注意力机制来获取这种内部联系，各模态会“观察”另一模态特征，发现感兴趣的部分(即权重)，将这种信息保留下来。最后通过拼接多模态特征，得到患者的最终表示，将此表示作为后续评估模型的输入，从而得到阿尔兹海默症的概率。通过这样的方法，不仅考虑了阿尔兹海默症疾病的独特临床特点，也解决了多模态数据融合的难点。
图5为本申请实施例提供的一种阿尔兹海默症评估系统的结构示意图，如图5所示，该系统包括获取模块510、融合模块520和评估模块530，其中：
获取模块510用于获取目标对象的多模态病情描述数据，所述多模态病情描述数据包括所述目标对象从不同方面的病情观测数据；
融合模块520用于基于多模态注意力机制，获取所述多模态病情描述数据中任意两个方面的病情观测数据之间的融合特征，并将所有融合特征进行拼接，得到多模态特征；
评估模块530用于将所述多模态特征输入到神经网络评估模型中，评估出所述目标对象是否处于阿尔兹海默症早期发病高风险阶段。
本实施例为与上述方法实施例相对应的系统实施例，其具体实施过程与上述方法实施例相同，详情请参考上述方法实施例，本系统实施例在此不再赘述。
在上述实施例的基础上，优选地，所述融合模块包括表示单元、感兴趣单元、跨膜态单元和融合单元，其中：
所述表示单元用于获取其中一个方面的病情观测数据的第一表示向量，获取另一个方面的病情观测数据的第二表示向量；
所述感兴趣单元用于获取所述第一表示向量对所述第二表示向量的第一感兴趣程度，获取所述第二表示向量对所述第一表示向量的第二感兴趣程度；
所述跨膜态单元用于根据所述第一感兴趣程度和所述第二表示向量，获取第一跨模态特征，根据所述第二感兴趣程度和所述第一表示向量，获取第二跨模态特征；
所述融合单元用于根据所述第一跨模态特征和所述第二跨模态特征，获取其中一个方面的病情观测数据和另一个方面的病情观测数据之间的融合特征。
在上述实施例的基础上，优选地，所述感兴趣单元通过如下公式获得：
S ab＝f(a,b)；
S ba＝f(b,a)；
其中，S ab表示所述第一感兴趣程度，S ba表示所述第二感兴趣程度，a表示所述第一表示向量，b表示所述第二表示向量，f(a,b)表示a中每一元素对b中每一元素的感兴趣 程度，f(b,a)表示b中每一元素对a中每一元素的感兴趣程度。
在上述实施例的基础上，优选地，f表示相似度函数。
在上述实施例的基础上，优选地，所述跨膜态单元通过如下公式获得：
a′＝S ab*b；
b′＝S ba*a；
其中，a′表示所述第一跨模态特征，b′表示所述第二跨模态特征，S ab表示所述第一感兴趣程度，S ba表示所述第二感兴趣程度，a表示所述第一表示向量，b表示所述第二表示向量。
在上述实施例的基础上，优选地，所述神经网络评估模型的表示公式如下：
Figure PCTCN2022089556-appb-000003
其中，
Figure PCTCN2022089556-appb-000004
表示患有阿尔兹海默症的概率，W表示所述神经网络评估模型中全连接层的权重系数，p表示偏置，x表示所述神经网络评估模型的输入数据。
在上述实施例的基础上，优选地，所述多模态病情描述数据包括脑部影像数据、精神心理测评数据、语言数据和电子病历数据，所述获取模块之前还包括获取单元和颗粒化单元，其中：
所述获取单元用于获取所述目标对象的脑部影像数据、精神心理测评数据、语言数据和电子病历数据；
所述颗粒化单元用于对所述脑部影像数据、所述精神心理测评数据、所述语言数据和所述电子病历数据进行颗粒化处理，获取所述多模态病情描述数据。
上述阿尔兹海默症评估系统中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。
图6为本申请实施例中提供的一种计算机设备的结构示意图，该计算机设备可以是服务器，其内部结构图可以如图6所示。该计算机设备包括通过系统总线连接的处理器、存储器、网络接口和数据库。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括计算机存储介质、内存储器。该计算机存储介质存储有操作系统、计算机程序和数据库。该内存储器为计算机存储介质中的操作系统和计算机程序的运行提供环境。该计算机设备的数据库用于存储执行阿尔兹海默症评估方法过程中生成或获取的数据，如多模态病情描述数据、多模态特征等。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机程序被处理器执行时以实现一种阿尔兹海默症评估方法。
在一个实施例中，提供了一种计算机设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，处理器执行计算机程序时实现上述实施例中的阿尔兹海默症评估方法的步骤。或者，处理器执行计算机程序时实现阿尔兹海默症评估系统这一实施例中的各模块/单元的功能。
在一实施例中，提供一计算机存储介质，该计算机存储介质上存储有计算机程序，所 述存储介质可以是非易失性，也可以是易失性。该计算机程序被处理器执行时实现上述实施例中阿尔兹海默症评估方法的步骤。或者，该计算机程序被处理器执行时实现上述阿尔兹海默症评估系统这一实施例中的各模块/单元的功能。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一非易失性计算机可读取存储介质中，该计算机程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将所述装置的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。
以上所述实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围，均应包含在本申请的保护范围之内。

Claims (22)

  1. 一种阿尔兹海默症评估方法，其中，包括：
    获取目标对象的多模态病情描述数据，所述多模态病情描述数据包括所述目标对象从不同方面的病情观测数据；
    基于多模态注意力机制，获取所述多模态病情描述数据中任意两个方面的病情观测数据之间的融合特征，并将所有融合特征进行拼接，得到多模态特征；
    将所述多模态特征输入到神经网络评估模型中，评估出所述目标对象是否处于阿尔兹海默症早期发病高风险阶段。
  2. 根据权利要求1所述的阿尔兹海默症评估方法，其中，所述获取所述多模态病情描述数据中任意两个方面的病情观测数据之间的融合特征，包括：
    获取其中一个方面的病情观测数据的第一表示向量，获取另一个方面的病情观测数据的第二表示向量；
    获取所述第一表示向量对所述第二表示向量的第一感兴趣程度，获取所述第二表示向量对所述第一表示向量的第二感兴趣程度；
    根据所述第一感兴趣程度和所述第二表示向量，获取第一跨模态特征，根据所述第二感兴趣程度和所述第一表示向量，获取第二跨模态特征；
    根据所述第一跨模态特征和所述第二跨模态特征，获取其中一个方面的病情观测数据和另一个方面的病情观测数据之间的融合特征。
  3. 根据权利要求2所述的阿尔兹海默症评估方法，其中，所述获取所述第一表示向量对所述第二表示向量的第一感兴趣程度，获取所述第二表示向量对所述第一表示向量的第二感兴趣程度，通过如下公式获得：
    S ab＝f(a,b)；
    S ba＝f(b,a)；
    其中，S ab表示所述第一感兴趣程度，S ba表示所述第二感兴趣程度，a表示所述第一表示向量，b表示所述第二表示向量，f(a,b)表示a中每一元素对b中每一元素的感兴趣程度，f(b,a)表示b中每一元素对a中每一元素的感兴趣程度。
  4. 根据权利要求3所述的阿尔兹海默症评估方法，其中，f表示相似度函数。
  5. 根据权利要求3所述的阿尔兹海默症评估方法，其中，所述根据所述第一感兴趣程度和所述第二表示向量，获取第一跨模态特征，根据所述第二感兴趣程度和所述第一表示向量，获取第二跨模态特征，通过如下公式获得：
    a′＝S ab*b；
    b′＝S ba*a；
    其中，a′表示所述第一跨模态特征，b′表示所述第二跨模态特征，S ab表示所述第一感兴趣程度，S ba表示所述第二感兴趣程度，a表示所述第一表示向量，b表示所述第二表示向量。
  6. 根据权利要求1所述的阿尔兹海默症评估方法，其中，所述神经网络评估模型的表示公式如下：
    Figure PCTCN2022089556-appb-100001
    其中，
    Figure PCTCN2022089556-appb-100002
    表示患有阿尔兹海默症的概率，W表示所述神经网络评估模型中全连接层的权重系数，p表示偏置，x表示所述神经网络评估模型的输入数据。
  7. 根据权利要求1至6任一所述的阿尔兹海默症评估方法，其中，所述多模态病情描述数据包括脑部影像数据、精神心理测评数据、语言数据和电子病历数据，所述获取目标对象的多模态病情描述数据之前还包括：
    获取所述目标对象的脑部影像数据、精神心理测评数据、语言数据和电子病历数据；
    对所述脑部影像数据、所述精神心理测评数据、所述语言数据和所述电子病历数据进行颗粒化处理，获取所述多模态病情描述数据。
  8. 一种阿尔兹海默症评估系统，其中，包括：
    获取模块，用于获取目标对象的多模态病情描述数据，所述多模态病情描述数据包括所述目标对象从不同方面的病情观测数据；
    融合模块，用于基于多模态注意力机制，获取所述多模态病情描述数据中任意两个方面的病情观测数据之间的融合特征，并将所有融合特征进行拼接，得到多模态特征；
    评估模块，用于将所述多模态特征输入到神经网络评估模型中，评估出所述目标对象是否处于阿尔兹海默症早期发病高风险阶段。
  9. 一种计算机设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其中，所述处理器执行所述计算机程序时实现如以下步骤：
    获取目标对象的多模态病情描述数据，所述多模态病情描述数据包括所述目标对象从不同方面的病情观测数据；
    基于多模态注意力机制，获取所述多模态病情描述数据中任意两个方面的病情观测数据之间的融合特征，并将所有融合特征进行拼接，得到多模态特征；
    将所述多模态特征输入到神经网络评估模型中，评估出所述目标对象是否处于阿尔兹海默症早期发病高风险阶段。
  10. 根据权利要求9所述的计算机设备，其中，所述获取所述多模态病情描述数据中任意两个方面的病情观测数据之间的融合特征，包括：
    获取其中一个方面的病情观测数据的第一表示向量，获取另一个方面的病情观测数据的第二表示向量；
    获取所述第一表示向量对所述第二表示向量的第一感兴趣程度，获取所述第二表示向量对所述第一表示向量的第二感兴趣程度；
    根据所述第一感兴趣程度和所述第二表示向量，获取第一跨模态特征，根据所述第二感兴趣程度和所述第一表示向量，获取第二跨模态特征；
    根据所述第一跨模态特征和所述第二跨模态特征，获取其中一个方面的病情观测数据 和另一个方面的病情观测数据之间的融合特征。
  11. 根据权利要求10所述的计算机设备，其中，所述获取所述第一表示向量对所述第二表示向量的第一感兴趣程度，获取所述第二表示向量对所述第一表示向量的第二感兴趣程度，通过如下公式获得：
    S ab＝f(a,b)；
    S ba＝f(b,a)；
    其中，S ab表示所述第一感兴趣程度，S ba表示所述第二感兴趣程度，a表示所述第一表示向量，b表示所述第二表示向量，f(a,b)表示a中每一元素对b中每一元素的感兴趣程度，f(b,a)表示b中每一元素对a中每一元素的感兴趣程度。
  12. 根据权利要求11所述的计算机设备，其中，f表示相似度函数。
  13. 根据权利要求11所述的计算机设备，其中，所述根据所述第一感兴趣程度和所述第二表示向量，获取第一跨模态特征，根据所述第二感兴趣程度和所述第一表示向量，获取第二跨模态特征，通过如下公式获得：
    a′＝S ab*b；
    b′＝S ba*a；
    其中，a′表示所述第一跨模态特征，b′表示所述第二跨模态特征，S ab表示所述第一感兴趣程度，S ba表示所述第二感兴趣程度，a表示所述第一表示向量，b表示所述第二表示向量。
  14. 根据权利要求9所述的计算机设备，其中，所述神经网络评估模型的表示公式如下：
    Figure PCTCN2022089556-appb-100003
    其中，
    Figure PCTCN2022089556-appb-100004
    表示患有阿尔兹海默症的概率，W表示所述神经网络评估模型中全连接层的权重系数，p表示偏置，x表示所述神经网络评估模型的输入数据。
  15. 根据权利要求9所述的计算机设备，其中，所述多模态病情描述数据包括脑部影像数据、精神心理测评数据、语言数据和电子病历数据，所述获取目标对象的多模态病情描述数据之前还包括：
    获取所述目标对象的脑部影像数据、精神心理测评数据、语言数据和电子病历数据；
    对所述脑部影像数据、所述精神心理测评数据、所述语言数据和所述电子病历数据进行颗粒化处理，获取所述多模态病情描述数据。
  16. 一种计算机存储介质，所述计算机存储介质存储有计算机程序，其中，所述计算机程序被处理器执行时实现以下步骤：
    获取目标对象的多模态病情描述数据，所述多模态病情描述数据包括所述目标对象从不同方面的病情观测数据；
    基于多模态注意力机制，获取所述多模态病情描述数据中任意两个方面的病情观测数据之间的融合特征，并将所有融合特征进行拼接，得到多模态特征；
    将所述多模态特征输入到神经网络评估模型中，评估出所述目标对象是否处于阿尔兹海默症早期发病高风险阶段。
  17. 根据权利要求16所述的存储介质，其中，所述获取所述多模态病情描述数据中任意两个方面的病情观测数据之间的融合特征，包括：
    获取其中一个方面的病情观测数据的第一表示向量，获取另一个方面的病情观测数据的第二表示向量；
    获取所述第一表示向量对所述第二表示向量的第一感兴趣程度，获取所述第二表示向量对所述第一表示向量的第二感兴趣程度；
    根据所述第一感兴趣程度和所述第二表示向量，获取第一跨模态特征，根据所述第二感兴趣程度和所述第一表示向量，获取第二跨模态特征；
    根据所述第一跨模态特征和所述第二跨模态特征，获取其中一个方面的病情观测数据和另一个方面的病情观测数据之间的融合特征。
  18. 根据权利要求17所述的存储介质，其中，所述获取所述第一表示向量对所述第二表示向量的第一感兴趣程度，获取所述第二表示向量对所述第一表示向量的第二感兴趣程度，通过如下公式获得：
    S ab＝f(a,b)；
    S ba＝f(b,a)；
    其中，S ab表示所述第一感兴趣程度，S ba表示所述第二感兴趣程度，a表示所述第一表示向量，b表示所述第二表示向量，f(a,b)表示a中每一元素对b中每一元素的感兴趣程度，f(b,a)表示b中每一元素对a中每一元素的感兴趣程度。
  19. 根据权利要求18所述的存储介质，其中，f表示相似度函数。
  20. 根据权利要求18所述的存储介质，其中，所述根据所述第一感兴趣程度和所述第二表示向量，获取第一跨模态特征，根据所述第二感兴趣程度和所述第一表示向量，获取第二跨模态特征，通过如下公式获得：
    a′＝S ab*b；
    b′＝S ba*a；
    其中，a′表示所述第一跨模态特征，b′表示所述第二跨模态特征，S ab表示所述第一感兴趣程度，S ba表示所述第二感兴趣程度，a表示所述第一表示向量，b表示所述第二表示向量。
  21. 根据权利要求16所述的存储介质，其中，所述神经网络评估模型的表示公式如下：
    Figure PCTCN2022089556-appb-100005
    其中，
    Figure PCTCN2022089556-appb-100006
    表示患有阿尔兹海默症的概率，W表示所述神经网络评估模型中全连接层的权重系数，p表示偏置，x表示所述神经网络评估模型的输入数据。
  22. 根据权利要求16所述的存储介质，其中，所述多模态病情描述数据包括脑部影 像数据、精神心理测评数据、语言数据和电子病历数据，所述获取目标对象的多模态病情描述数据之前还包括：
    获取所述目标对象的脑部影像数据、精神心理测评数据、语言数据和电子病历数据；
    对所述脑部影像数据、所述精神心理测评数据、所述语言数据和所述电子病历数据进行颗粒化处理，获取所述多模态病情描述数据。
PCT/CN2022/089556 2022-03-16 2022-04-27 阿尔兹海默症评估方法、系统、设备及存储介质 WO2023173538A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210259559.0 2022-03-16
CN202210259559.0A CN114628034A (zh) 2022-03-16 2022-03-16 阿尔兹海默症评估方法、系统、设备及存储介质

Publications (1)

Publication Number Publication Date
WO2023173538A1 true WO2023173538A1 (zh) 2023-09-21

Family

ID=81901647

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/089556 WO2023173538A1 (zh) 2022-03-16 2022-04-27 阿尔兹海默症评估方法、系统、设备及存储介质

Country Status (2)

Country Link
CN (1) CN114628034A (zh)
WO (1) WO2023173538A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115830017B (zh) * 2023-02-09 2023-07-25 智慧眼科技股份有限公司 基于图文多模态融合的肿瘤检测系统、方法、设备及介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019223508A1 (zh) * 2018-05-25 2019-11-28 深圳市前海安测信息技术有限公司 阿尔兹海默病症分级评估模型创建方法及计算机装置
CN111916207A (zh) * 2020-08-07 2020-11-10 杭州深睿博联科技有限公司 一种基于多模态融合的疾病识别方法及装置
CN112507947A (zh) * 2020-12-18 2021-03-16 宜通世纪物联网研究院(广州)有限公司 基于多模态融合的手势识别方法、装置、设备及介质
CN112967713A (zh) * 2021-01-23 2021-06-15 西安交通大学 一种基于多次模态融合的视听语音识别方法、装置、设备和存储介质
CN113317763A (zh) * 2021-06-30 2021-08-31 平安科技（深圳）有限公司 基于多模态的帕金森病检测装置及计算机可读存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019223508A1 (zh) * 2018-05-25 2019-11-28 深圳市前海安测信息技术有限公司 阿尔兹海默病症分级评估模型创建方法及计算机装置
CN111916207A (zh) * 2020-08-07 2020-11-10 杭州深睿博联科技有限公司 一种基于多模态融合的疾病识别方法及装置
CN112507947A (zh) * 2020-12-18 2021-03-16 宜通世纪物联网研究院(广州)有限公司 基于多模态融合的手势识别方法、装置、设备及介质
CN112967713A (zh) * 2021-01-23 2021-06-15 西安交通大学 一种基于多次模态融合的视听语音识别方法、装置、设备和存储介质
CN113317763A (zh) * 2021-06-30 2021-08-31 平安科技（深圳）有限公司 基于多模态的帕金森病检测装置及计算机可读存储介质

Also Published As

Publication number Publication date
CN114628034A (zh) 2022-06-14

Similar Documents

Publication Publication Date Title
CN109447183B (zh) 预测模型训练方法、装置、设备以及介质
US11810037B2 (en) Automatic patient recruitment system and methods for use therewith
Tong et al. Automated identification of retinopathy of prematurity by image-based deep learning
Wu et al. A deep learning, image based approach for automated diagnosis for inflammatory skin diseases
US10327637B2 (en) Systems, methods, and computer-readable media for patient image analysis to identify new diseases
US9524304B2 (en) Systems and methods for diagnosing inherited retinal diseases
EP3944185A1 (en) Computer-implemented method, system and computer program product for detecting a retinal condition from eye fundus images
US20220157470A1 (en) Method and system for identifying subjects who are potentially impacted by a medical condition
US20230360143A1 (en) Using images and voice recordings to facilitate underwriting life insurance
WO2010005656A2 (en) Brain condition assessment
Shah et al. Automated classification of normal and Stargardt disease optical coherence tomography images using deep learning
Rahim et al. Prediction of Alzheimer's progression based on multimodal deep-learning-based fusion and visual explainability of time-series data
CN114999629A (zh) 一种基于多特征融合的ad早期预测方法、系统、装置
Al-Adhaileh Diagnosis and classification of Alzheimer's disease by using a convolution neural network algorithm
WO2023173538A1 (zh) 阿尔兹海默症评估方法、系统、设备及存储介质
Huang et al. A structure-related fine-grained deep learning system with diversity data for universal glaucoma visual field grading
Niyas et al. A systematic review on early prediction of Mild cognitive impairment to alzheimers using machine learning algorithms
Romero-Aroca et al. Validation of a deep learning algorithm for diabetic retinopathy
US20220005566A1 (en) Medical scan labeling system with ontology-based autocomplete and methods for use therewith
Singh et al. Brain MRI Image Analysis for Alzheimer’s Disease (AD) Prediction Using Deep Learning Approaches
Ning et al. 3D convolutional neural networks uncover modality-specific brain-imaging predictors for Alzheimer’s disease sub-scores
WO2016127185A1 (en) Methods and approach for detection and prediction of change in dementia severity or clinical diagnosis over time
Chen et al. Multi-kernel Learning Fusion Algorithm Based on RNN and GRU for ASD Diagnosis and Pathogenic Brain Region Extraction
Irfan et al. Machine learning in neurological disorders: A multivariate LSTM and AdaBoost approach to Alzheimer's disease time series analysis
Mamyrbayev et al. INCREASING THE RELIABILITY OF DIAGNOSIS OF DIABETIC RETINOPATHY BASED ON MACHINE LEARNING.

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22931565

Country of ref document: EP

Kind code of ref document: A1