CN114359164A

CN114359164A - Method and system for automatically predicting Alzheimer disease based on deep learning

Info

Publication number: CN114359164A
Application number: CN202111510101.XA
Authority: CN
Inventors: 廖祥云; 唐咏梅; 王琼; 王平安
Original assignee: Shenzhen Institute of Advanced Technology of CAS
Current assignee: Shenzhen Institute of Advanced Technology of CAS
Priority date: 2021-12-10
Filing date: 2021-12-10
Publication date: 2022-04-15

Abstract

The invention discloses a method and a system for automatically predicting Alzheimer's disease based on deep learning. The method comprises the following steps: acquiring a target image to be detected; inputting a target image into a trained deep learning framework to obtain a prediction classification result of the Alzheimer's disease, wherein the deep learning framework sequentially comprises a coordinated attention model, a deep learning model and an excitation and extrusion attention model, the coordinated attention model respectively extracts independent position perception from different directions for an input feature map, and the obtained spatial information is weighted on a channel and then fused with the input feature map to obtain a first feature map; the deep learning model takes the first feature map as input and extracts a second feature map; and the excitation and extrusion attention model carries out extrusion operation on the second feature map to obtain the global features of the channels, and carries out excitation operation on the global features to obtain the weights of different channels. The invention obviously improves the prediction accuracy and reduces the workload of doctors.

Description

Method and system for automatically predicting Alzheimer disease based on deep learning

Technical Field

The invention relates to the technical field of image processing, in particular to a method and a system for automatically predicting Alzheimer's disease based on deep learning.

Background

Alzheimer's Disease (AD) is a progressive degenerative disease of the nervous system, manifested clinically as a decline in cognitive function, mental symptoms and behavioral disorders, a gradual decline in daily living capacity, which is slow to start with and cannot be judged when. With the increase of age, the probability of suffering from the disease of the old people is increased, the number of death people per year is also increased, and the health and normal family life of the old people are seriously influenced. Alzheimer's disease is irreversible, and after the disease is developed, the symptoms can be relieved only by means of drugs, so that the disease development process of patients cannot be changed. Since the pathogenesis is related to many aspects and is not amenable to simple drug therapy, screening and diagnosis at the earliest possible stage is necessary to take appropriate intervention before further cognitive impairment occurs. Mild Cognitive Impairment (MCI) is an early stage of dementia, in which patients experience a decline in cognitive function in one or more areas, but maintain independent daily living abilities, yet have not met the criteria for dementia. MCI can be classified into a stable cognitive disorder (stmci) in which the cognitive state remains stable and a progressive cognitive disorder (pMCI) in which the cognitive state gradually declines to progress to alzheimer's disease. Patients are therefore classified during the MCI stage, facilitating early intervention and treatment.

At present, the deep learning method for solving the medical image problem also becomes a research hotspot. The deep learning can automatically learn the relation between the tasks and the features without manually extracting the features, so that the recognition effects of different categories are improved. In recent years, deep learning, which exhibits excellent performance in image classification, is also beginning to be applied to the field of medical images, and deep learning models include a Recurrent Neural Network (RNN), a Convolutional Neural Network (CNN), a Deep Belief Network (DBN), a stacked self-encoder (SAE), and the like. Kanghan et al, based on unsupervised learning of AD and NC classification by Convolutional Automatic Encoder (CAE) and migratory learning for pMCI and stmci classification, presented an end-to-end concept for classification and data enhancement and regularization, applying a gradient back-propagation based visualization technique to the learned model. There are studies that propose fully stacked bidirectional long-term short-term memory FSBi-LSTM, and simultaneously analyze MRI and PET data, and the long-term memory LSTM can solve the problem of gradient explosion or gradient disappearance. MRI and PET are input into 3DCNN to extract features, and FSBi-LSTM is used for replacing a traditional full connection layer to extract high-level semantic and spatial information, and the FSBi-LSTM can acquire the spatial and semantic information from a feature map.

Through analysis, in the prior art, the traditional machine learning method generally extracts features from medical images in a manual mode and then uses a classifier for classification, and the method relies on prior knowledge to extract the features, so that deep analysis is needed on a data set, and time and labor are wasted. The voxel-based feature extraction method takes global information into consideration when processing 3D data, but requires a large amount of computation power and computation cost. The region-based extraction method focuses features on a specified region, reducing the amount of calculation for the whole picture, but inevitably ignores global structural information. In summary, when a specific simple task is processed by traditional machine learning, manual feature extraction is simple and effective, but generalization is poor, and the method is only suitable for the specific task. In the current scheme of automatically extracting features by using deep learning, the features extracted by different channels are endowed with the same weight in the convolution and pooling process of deep learning, but in an actual problem, the importance of different features on the classification effect should be different.

Disclosure of Invention

The present invention is directed to overcoming the above-mentioned deficiencies of the prior art and providing a method and system for automatic prediction of alzheimer's disease based on deep learning.

According to a first aspect of the present invention, there is provided a method for automatically predicting alzheimer's disease based on deep learning. The method comprises the following steps:

acquiring a target image to be detected;

inputting the target image into a trained deep learning framework to obtain a prediction classification result of the Alzheimer's disease, wherein the deep learning framework sequentially comprises a coordinated attention model, a deep learning model and an excitation and extrusion attention model, the coordinated attention model respectively extracts independent position perception from different directions for the input feature map, and the obtained spatial information is weighted on a channel and then fused with the input feature map to obtain a first feature map; the deep learning model takes the first feature map as input and extracts a second feature map; and the excitation and extrusion attention model carries out extrusion operation on the second feature map to obtain the global features of the channels, and carries out excitation operation on the global features to obtain the weights of different channels.

According to a second aspect of the present invention, there is provided a system for automatically predicting alzheimer's disease based on deep learning. The system comprises:

an image acquisition unit: the method comprises the steps of obtaining a target image to be detected;

a prediction unit: the system comprises a deep learning framework, an excitation and extrusion model and a first feature map, wherein the deep learning framework sequentially comprises a coordinated attention model, a deep learning model and an excitation and extrusion attention model; the deep learning model takes the first feature map as input and extracts a second feature map; and the excitation and extrusion attention model carries out extrusion operation on the second feature map to obtain the global features of the channels, and carries out excitation operation on the global features to obtain the weights of different channels.

Compared with the prior art, the method has the advantages that the Alzheimer disease is automatically predicted, early diagnosis of the Alzheimer disease is realized, an attention mechanism is added into a deep learning network structure, the problem of loss caused by different channel occupied importance is solved, the prediction accuracy is obviously improved, auxiliary work can be performed for diagnosis of doctors, and accordingly the workload of the doctors is reduced.

Other features of the present invention and advantages thereof will become apparent from the following detailed description of exemplary embodiments thereof, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention.

FIG. 1 is a schematic diagram of a deep learning framework for automatic prediction of Alzheimer's disease, according to one embodiment of the present invention;

FIG. 2 is a schematic diagram of a dense convolutional network model according to one embodiment of the present invention;

FIG. 3 is a schematic view of an excitation and compression attention model in accordance with one embodiment of the present invention;

FIG. 4 is a schematic diagram of a coordinated attention model according to one embodiment of the invention;

in the drawing, Conv-convolutional layer; CA Block-coordinate attention Module; dense Block-Dense Block; Avg-Pool-average pooling; SE Block-excitation and compression attention module; FC-full connection; global posing-Global pooling; fully Connected-full ligation; re-weight-readjust weight; non-liner-non-linear; transitions layers-Transition layers.

Detailed Description

Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.

In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

The invention provides a deep learning framework for automatically predicting Alzheimer disease, which mainly comprises a deep learning model, a coordinated Attention model (CA) and an Excitation and extrusion Attention model (SE) as shown in figure 1. In the following, an example based on MRI data as a target image is described, but it should be understood that the invention is equally applicable to other types of medical images, such as PET and CSF, which are expensive to examine, but CSF is an invasive way of examining.

Hereinafter, the deep learning model, the excitation and compression attention model (or called excitation and compression attention module), and the coordinated attention model (or called coordinated attention module) will be described in detail.

1) Deep learning model

The deep learning model can adopt various types, such as a convolutional neural network or a deep belief network. Considering that in the deep learning network, as the number of network layers increases, the problem that the gradient disappears when the input information and the gradient information are transmitted becomes more and more serious, in a preferred embodiment, a dense convolutional network (DenseNet) is used to solve the problem so as to ensure that the maximum information transmission can be performed between the layers in the network, all the previous layers are directly spliced, and the structure of the dense convolutional network is as shown in fig. 2.

The dense connection mode is equivalent to the direct connection input and loss of each layer, so that the gradient disappearance problem can be alleviated, and the network can be built deeper and deeper. By the connection mode, the transfer of the characteristics and the gradient is more effective, and each layer can utilize the initially input information to facilitate the training of the network. In addition, the dense convolutional network also strengthens the transfer of the characteristics, more effectively utilizes the characteristics and simultaneously reduces the parameter quantity to a certain extent

2) Excitation and compression attention model

As shown in fig. 1, the purpose of convolution is to aggregate spatial information and information in a characteristic dimension on a local receptive field, and a convolutional neural network is formed by a series of convolutional layers, nonlinear layers and downsampling layers, so that the network can capture information from a global receptive field. The convolution operation is defaulted to be carried out on all channels of the input feature map, attention is focused on the relationship among the channels by the SE model, and the importance degree of different channel features can be automatically learned.

Referring to fig. 3, the SE model first performs an extrusion operation on the obtained feature picture to obtain global features of the channels, then performs an excitation operation on the global features to learn the relationship between the channels, obtains weights of different channels, and finally gives weights to the original feature picture. The attention mechanism can enable the model to pay more attention to the channel features with the largest information quantity, enhance important features and restrain unimportant channel features. The SE model can also be easily integrated into existing networks, which can improve the performance of the network at a small cost. It should be understood that the excitation or activation operations involved may also employ other non-linear processing functions in addition to the Sigmoid function.

3) Model for coordinating attention

At present, the SE model only considers the internal channel information and ignores the importance of the position information, but the spatial structure of the target is very important in vision, and the CA model adds the position information to the channel attention, so that the network can obtain larger area information and simultaneously avoid large overhead. In order to avoid the loss of position information caused by global pooling, the CA model performs average pooling from the horizontal direction and the vertical direction respectively, and extracts features to efficiently integrate the spatial coordinate information.

Referring to fig. 4, the CA model extracts two separate location perceptions from the vertical and horizontal directions, encodes feature maps with information in specific directions, and fuses the acquired spatial information by weighting on channels. The CA model considers channel and position information at the same time, not only can capture information across channels, but also contains sensitive information of position and direction, and the model is flexible and light and is easy to insert into the existing network.

Accordingly, the present invention also provides a system for automatically predicting alzheimer's disease based on deep learning, which is used for implementing one or more aspects of the above method. For example, the system includes: the image acquisition unit is used for acquiring a target image to be detected; the prediction unit is used for inputting the target image into a trained deep learning framework to obtain a prediction classification result of the Alzheimer's disease, wherein the deep learning framework sequentially comprises a coordinated attention model, a deep learning model and an excitation and extrusion attention model, the coordinated attention model respectively extracts independent position perceptions from different directions for an input feature map, and the acquired spatial information is weighted on a channel and then fused with the input feature map to obtain a first feature map; the deep learning model takes the first feature map as input and extracts a second feature map; and the excitation and extrusion attention model carries out extrusion operation on the second feature map to obtain the global features of the channels, and carries out excitation operation on the global features to obtain the weights of different channels. The units involved can be implemented using dedicated hardware, processors or FPGAs.

In summary, the present invention combines two attention mechanisms with the DenseNet, wherein the attention mechanism is to find the focus for different pictures and focus on the area related to the target task according to the mechanism of brain processing visual signals. Dense concatenation of DenseNet has a strong regularization effect, which can reduce overfitting on a smaller training set. From the aspect of the characteristic channel, the SE automatically acquires the importance degree of each channel in a learning mode, and the extraction capability of important information is enhanced. The CA aggregates features from two spatial directions, embedding location information into the channel attention, and can obtain not only channel information but also direction and location information.

In summary, the present invention adopts a dense convolutional network as a main frame, and two attention models are integrated in the network, and first learning the importance degree of features by combining channel and position information, and then enhancing useful features according to the importance degree and suppressing features which are less important to the current task. For the proposed attention model, the coordinated attention model with the channel and the position information is arranged in front of the network to obtain more characteristic information, the excitation and extrusion attention model with only the channel information is arranged at the tail position of the network, after continuous down sampling of the network, the final output picture has smaller dimensionality and less position information amount, and the selection of the SE model is more favorable for improving the efficiency and the accuracy of classification and identification. In a word, the attention mechanism is added on the existing deep learning network framework, the feature weight is learned through the loss function, the interdependence relation between the feature position and the channel can be obtained, and the features are recalibrated. The attention module is flexible and light, can be easily embedded into the existing network model, and the attention embedding has little increase on the parameters and the calculation amount of the model and improves the performance of the network.

The present invention may be a system, method and/or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied therewith for causing a processor to implement various aspects of the present invention.

The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present invention may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + +, Python, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present invention are implemented by personalizing an electronic circuit, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA), with state information of computer-readable program instructions, which can execute the computer-readable program instructions.

Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, by software, and by a combination of software and hardware are equivalent.

Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. A method for automatically predicting Alzheimer's disease based on deep learning, comprising the following steps;

acquiring a target image to be detected;

2. The method of claim 1, wherein the deep learning model is a dense convolutional network model comprising a plurality of dense blocks.

3. The method of claim 1, wherein the attentional model extracts two separate location perceptions from vertical and horizontal directions for the input feature map, respectively, and encodes information feature maps with specific directions, respectively, and then weights the acquired spatial information on channels.

4. The method of claim 3, wherein the attentional model performs an average pooling and a three-dimensional convolution of the input feature maps from horizontal and vertical directions, respectively, and performs a fusion and a nonlinear process, and then obtains weights after excitation, respectively.

5. The method of claim 1, wherein the excitation and compression attention model comprises, in order, a global pooling layer, a first fully-connected layer, a non-linear processing layer, a second fully-connected layer, and an activation layer.

6. The method of claim 6, wherein the active layer is processed using a Sigmoid function.

7. The method of claim 1, wherein the target image is magnetic resonance imaging.

8. A system for automatically predicting alzheimer's disease based on deep learning, comprising:

9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.

10. A computer device comprising a memory and a processor, on which memory a computer program is stored which is executable on the processor, characterized in that the steps of the method of any of claims 1 to 7 are implemented when the processor executes the program.