CN114565572A

CN114565572A - Cerebral hemorrhage CT image classification method based on image sequence analysis

Info

Publication number: CN114565572A
Application number: CN202210162764.5A
Authority: CN
Inventors: 张道强; 祝小惟
Original assignee: Nanjing University of Aeronautics and Astronautics
Current assignee: Nanjing University of Aeronautics and Astronautics
Priority date: 2022-02-22
Filing date: 2022-02-22
Publication date: 2022-05-31

Abstract

The invention discloses a cerebral hemorrhage CT image classification method based on image sequence analysis. The method belongs to the technical field of medical image analysis, and comprises the steps of obtaining brain CT image data, carrying out imaging processing on the data, constructing a model and realizing a classification method in the operation process; the specific operation steps are as follows: obtaining a brain CT scanning result to obtain brain CT image original data; carrying out imaging processing on original data of the brain CT image to form a three-channel image sequence as the input of a subsequent model; constructing a model comprising a channel attention module and a space attention module to extract the characteristics of a three-channel image sequence; the model is trained to optimize parameters, positive cerebral hemorrhage is identified, cerebral hemorrhage subtypes are distinguished, classification of cerebral hemorrhage CT images can be achieved, CT scanning image information can be fully utilized by the model, attention to key information in an image sequence is improved, the judging process of the whole method accords with the diagnosis process of a doctor in a real scene, and the cerebral hemorrhage classification effect is improved.

Description

Cerebral hemorrhage CT image classification method based on image sequence analysis

Technical Field

The invention belongs to the technical field of medical image analysis, and relates to a cerebral hemorrhage CT image classification method based on image sequence analysis.

Background

With the continuous development of computer related technologies, artificial intelligence methods are increasingly applied to various fields, and medical image analysis related work based on machine learning and deep learning methods can improve the diagnosis efficiency of clinical related diseases and reduce the workload of doctors; in clinical diagnosis of cerebral hemorrhage, the medical image is usually based on the result of Computer Tomography (CT) image of brain of a patient, the medical image carries out tomography on the brain at preset intervals and obtains a plurality of image layers, the scanning time is fast, the image has enough definition, and the CT examination in clinic generally preferentially adopts a non-contrast enhanced flat scanning mode; the cerebral hemorrhage is usually sudden, and can generate a stronger image for a patient, the image has higher lethality rate, and the rapid diagnosis has great significance; meanwhile, the number of brain CT images to be examined is large, and therefore, in the whole important and repeated work, the radiologist responsible for the radiographing is required to have a considerable degree of professional knowledge, rich experience, a certain recognition speed and a certain work efficiency; artificial intelligence techniques are gradually playing a role in the intelligent identification of CT images and the rapid diagnosis of cerebral hemorrhage diseases, many of which have better performance, higher accuracy and faster diagnosis speed, and hopefully assist the diagnosis and treatment of doctors in practical application to achieve the purpose of rapidly screening cerebral hemorrhage and starting symptomatic treatment as soon as possible.

Currently, the work related to diagnosis of cerebral hemorrhage based on brain CT images can be mainly divided into two categories: 1) identifying and segmenting details and forms of a cerebral hemorrhage area of a patient, 2) screening and subtype analysis of cerebral hemorrhage for a subject to be diagnosed, judging whether cerebral hemorrhage occurs in an image result mainly through a related classification method, and judging a specific hemorrhage type according to the state and characteristics of the bleeding; in the cerebral hemorrhage recognition and segmentation work under the artificial intelligence technology, the training and verification of the model generally need the marking of image level and even pixel level, which is often difficult to obtain, on one hand, because the workload is large; on the other hand, the procedure itself is cumbersome and rigid, requiring a physician with a certain experience to perform the marking task, which consumes a lot of time and effort; meanwhile, the cerebral hemorrhage recognition and segmentation model can enable doctors to quickly obtain hemorrhage position information, but the judgment of the hemorrhage type is not visual enough, so that a model specially used for directly diagnosing the hemorrhage condition and the hemorrhage type is often needed; for this purpose, the classification model is more able to meet this requirement in contrast; in the task of classifying cerebral hemorrhage, the specific purpose of the task is divided into two classes of bleeding and two classes of bleeding subtypes, furthermore, on the setting of a label, part of the work is to mark each scanning layer image of a single scanning object, and one part of the work is to mark one scanning object, so that after the scanning images are converted into image representation suitable for artificial intelligence technology, the former trains a model by using a single image and generates a diagnosis result for an independent image; the method considers a plurality of scanning surfaces of the whole scanning, utilizes the image sequence to train the model, can obtain diagnosis aiming at the whole scanning, avoids discarding the context image information in a scanning sample by fusing the image characteristics in the sequence, and simultaneously is more consistent with the diagnosis mode of clinical experts; in addition, when the DICOM standard format of CT scanning medical images is converted into common image representation, most of the conventional methods fix a specific window level window width for displaying the whole brain state, and the single fixed value has no determined standard and simultaneously loses information in other windows; some methods obtain image display at this time by setting a plurality of windows, and perform feature fusion in a model, but actually, the importance levels of the windows are different, and attention needs to be paid to the windows in different levels; in summary, in order to improve clinical diagnosis efficiency and reduce the workload of doctors, it is desirable to provide direct assistance for diagnosis of cerebral hemorrhage by an artificial intelligence auxiliary diagnosis method, and provide a cerebral hemorrhage CT image classification method based on image sequence analysis by using more comprehensive and more appropriate scan intralayer information and interlaminar information.

Disclosure of Invention

The purpose of the invention is as follows: the invention aims to solve the problems in the related work of the existing cerebral hemorrhage diagnosis based on the cerebral CT image, which comprises the following steps: the fine marks in the identification and segmentation tasks are complex to obtain, the context relation contained in the whole sample scanning sequence is ignored aiming at the diagnosis method of a single scanning layer, information is easy to lose when the window level window width is set in the imaging display process of DICOM data, special attention to important features is lacked in a model method, and the like; the method for classifying the cerebral hemorrhage CT images based on image sequence analysis has significance in assisting clinical diagnosis of cerebral hemorrhage diseases through artificial intelligence; specifically, the present invention solves the above problems to some extent; the method comprises the steps of improving the utilization degree of image information through a multi-window and attention mode of a whole scanning layer sequence, introducing a channel attention and space attention convolution short-time memory (CLSTM) network to extract and pay attention to the features in a multi-channel image sequence in a distinguishing mode, training an end-to-end cerebral hemorrhage classification model based on reasonable global feature representation extracted from a scanning sample, intelligently diagnosing the cerebral hemorrhage occurrence condition and distinguishing cerebral hemorrhage subtypes, and providing help for screening and detecting of cerebral hemorrhage diseases clinically.

The technical scheme is as follows: the invention relates to a cerebral hemorrhage CT image classification method based on image sequence analysis, which comprises the steps of classifying cerebral hemorrhage CT images; the method specifically comprises the following steps:

determining whether cerebral hemorrhage occurs or not from the cerebral CT scanning, namely judging whether the cerebral CT scanning result is cerebral hemorrhage positive or cerebral hemorrhage negative; and (3) identifying cerebral hemorrhage subtypes according to the cerebral hemorrhage positive sample, namely judging whether the cerebral hemorrhage displayed in the brain CT scanning result is intracerebral hemorrhage, brain parenchyma hemorrhage, subarachnoid hemorrhage, epidural hemorrhage or subdural hemorrhage.

Further, the operation process of the cerebral hemorrhage CT image classification method based on image sequence analysis comprises the steps of obtaining cerebral CT image data, carrying out imaging processing on the data, constructing a model and realizing the classification method;

specifically, the method comprises the following steps: obtaining a brain CT scanning result to obtain brain CT image original data; carrying out imaging processing on original data of the brain CT image to form a three-channel image sequence as the input of a subsequent model; constructing a model comprising a channel attention module and a space attention module to extract the characteristics of a three-channel image sequence; and training model optimization parameters, identifying cerebral hemorrhage positive and distinguishing cerebral hemorrhage subtypes, and realizing classification of cerebral hemorrhage CT images.

Further, the operation steps are as follows:

(1) and processing brain CT image data:

(1.1) obtaining a brain CT scanning result, storing the obtained brain CT scanning result in a standard mode of a DICOM medical image format, and taking the stored brain CT scanning result as original data of a brain CT image;

(1.2) determining the coarse-grained type labels of all samples in the stored original data of the brain CT image,

marking original data of the CT brain image as positive brain hemorrhage and negative brain hemorrhage, namely ICH positive and ICH negative, according to whether the image shows high-density images representing the occurrence of the brain hemorrhage;

(1.3) determining fine-grained class labels of all samples according to the marked ICH positive brain CT image original data;

(1.4) carrying out imaging processing on all the brain CT image original data in the DICOM format, and extracting each sample in the brain CT image original data into a three-channel image sequence;

(2) and constructing a model:

(2.1) weighting the features extracted from the three-channel image sequence by using a channel attention module, and fusing the weighted features;

(2.2) generating spatial region attention for the fused features by using a CLSTM network with a spatial attention module, extracting space-time features of the three-channel image sequence and classifying the features;

(3) and the classification method is realized as follows:

and taking a three-channel image sequence obtained after the imaging processing as the input of the constructed model, and then training parameters in the model by utilizing the determined coarse-grained and fine-grained class labels to obtain the model finally used for classifying the cerebral hemorrhage CT images.

Further, in step (1.3), the determining the fine-grained class label of each sample for the ICH positive data specifically includes: according to the position and shape of the high-density shadow shown in the image, the primary data of the brain CT image with positive ICH is marked as five subtypes of cerebral hemorrhage, including: intraventricular hemorrhage, parenchymal cerebral hemorrhage, subarachnoid hemorrhage, epidural hemorrhage, and subdural hemorrhage, i.e., IVH class, IPH class, SAH class, EDH class, and SDH class.

Further, in step (1.4), the step of performing the imaging processing on the original data of the brain CT images in all DICOM formats specifically includes: setting different window levels and window widths, respectively selecting a whole brain window, a blood window and a skull window, and converting information in a corresponding HU value range into a common image for representation, so that each sample in the original data of the brain CT image is extracted into a three-channel image sequence, wherein the channel obtained after processing is in an image PNG format, and the sequence length is the number of tomography layers in the original data of the sample brain CT image.

Has the advantages that: compared with the prior art, the invention has the characteristics that: the method of the invention splits CT image information and pays attention to the information in a distinguishing way, and improves the diagnosis capability of the model by obtaining high-level characteristic representation of channel key information, space key information and time axis context information. Comprehensively simulating the mode of a doctor when viewing the image result on the whole, comprehensively paying attention to each scanning layer while more considering key CT window channels and key areas in the image, capturing spatial dimension information contained in medical image data and context relation of each scanning on a time axis, and finally obtaining a reasonable diagnosis result. In practical application, after a training set is divided, according to label information, corresponding models are respectively trained according to the existence of bleeding and whether the bleeding can be diagnosed as five subtypes, and the models are used in respective diagnosis tasks. Specifically, for a sample to be diagnosed, a coarse-grained diagnosis model with or without bleeding is used to obtain a result of whether cerebral hemorrhage exists in an image, if so, the five fine-grained models are used to perform cerebral hemorrhage subtype subdivision diagnosis, and the output results of the models are integrated to obtain a final cerebral CT image classification result for cerebral hemorrhage diseases.

Drawings

FIG. 1 is a flow chart of the operation of the present invention;

FIG. 2 is a schematic diagram illustrating the operation of the imaging process of the CT brain image data according to the present invention;

FIG. 3 is a schematic diagram of an operation of extracting three-channel image sequence features by using a channel attention module in model construction according to the present invention;

FIG. 4 is a schematic diagram of the operation of extracting features to complete classification by using a spatial attention module in the model construction of the present invention.

Detailed Description

The invention is further described with reference to the following figures and examples.

As shown in the figure, according to the method for classifying the cerebral hemorrhage CT image based on the image sequence analysis, the cerebral hemorrhage CT image is firstly classified; the method specifically comprises the following steps:

Specifically, in the brain CT image data processing part, the obtained brain CT scan is stored as the brain CT imaging data based on the DICOM medical image format standard, and a category label is determined for each object sample; classifying the occurrence of cerebral hemorrhage into cerebral hemorrhage positive and cerebral hemorrhage negative according to whether the cerebral CT image data can observe the cerebral hemorrhage according to the coarse particle category, namely ICH (intracerebral hemorrhage) positive and ICH negative;

whereas cerebral hemorrhage comprises five specific subtypes: intraventricular hemorrhage, cerebral parenchymal hemorrhage, subarachnoid hemorrhage, epidural hemorrhage, and subdural hemorrhage;

ICH positive samples were labeled IVH (intravenous hemorrhage) class, IPH (Intra pathological hemorrhage) class, SAH (Subarachroned hemorrhage) class, EDH (anterior hemohemomatoma) class, and SDH (SDH hemotoma) class, respectively, according to the fine-grained bleeding subtype;

then, in order to enable the data to be identified by a model method, carrying out imaging processing on the brain CT medical imaging data in the DICOM format according to a clinical picture reading mode;

in the medical field, the absorption degree of different tissues to X-rays is different due to different densities of the tissues, and the degree is measured by Hounsfield Unit (HU), and a pair of Window Level (WL) and Window Width (WW) is set, so that a Window is defined, and the lower bound W of the Window_minAnd an upper bound W_maxRespectively as follows:

tissues and lesions with HU values above this range will appear as white shadows in the gray-scale image, while parts below this range will appear as black shadows; the HU value (HU value) in the window range corresponds to a Gray level (Gray level) in a range of 0 to 255, and the closer to 255, the brighter the HU value (HU value) is, the corresponding manner is:

wherein I is an indicator function;

in this way, the organization structure in the range can be distinguished through different gray scales, and a gray scale image corresponding to the window can be obtained; respectively selecting a whole brain window (WL: 50, WW: 100), a hematoma window (WL: 70, WW: 50) and a skull window (WL: 500, WW: 3000), converting information in the corresponding HU value range into a common image representation, as shown in fig. 2, forming a three-channel medical image for single sample data, wherein the length L of the image sequence is equal to the length L of the three-channel medical image, and the three-channel medical image is a three-channel image represented by a three-channel image representation_iEqual to the number of tomographic layers contained in a CT scan sample of the object;

on the other hand, the samples may have different distances between tomography, and in order to ensure that the diagnosis task of the model is not interfered as much as possible, a resampling technology is adopted to fix the distance on the z axis, and all data are sampled to be consistent standard intervals; for the sampled image sequences with different lengths, because the framework of the network is fixed, the sequence needs to be cut to a fixed length, namely N windows with the length of L are randomly intercepted on three channels of the sequence at the same time, N L-length subsequences are obtained on each sample, the labels of the subsequences are consistent with the source sequence, and finally N L-length three-channel image sequence samples are obtained from all diagnostic objects and are used for training a channel and space dual-attention CLSTM network model;

in the construction of the model, the composition of the main body part of the model mainly comprises: 1) the channel attention module is utilized to give corresponding weight to the features extracted from the processed three-channel image sequence, the features are fused based on different attention degrees, and the specific structure of the feature is shown in FIG. 3;

2) generating space region attention for the fused feature map by using a CLSTM network with a space attention module, and extracting space-time features of the sequence for classification, wherein the specific structure of the sequence is shown in FIG. 4;

wherein, after the data is preprocessed, N sub-sequences with labels are obtained in total and are marked as

X_nRepresents a subsequence;

wherein N belongs to {1,2, …, N }; inputting a fixed-length subsequence as an image sequence of the model, wherein the length of each subsequence is L and is marked as { X_n:X_n,1,X_n,2,…,X_n,LN-1, 2, …, N, wherein each image of the sub-sequence consists of three channel images taken under the whole brain, haematoma and skull windows,

thus, a single image contained in a subsequence can be represented in the form of three channels:

firstly, each channel of each image in the sequence is input into convolution with Batch Normalization (BN) and Relu activation, wherein the convolution kernel size of the convolution is 2 multiplied by 2, the step size is 2, and feature maps of three channels are obtained respectively

Then, the feature map of a single scan plane is given weights in channel dimensions using channel attention

The specific implementation modules comprise a Global Max Pooling (GMP) and a Global Average Pooling (GAP) which are parallel, a Multi-layer perceptron (MLP) and a sigmoid activation function; after passing through GMP and GAP, the input feature graph is respectively compressed into two one-dimensional vectors in the spatial dimension, and the two vectors are input and activated by ReluThe multilayer perceptron fuses the obtained results;

finally, a channel weight vector is obtained after sigmoid activation, the vector length is equal to the number of input channels, and the whole process is represented as follows:

CW_n,t＝CAtt(f_n,t)

＝Sigmoid(MLP(GMP(f_n,t))+MLP(GAP(f_n,t)))

the attention weight of each channel is superposed on the respective characteristics, and the obtained weighted characteristic graphs are superposed pixel by pixel to obtain the characteristic representation of the current scanning surface fusing three channels:

will be composed of L F_n,tInputting the formed characteristic image sequence into a CLSTM network capable of capturing space-time characteristics; the network comprises a CLSTM layer with spatial attention, a common CLSTM layer, a global average pooling layer, a Full Connected (FC) layer and a sigmoid activation function;

in particular, in the CLSTM layer, for a subsequence sample X_nInputting the current two-dimensional feature map F into the network at the t time step_n,tHidden states obtained by CLSTM computing the first layer

The calculation method is as follows:

in the formula (I), the compound is shown in the specification,

the input gate is shown to be one of,

the forgetting of the door is indicated,

an output gate is shown which is shown,

indicating the new memory cell in the memory cell array,

indicating a finally updated memory cell, indicating a Hadamard product (Hadamard product) operation between matrices, Conv indicating a convolution operation;

wherein, the first and the second end of the pipe are connected with each other,

representing the weight parameters to be trained in the convolution operation of the respective gate, b_*Indicating an offset within the door; CLSTM layer integrated with space attention mechanism and based on initial input F on time step_n,tAnd current step hidden state obtained after passing through common CLSTM layer

To obtain a spaceThe weight of (1); f is to be_n,tAnd

merging the features into a two-channel feature by splicing in series, carrying out convolution operation on the two-channel feature by utilizing a hidden layer with a convolution kernel with the size of 1 multiplied by 2 and the step length of 1, and obtaining a spatial attention weight SW corresponding to the feature map after sigmoid activation_n,tThe calculation process is expressed as:

in the formula (I), the compound is shown in the specification,

representing the concatenation of two features in a dimension,

and b is the parameter to be trained in the convolution, and the attention weight SW is obtained_n,tAnd F_n,tHaving the same size, SW_n,tAnd F_n,tOverlapping with Hadamard product to obtain the feature with space distinguishing attention

The hidden state is calculated in the general CLSTM layer input to the next layer

Sequentially inputting a two-dimensional feature map containing channel attention in the sequence according to time steps, and obtaining the last time step F by modeling attention weight and space-time features through a model_n,LHidden state of

Wherein the overall information of the image sequence is contained; thus, as high-level information extracted throughout the CT scan sequence, the

And performing GAP operation, inputting the result into an FC layer, and obtaining the classification result of each sequence through a sigmoid activation function, wherein the classification result is represented as the probability of belonging to a class.

The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above-mentioned embodiments, and all technical solutions belonging to the idea of the present invention belong to the protection scope of the present invention. It should be noted that modifications and embellishments within the scope of the invention may be made by those skilled in the art without departing from the principle of the invention.

Claims

1. A cerebral hemorrhage CT image classification method based on image sequence analysis is characterized in that cerebral hemorrhage CT images are classified firstly; the method specifically comprises the following steps:

2. The method for classifying the cerebral hemorrhage CT image based on the image sequence analysis according to claim 1, wherein the operation process comprises the acquisition of the brain CT image data, the imaging processing of the data, the construction of the model and the implementation of the classification method;

3. The method for classifying the cerebral hemorrhage CT image based on the image sequence analysis as claimed in claim 2,

the operation steps are as follows:

(1) and processing brain CT image data:

(2) and constructing a model:

(3) and the classification method is realized as follows:

4. The method for classifying the cerebral hemorrhage CT image based on the image sequence analysis as claimed in claim 3,

in step (1.3), the determining the fine-grained class label of each sample for the ICH positive data specifically includes: according to the position and shape of the high-density shadow shown in the image, the primary data of the brain CT image with positive ICH is marked as five subtypes of cerebral hemorrhage, including: intraventricular hemorrhage, parenchymal cerebral hemorrhage, subarachnoid hemorrhage, epidural hemorrhage, and subdural hemorrhage, i.e., IVH class, IPH class, SAH class, EDH class, and SDH class.

5. The method for classifying the cerebral hemorrhage CT image based on the image sequence analysis as claimed in claim 3,

in the step (1.4), the step of performing the imaging processing on the original brain CT image data in all DICOM formats specifically includes: setting different window levels and window widths, respectively selecting a whole brain window, a blood window and a skull window, and converting information in a corresponding HU value range into a common image for representation, so that each sample in the original data of the brain CT image is extracted into a three-channel image sequence, wherein the channel obtained after processing is in an image PNG format, and the sequence length is the number of tomography layers in the original data of the sample brain CT image.