CN113850796A

CN113850796A - Lung disease identification method and device based on CT data, medium and electronic equipment

Info

Publication number: CN113850796A
Application number: CN202111187917.3A
Authority: CN
Inventors: 王雄; 裴璇
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2021-10-12
Filing date: 2021-10-12
Publication date: 2021-12-28

Abstract

The disclosure provides a lung disease identification method and device based on CT data, a computer readable medium and electronic equipment, and relates to the technical field of image processing. The method comprises the following steps: acquiring a 3D tensor corresponding to lung CT data to be identified, and partitioning the 3D tensor to obtain k partitioned tensors; carrying out lung disease classification and identification on the k block tensors to obtain k block classification results; and outputting an identification result corresponding to the lung CT data to be identified according to the k block classification results, and determining a focus block tensor corresponding to the lung CT data to be identified in the k block tensors when the identification result is an abnormal result. According to the method, the 3D tensor is partitioned, so that the 3D tensor can be prevented from being sampled, and further the loss of key focus information or the introduction of repeated invalid noise can be avoided; meanwhile, the focus blocking tensor can be determined in the k blocking tensors directly according to the k blocking classification results, and further the focus is roughly positioned on the basis of the focus blocking tensor.

Description

Lung disease identification method and device based on CT data, medium and electronic equipment

Technical Field

The present disclosure relates to the field of image processing technologies, and in particular, to a lung disease recognition method based on CT data, a lung disease recognition apparatus based on CT data, a computer-readable medium, and an electronic device.

Background

Ct (computed tomography), i.e. computed tomography, the imaging principle is: the layer surface with a certain thickness of the specific part of the human body is scanned by X-ray beams, gamma rays, ultrasonic waves and the like, and a medical image is obtained after the layer surface is processed by a computer. Compared with the conventional imaging examination means, the CT has the advantages of capability of acquiring a real sectional image, high density resolution, quantitative analysis, convenience for subsequent image processing and the like, so that the CT is more and more widely applied to medical image detection.

In recent years, the rapid development of computer-aided diagnosis techniques has greatly facilitated the diagnostic analysis of medical CT images. For pulmonary CT imaging, there are generally three methods of aided analysis: firstly, 3D data of lung CT are sampled to a 3D tensor with a fixed size in an interpolation or sampling mode, and then the 3D tensor with the fixed size is input into a 3D convolution model for diagnosis; secondly, a feature classifier based on a preorder segmentation task realizes diagnosis, and the type, the type and the quantity of features needing to be manually designed and extracted, such as color, texture, shape features and the like of a focus, are input into the classifier to realize diagnosis; thirdly, diagnosis is realized by the feature classifier based on the fuzzy mode, a feature space can be constructed by manually designing or automatically extracting features through a convolutional neural network, and dimension reduction and weighting of the features are carried out on the feature space based on the fuzzy mode.

However, the first of the three auxiliary analysis methods involves sampling of a fixed-size 3D tensor, and due to the lack of prior information, the loss of critical focus information or the introduction of repeated invalid noise is easily caused; the second approach involves providing an accurate lesion area label when training the feature classifier, so labeling is time consuming and labor intensive; the third mode needs to manually design a feature space or a feature space screening strategy, has high dependency on the feature quality of an input end, and is easy to cause model overfitting.

Disclosure of Invention

The present disclosure is directed to a method for identifying a lung disease based on CT data, a device for identifying a lung disease based on CT data, a computer readable medium and an electronic device, which can provide spatial localization information of a lesion by determining a segmentation tensor of the lesion on the basis of identifying a lung disease based on complete original CT data.

According to a first aspect of the present disclosure, there is provided a method for lung disease identification based on CT data, comprising: acquiring a 3D tensor corresponding to lung CT data to be identified, and partitioning the 3D tensor to obtain k partitioned tensors; k is a positive integer greater than or equal to 2; carrying out lung disease classification and identification on the k block tensors to obtain k block classification results; and outputting an identification result corresponding to the lung CT data to be identified according to the k block classification results, and determining a focus block tensor corresponding to the lung CT data to be identified in the k block tensors when the identification result is an abnormal result.

According to a second aspect of the present disclosure, there is provided a lung disease identification apparatus based on CT data, comprising: the blocking acquisition module is used for acquiring a 3D tensor corresponding to the lung CT data to be identified and blocking the 3D tensor to obtain k blocking tensors; k is a positive integer greater than or equal to 2; the classification identification module is used for performing lung disease classification identification on the k block tensors to obtain k block classification results; and the result output module is used for outputting an identification result corresponding to the lung CT data to be identified according to the k block classification results, and determining a focus block tensor corresponding to the lung CT data to be identified in the k block tensors when the identification result is an abnormal result.

According to a third aspect of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, is adapted to carry out the above-mentioned method.

According to a fourth aspect of the present disclosure, there is provided an electronic apparatus, comprising: a processor; and memory storing one or more programs that, when executed by the one or more processors, cause the one or more processors to implement the above-described method.

According to the lung disease identification method based on CT data provided by the embodiment of the disclosure, the 3D tensor corresponding to the CT data of the lung to be identified is obtained, k blocking tensors can be obtained by blocking the 3D tensor, then the classification identification of the lung disease is respectively carried out on the k blocking tensors to obtain k blocking classification results, the identification result corresponding to the CT data of the lung to be identified can be output according to the k blocking classification results, and the focus blocking tensor is determined in the k blocking tensors. By partitioning the 3D tensor, the 3D tensor can be prevented from being sampled, and further the loss of key focus information or the introduction of repeated invalid noise can be avoided; meanwhile, the focus blocking tensor can be determined in the k blocking tensors directly according to the k blocking classification results, and further the focus is roughly positioned on the basis of the focus blocking tensor.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty. In the drawings:

FIG. 1 illustrates a schematic diagram of an exemplary system architecture to which embodiments of the present disclosure may be applied;

FIG. 2 shows a schematic diagram of an electronic device to which embodiments of the present disclosure may be applied;

FIG. 3 schematically illustrates a flow chart of a method for lung disease identification based on CT data in an exemplary embodiment of the present disclosure;

figure 4 schematically illustrates a flow chart for determining recognition results and focal segmentation tensors in an exemplary embodiment of the present disclosure;

FIG. 5 is a schematic diagram schematically illustrating a training process of a lung disease recognition neural network in an exemplary embodiment of the present disclosure;

FIG. 6 schematically illustrates another flow chart for determining recognition results and focal segmentation tensors in exemplary embodiments of the present disclosure;

fig. 7 schematically shows a composition diagram of a lung disease identification device based on CT data in an exemplary embodiment of the disclosure.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

Fig. 1 is a schematic diagram illustrating a system architecture of an exemplary application environment to which a method and apparatus for identifying a lung disease based on CT data according to an embodiment of the present disclosure may be applied.

As shown in fig. 1, the system architecture 100 may include one or more of

terminal devices

101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the

terminal devices

101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few. The

terminal devices

101, 102, 103 may be various electronic devices having an image recognition function, including but not limited to desktop computers, portable computers, smart phones, tablet computers, and the like. It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation. For example, server 105 may be a server cluster comprised of multiple servers, or the like.

The lung disease identification method based on the CT data provided by the embodiment of the present disclosure can be executed by the

terminal devices

101, 102, and 103, and accordingly, the lung disease identification apparatus based on the CT data is generally disposed in the

terminal devices

101, 102, and 103; the lung disease recognition may be performed by the server 105, and accordingly, a lung disease recognition apparatus based on CT data may be disposed in the server 105, which is not particularly limited in the exemplary embodiment. For example, in an exemplary embodiment, the terminal device 101 may be a device capable of directly acquiring a 3D tensor corresponding to lung CT data, for example, a tomography device, and the other

terminal devices

102 and 103 and the server 105 may acquire a 3D tensor corresponding to lung CT data acquired by the terminal device 101 through a network, so as to perform lung disease identification.

An exemplary embodiment of the present disclosure provides an electronic device for implementing a lung disease identification method based on CT data, which may be the

terminal device

101, 102, 103 or the server 105 in fig. 1. The electronic device comprises at least a processor and a memory for storing executable instructions of the processor, the processor being configured to perform the method for lung disease identification based on CT data via execution of the executable instructions.

The following takes the mobile terminal 200 in fig. 2 as an example, and exemplifies the configuration of the electronic device. It will be appreciated by those skilled in the art that the configuration of figure 2 can also be applied to fixed type devices, in addition to components specifically intended for mobile purposes. In other embodiments, mobile terminal 200 may include more or fewer components than shown, or some components may be combined, some components may be split, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware. The interfacing relationship between the components is only schematically illustrated and does not constitute a structural limitation of the mobile terminal 200. In other embodiments, the mobile terminal 200 may also interface differently than shown in fig. 2, or a combination of multiple interfaces.

As shown in fig. 2, the mobile terminal 200 may specifically include: a processor 210, an internal memory 221, an external memory interface 222, a Universal Serial Bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 271, a microphone 272, a microphone 273, an earphone interface 274, a sensor module 280, a display 290, a camera module 291, an indicator 292, a motor 293, a button 294, and a Subscriber Identity Module (SIM) card interface 295. Wherein the sensor module 280 may include a depth sensor 2801, a pressure sensor 2802, a gyroscope sensor 2803, and the like.

Processor 210 may include one or more processing units, such as: the Processor 210 may include an Application Processor (AP), a modem Processor, a Graphics Processing Unit (GPU), an Image Signal Processor (ISP), a controller, a video codec, a Digital Signal Processor (DSP), a baseband Processor, and/or a Neural-Network Processing Unit (NPU), and the like. The different processing units may be separate devices or may be integrated into one or more processors.

The NPU is a Neural-Network (NN) computing processor, which processes input information quickly by using a biological Neural Network structure, for example, by using a transfer mode between neurons of a human brain, and can also learn by itself continuously. The NPU can implement applications such as intelligent recognition of the mobile terminal 200, for example: image recognition, face recognition, speech recognition, text understanding, and the like.

In some embodiments, the step of performing lung disease classification and identification on the k blocking tensors to obtain k blocking classification results, the step of outputting identification results corresponding to-be-identified lung CT data according to the k blocking classification results, the step of determining a focus blocking tensor corresponding to-be-identified lung CT data in the k blocking tensors, and the step of adjusting and determining parameters in the lung disease identification neural network may be implemented by an NPU.

A memory is provided in the processor 210. The memory may store instructions for implementing six modular functions: detection instructions, connection instructions, information management instructions, analysis instructions, data transmission instructions, and notification instructions, and execution is controlled by processor 210.

The mobile terminal 200 implements a display function through the GPU, the display screen 290, the application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display screen 290 and an application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 210 may include one or more GPUs that execute program instructions to generate or alter display information. In some embodiments, the partitioning of the 3D tensor, etc. steps may be implemented by the GPU.

In the related art, three auxiliary analysis methods are generally included for lung CT images.

Firstly, 3D data of lung CT is sampled to a 3D tensor with a fixed size in an interpolation or sampling mode, and then the 3D tensor with the fixed size is input into a 3D convolution model for diagnosis. For example, in an article written by Wang X, Jiang L, Li L, etc., and named Joint learning of 3D learning segmentation and classification for extensible covi-19 diagnosis, a 3D convolutional neural network with 3 branches is proposed, which extracts the underlying fusion features at multiple scales by scaling and interpolating down-sampling the CT data and using the 3D convolutional neural network 1; then, processing partial characteristics output by the branch 1 by using a 3D convolutional neural network branch 2, applying the partial characteristics to a focus segmentation task, and outputting a focus segmentation result; and finally, receiving partial characteristics output by the branch 1 and segmentation results output by the branch 2 by using the 3D convolutional neural network branch 3, and outputting final recognition results of the new coronary lung diseases and community acquired lung diseases.

However, this method needs to sample the lung CT data into a fixed-size 3D tensor by interpolation or sampling at the input end, and then input the 3D tensor into the 3D convolution model, which is very easy to cause the missing of the key lesion information or the introduction of repeated invalid noise due to the lack of prior information.

Secondly, the diagnosis is realized by a feature classifier based on a preorder segmentation task, and the type, the type and the quantity of features which need to be manually designed and extracted, such as the color, the texture, the shape features and the like of the focus, are input into the classifier to realize the diagnosis. For example, patent application publication No. CN111724356A provides an image processing method and system for identifying lung diseases in CT images, in which the image processing method first extracts a lung disease region mask based on a lung disease segmentation model, and then performs weight adjustment on convolution features at a spatial level by introducing an attention mechanism, so as to make the lung disease region features more prominent; the three-dimensional CT sequence is zoomed according to the cross section, each positive layer (layer with lung diseases) of the sequence is classified layer by layer, then the lung disease classification result of the three-dimensional CT sequence is obtained by weighting the area of the corresponding lung disease mask area and the classification result and voting the classification with the maximum probability.

However, the feature classifier based on the preamble segmentation task adopted in this method needs to provide an accurate lesion region label when training the model. Because labeling of the labels needs to be carried out manually, the labeling is time-consuming and labor-consuming, the labeling quality is poor due to different understandings of labeling personnel to professional knowledge, the corresponding training sample size is small, and the generalization ability of the trained model needs to be verified.

Thirdly, diagnosis is realized by the feature classifier based on the fuzzy mode, a feature space can be constructed by manually designing or automatically extracting features through a convolutional neural network, and dimension reduction and weighting of the features are carried out on the feature space based on the fuzzy mode. For example, patent application publication No. CN111414956A provides a multi-example learning identification method for fuzzy patterns in lung CT images, which utilizes a pre-trained convolutional neural network to perform feature extraction on an acquired CT image, treats a single CT image as a packet, takes the features extracted on the single CT image as an example, and performs classification tasks using multi-example learning models (including k-nearest neighbor model rotation-KNN, multi-example support vector machine model MI-SVM, and expectation maximization-diversity density model EM-DD).

However, the method mainly uses artificial design or feature construction feature space automatically extracted by a convolutional neural network or a feature space screening strategy, and has high dependency on the feature quality of an input end, which easily causes model overfitting.

In view of one or more of the above problems, the present example embodiment provides a lung disease identification method based on CT data. The lung disease identification method based on CT data may be applied to the server 105, and may also be applied to one or more of the

terminal devices

101, 102, and 103, which is not particularly limited in this exemplary embodiment. Referring to fig. 3, the method for identifying a lung disease based on CT data may include the following steps S310 to S320:

in step S310, a 3D tensor corresponding to the CT data of the lung to be identified is obtained, and the 3D tensor is segmented to obtain k segmented tensors.

Wherein k is a positive integer greater than or equal to 2; the lung CT data to be identified can comprise lung CT data with different formats, such as DICOM format, NII format and single-channel image sequence mode; correspondingly, the 3D tensor may also include a 3D tensor obtained by converting CT data of different formats of lungs.

In an exemplary embodiment, the 3D tensor can be preprocessed before subsequent processing of the 3D tensor. Specifically, the preprocessing process may include a normalization process, a scaling process, a denoising process, and the like.

Wherein the contents stored in the CT data are different due to different formats. For example, the DICOM, NII format stores HU values and the single-channel image sequence stores pixel gray values. The 3D tensor needs to be normalized to a certain range, e.g., [0, 1 ]. It should be noted that the normalization process is to process numerical values, so the normalization process can also be directly performed on the lung CT data to be identified, and then the lung CT data to be identified is converted into a 3D tensor.

Before normalization processing, windowing processing may be performed on the 3D tensor, that is, HU values or pixel gray values outside the window range are screened out, and HU values or pixel gray values within the window range are retained. By the processing mode, some extra-large or extra-small HU values or pixel gray values can be screened out, and the problem that data after normalization processing is excessively gathered and cannot be identified due to the existence of extra-large values or extra-small values is solved. For example, assuming that the pixel gray scale is distributed between 0-100, if normalized to the range of [0, 1], it is only necessary to divide each value by 100; assuming that the gray levels of pixels are mostly distributed between 0 and 100, but there is a maximum value of 255, it is necessary to divide each value by 255, which results in that the gray levels of pixels distributed between 0 and 100 are compressed between 0 and 0.39, while the gray levels of pixels distributed between 0.39 and 1 have only 1 pixel value, and there is a high possibility that the gray levels are too concentrated and the difference between the gray levels is small, thereby causing an unrecognizable problem.

The 3D tensor obtained by converting the CT data of the lung to be identified acquired by the CT apparatus may be of different scales, for example, different resolutions. Therefore, the scaling process may be performed on the basis of the 3D tensor, and the 3D tensors of different resolutions are subjected to nearest neighbor interpolation on the cross-sectional slice and scaled to a predetermined size, for example, H (image height) × W (image width). It should be noted that in scaling, the image data contained in the 3D tensor need not be scaled to avoid losing data at certain locations of the lungs.

Wherein, noise may be introduced in the acquisition process, so that the noise that may exist in the 3D tensor can be removed through the denoising process.

In order to convert the 3D tensor into a form more suitable for analysis processing by a computer device such as a GPU, contrast enhancement may be performed on the 3D tensor. For example, contrast enhancement based on histogram equalization.

In an exemplary embodiment, when the 3D tensor is partitioned, the partitioning may be performed according to different partitioning rules. When the size of the 3D tensor is the number of images × the image height × the image width, the 3D tensor may be sequentially partitioned based on preset partitioning parameters to obtain k-1 partitioning tensors whose size is the partitioning parameter × the image height × the image width, and then the remaining part of the remaining non-partitioned tensors may be used as 1 partitioning tensor to obtain k tensors. In addition, when the blocking is performed, the size of the remaining portion is the remaining number × the image height × the image width, and the remaining number is less than or equal to the blocking parameter.

For example, assume that the size of a certain 3D tensor is D × H × W, and the preset blocking parameter is N. If D/N can be divided evenly, D/N is k, k block tensors with the size of N × H × W are obtained, that is, the remaining number is equal to the block parameter; if D/N can not be divided evenly, the D/N is subjected to upward rounding calculation and then is equal to k, and k-1 block tensors with the size of N multiplied by H multiplied by W and 1 block tensor with the size of (D- (k-1) N multiplied by H multiplied by W are obtained. Wherein (D- (k-1) N) is the remaining number, and the remaining number is less than the blocking parameter.

In step S320, the lung disease classification and identification are performed on the k blocking tensors to obtain k blocking classification results.

In an exemplary embodiment, after k block tensors are obtained, the pulmonary disease classification and identification may be performed on each block tensor, and k block classification results are obtained correspondingly. Specifically, when the lung diseases are classified and identified, the classification and the identification can be performed by adopting modes such as machine learning and deep learning. For example, classification and identification of lung diseases can be performed on k block tensors through a trained 3D convolutional neural network (such as 3D ResNet-18 Model, 3D DenseNet, etc.).

In step S330, an identification result corresponding to the lung CT data to be identified is output according to the k block classification results, and a focus block tensor corresponding to the lung CT data to be identified is determined in the k block tensors when the identification result is an abnormal result.

The identification result corresponding to the lung CT data to be identified can have a plurality of preset types. For example, for pneumonia identification, the identification result corresponding to a certain lung CT data to be identified may include three types: normal state, common pneumonia, new coronary pneumonia. Correspondingly, the preset types can be set as normal pneumonia, common pneumonia and new coronary pneumonia. On the basis, each block classification result comprises the probability that the block tensor belongs to each preset category. For example, after a certain block is identified, the obtained identification result may include: the probability of belonging to the normal state is 80%, the probability of belonging to the common pneumonia is 11%, and the probability of belonging to the new coronary pneumonia is 9%.

The abnormal result means that the identification result corresponding to the lung CT data to be identified is a result containing lung diseases. For example, for pneumonia, the identification results include normal pneumonia, common pneumonia and new coronary pneumonia, wherein the common pneumonia and the new coronary pneumonia are both abnormal results.

In an exemplary embodiment, after k block classification results are obtained, the identification result and the focus block tensor corresponding to the lung CT data to be identified may be determined based on the k block classification results. Specifically, the method can be performed by machine learning, deep learning, and the like. For example, the k segmented classification results can be integrated through a bayesian noise-Or model to output the identification result corresponding to the lung CT data to be identified and the focus segmented tensor.

In an exemplary embodiment, when the recognition result includes a plurality of preset categories, and the block classification result includes probabilities of block tensors belonging to the preset categories, when the recognition result corresponding to the CT data of the lung to be recognized is output according to k block classification results, for each preset category, a total probability corresponding to the preset category may be calculated based on the k block classification results, and then the preset category with the highest total probability is determined as the recognition result corresponding to the CT data of the lung to be recognized.

In an exemplary embodiment, when calculating the total probability corresponding to the preset category, the calculation may be performed based on formula 1. For example, assuming that k is 2, the partition classification results corresponding to 2 partitions are: the probability of belonging to the normal state is 80%, the probability of belonging to the common pneumonia is 10%, and the probability of belonging to the new coronary pneumonia is 10%; the probability of belonging to the normal state is 60%, the probability of belonging to the common pneumonia is 30%, and the probability of belonging to the new coronary pneumonia is 10%. At this time, the total probability of belonging to the normal state is 92%, the total probability of belonging to the common pneumonia is 37%, and the total probability of belonging to the new coronary pneumonia is 19% based on the formula 1. Equation (1) is as follows:

wherein, P is a total probability corresponding to a certain preset category, and pt is a probability that the tth block belongs to the preset category.

In an exemplary embodiment, when the identification result is an abnormal result, that is, when a lung disease exists, a lesion blocking tensor corresponding to the lung CT data to be identified may be determined from the k blocking tensors according to the k blocking classification results. Specifically, when the block classification result includes probabilities that block tensors belong to each preset category, a target probability that a block tensor belongs to an identification result in the block classification result may be obtained for each block classification result, and then the block tensor corresponding to the maximum target probability of the obtained k target probabilities is determined as a focus block tensor corresponding to the lung CT data to be identified. By determining the focal segmentation tensor in the 3D tensor, the position where the abnormal result is most likely to occur can be located, so that the diagnosis of personnel such as medical care can be assisted based on the determined spatial position.

In addition, in an exemplary embodiment, when the above steps S320 and S330 are performed based on artificial intelligence such as machine learning or deep learning, as shown in fig. 4, k segment tensors may be used as input, the segment tensors are classified based on the lung disease classification model to obtain k segment classification results, after the k segment classification results are obtained, the k segment classification results are used as input, the k segment classification results are integrated based on the integration model to output an identification result corresponding to the lung CT data to be identified, and when the identification result is an abnormal result, the lesion segment tensor corresponding to the lung CT data to be identified is determined in the k segment tensors.

The lung disease classification model and the integration model can be obtained by training the constructed lung disease recognition neural network. Specifically, during training, a multi-group of sample data can be obtained, wherein each group of sample data comprises a blocking tensor corresponding to lung CT data and an identification result label corresponding to the lung CT data; and then dividing a plurality of groups of sample data into a training set, a verification set and a test set by taking the group as a unit, then constructing a lung disease recognition neural network, adjusting and determining parameters in the lung disease recognition neural network according to lung CT data included in the training set, the verification set and the test set, and obtaining a lung disease classification model and an integration model. The constructed lung disease recognition neural network can comprise a lung disease classification network and an integration network connected in series behind the lung disease classification network.

The following Model is initialized based on a 3D ResNet-18 Model using the Model weights pre-trained on the ImageNet dataset, the Model after the output dimension of the last fully-connected layer of the modified Model is 3 is a lung disease classification network, the bayesian noise-Or Model is an integration network, k is 4, and the training process of the above Model is explained in detail with reference to fig. 5 and 6, taking the preset classification as an example including normal pneumonia, common pneumonia, and new coronary pneumonia:

referring to fig. 5, when performing model training, sample data may be divided into a training set, a verification set, and a test set, and then training is performed based on the training set, verification is performed based on the verification set, and testing is performed based on the test set, so as to obtain a trained lung disease classification model and an integrated model.

Before training based on the training set, data enhancement may be performed on the data in the training set, for example, performing random horizontal and vertical flipping, random rotation angle (range 0-359 °) on the 3D tensor on the whole level, random cropping (at least retaining 81% of the original image area), adding random gaussian noise (mean of gaussian noise is 0, variance range 0-0.1, noise coefficient range 0-4), and so on.

After the enhanced training set is obtained, the lung disease recognition network may be trained based on the enhanced training set, and parameters and weights in the network may be updated. Wherein, the lung disease recognition neural network can be obtained by connecting the integration network in series with the lung disease classification network.

Specifically, each group of sample data includes 4 blocking tensors, which are input into a lung disease classification network, and 4 normalized blocking prediction information vectors (blocking classification results) with the size of 3 × 1 are derived, wherein vector elements represent the probability that a block belongs to normal pneumonia, common pneumonia and new coronary pneumonia. And after the block classification result is obtained, inputting the block classification result into an integration network, outputting a final identification result by the integration network, comparing the final identification result with an identification result label corresponding to the sample data, and calculating a loss function. Parameters, weights in the neural network for lung disease identification are then updated by back propagation using a stochastic gradient descent method to minimize the loss function.

Wherein, the cross entropy with L2 regularization can be used as the loss function of the identification task of new coronary pneumonia and common pneumonia, and the following formula (2):

where yi is a boolean value, and when the sample data belongs to the ith class, yi is 1; and yi is 0 in the rest cases. pi represents the final prediction probability of the ith class output by the integration layer; | w | non-woven phosphor²The larger the L2 norm representing the model parameters, the higher the model complexity, and the more likely overfitting occurs. λ is the L2 regularization coefficient used to constrain the model complexity.

In the present embodiment, λ is 1 × 10^-4The initial learning rate lr is 2 × 10^-3And the learning rate can be multiplied by an attenuation coefficient of 0.8 every 20 rounds of iteration to obtain a better identification result.

After training, the lung disease classification network after the parameters are updated by the training set may be validated based on the validation set. Specifically, the lung disease classification network is debugged based on the verification set data, and whether parameters and weights of the lung disease classification network are stored or not is determined. Specifically, model performance can be evaluated using the Kappa coefficient κ as an evaluation index. For the three classification problems of normal, common and new coronary pneumonia, the Kappa coefficient κ can be determined by the following formula (3):

wherein p is_oIndicating the accuracy of the overall classification, i.e. the ratio of the number of sample data correctly classified to the total number of sample data, p_eThe assumed probability of chance consistency is expressed by the following formulaFormula (4):

wherein, M represents the total amount of sample data, xi represents the total amount of the ith type identification result tag, and zi represents the total amount of the sample data predicted as the ith type identification result. The closer the Kappa coefficient Kappa is to 1, the better the recognition effect of the lung disease classification network on new coronary pneumonia and common pneumonia based on the updated parameters of the training set is.

When the degree that the Kappa coefficient calculated on the verification set is close to 1 satisfies the preset requirement, the parameters and the weights may be retained, and then the retained parameters are tested based on the test set. Specifically, the test set data is input so that the lung disease classification network retaining the parameters and weights can output the recognition result and the focus segmentation tensor. And aiming at the identification result, in the block classification result corresponding to the block tensor, the block tensor with the maximum probability corresponding to the identification result is the focus block tensor, and finally, the position where the focus exists most possibly is also the block tensor.

The lung disease classification network in the lung disease classification network after the test is the trained lung disease classification model, and the integration network in the lung disease classification network after the test is the trained integration model.

After the trained lung disease classification model and integration model are obtained, the identification of the lung disease can be completed based on the lung disease classification model and the integration model. Specifically, as shown in fig. 6, after the 3D tensor corresponding to the lung CT data to be identified is targeted, the 3D tensor can be segmented to obtain 4 segmented tensors, then the 4 segmented tensors are input into the lung disease classification model, 4 segmented classification results corresponding to the 4 segmented tensors can be output, then the 4 segmented classification results are input into the value integration model, and the identification result and the focus segmented tensor corresponding to the lung CT data to be identified can be output.

It should be noted that, in the above embodiment, where the Model is initialized based on the 3D ResNet-18 Model using the Model weights pre-trained on the ImageNet dataset, the Model after the output dimension of the last fully-connected layer of the modified Model is 3 is the lung disease classification network, and the bayesian noise-Or Model is the integration network, it is found through experiments that a better recognition effect can be obtained when the 3D tensor is scaled to H51, W512, and N2 Or 4.

Furthermore, data enhancement may not be performed for data entry in the validation set and the test set.

In summary, the present exemplary embodiment provides a lung disease identification method based on multi-instance learning, which is based on lung CT data and can realize accurate identification of lung diseases. On one hand, the method does not depend on a specific interpolation or sampling technology, retains the detailed information of original lung CT data, does not make any prior limit on the size and resolution of the lung CT data at the input end, and can be suitable for lung CT data with different resolutions and different cross section slice numbers; on the other hand, when model training is carried out, the rough spatial position of the focus can be positioned under the condition of no segmentation region marking without carrying out additional marking on the blocks.

It is noted that the above-mentioned figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the present disclosure, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.

Further, referring to fig. 7, the present exemplary embodiment further provides a lung disease recognition apparatus 700 based on CT data, which includes a segmentation obtaining module 710, a classification recognizing module 720 and a result outputting module 730. Wherein:

the blocking acquisition module 710 may be configured to acquire a 3D tensor corresponding to the lung CT data to be identified, and block the 3D tensor to obtain k blocking tensors; k is a positive integer greater than or equal to 2.

The classification identification module 720 may be configured to perform classification identification on the lung diseases of the k blocking tensors to obtain k blocking classification results.

The result output module 730 may be configured to output an identification result corresponding to the lung CT data to be identified according to the k blocking classification results, and determine a focus blocking tensor corresponding to the lung CT data to be identified in the k blocking tensors when the identification result is an abnormal result.

In an exemplary embodiment, when the size of the 3D tensor is the number of images × the height of the images × the width of the images, the block obtaining module 710 may be configured to block the 3D tensor in order based on the blocking parameters, so as to obtain k-1 block tensors having the size of the blocking parameters × the height of the images × the width of the images; taking the rest part which is not blocked in the 3D tensor as 1 blocked tensor; the size of the 1 blocking tensor is the residual number multiplied by the image height multiplied by the image width, and the residual number is smaller than or equal to the blocking parameter.

In an exemplary embodiment, when the block classification result includes probabilities that block tensors belong to each preset category, the result output module 730 may be configured to, for each block classification result, obtain a target probability that a block tensor in the block classification result belongs to the recognition result; and determining the blocking tensor corresponding to the maximum target probability in the k target probabilities as the focus blocking tensor corresponding to the lung CT data to be identified.

In an exemplary embodiment, when the block classification result includes probabilities that the block tensors belong to the preset categories, the result output module 730 may be configured to calculate, for each preset category, a total probability corresponding to the preset category based on the k block classification results; and determining the preset category with the maximum total probability as the identification result corresponding to the lung CT data to be identified.

In an exemplary embodiment, the block obtaining module 710 may be configured to pre-process the 3D tensor to obtain a 3D tensor satisfying a preset condition; the pre-treatment comprises at least one of the following treatments: normalization processing, scale scaling processing, denoising processing, window taking processing and contrast enhancement processing.

In an exemplary embodiment, the classification identification module 720 may be configured to classify the blocking tensor into k blocking classification results based on the lung disease classification model with the k blocking tensors as inputs.

In an exemplary embodiment, the result output module 730 may be configured to integrate the k segmented classification results based on the integration model with the k segmented classification results as input to output an identification result corresponding to the lung CT data to be identified, and determine a lesion segmented tensor corresponding to the lung CT data to be identified in the k segmented tensors when the identification result is an abnormal result.

In an exemplary embodiment, the apparatus for identifying a lung disease based on CT data may further include a model training module, configured to acquire a plurality of sets of sample data, and divide the plurality of sets of sample data into a training set, a verification set, and a test set by taking a set as a unit; each group of sample data comprises a group of block tensors corresponding to lung CT data and identification result labels corresponding to the lung CT data; constructing a lung disease recognition neural network, adjusting and determining parameters in the lung disease recognition neural network based on lung CT data included in the training set, the verification set and the test set to obtain a lung disease classification model and an integration model; wherein the lung disease recognition neural network comprises a lung disease classification network and an integration network connected in series behind the lung disease classification network.

The specific details of each module in the above apparatus have been described in detail in the method section, and details that are not disclosed may refer to the method section, and thus are not described again.

As will be appreciated by one skilled in the art, aspects of the present disclosure may be embodied as a system, method or program product. Accordingly, various aspects of the present disclosure may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.

Exemplary embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, various aspects of the disclosure may also be implemented in the form of a program product including program code for causing a terminal device to perform the steps according to various exemplary embodiments of the disclosure described in the above-mentioned "exemplary methods" section of this specification, when the program product is run on the terminal device, for example, any one or more of the steps in fig. 3 may be performed.

It should be noted that the computer readable media shown in the present disclosure may be computer readable signal media or computer readable storage media or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Furthermore, program code for carrying out operations of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is to be limited only by the terms of the appended claims.

Claims

1. A lung disease identification method based on CT data is characterized by comprising the following steps:

acquiring a 3D tensor corresponding to lung CT data to be identified, and partitioning the 3D tensor to obtain k partitioned tensors; wherein k is a positive integer greater than or equal to 2;

carrying out lung disease classification and identification on the k block tensors to obtain k block classification results;

and outputting an identification result corresponding to the lung CT data to be identified according to the k block classification results, and determining a focus block tensor corresponding to the lung CT data to be identified in the k block tensors when the identification result is an abnormal result.

2. The method of claim 1, wherein when the size of the 3D tensor is image number x image height x image width, the blocking the 3D tensor results in k blocked tensors, comprising:

partitioning the 3D tensor in sequence based on the partitioning parameters to obtain k-1 partitioning tensors with the size of the partitioning parameters, the image height and the image width;

taking the remaining part of the 3D tensor that is not blocked as 1 blocked tensor; wherein the size of the 1 blocking tensor is a remaining number x an image height x an image width, and the remaining number is less than or equal to the blocking parameter.

3. The method according to claim 1, wherein the block classification result includes probabilities of block tensors belonging to preset categories;

the determining, when the identification result is an abnormal result, a focus block tensor corresponding to the lung CT data to be identified in the k block tensors includes:

for each block classification result, acquiring the target probability of the block tensor belonging to the identification result in the block classification result;

and determining the blocking tensor corresponding to the maximum target probability in the k target probabilities as the focus blocking tensor corresponding to the lung CT data to be identified.

4. The method according to claim 1, wherein the block classification result includes probabilities of block tensors belonging to preset categories;

the outputting the identification result corresponding to the lung CT data to be identified according to the k block classification results comprises:

for each preset category, calculating a total probability corresponding to the preset category based on the k block classification results;

and determining the preset type with the maximum total probability as the identification result corresponding to the lung CT data to be identified.

5. The method of claim 1, wherein prior to the blocking the 3D tensor into k blocked tensors, the method further comprises:

preprocessing the 3D tensor to obtain a 3D tensor meeting preset conditions;

the pre-processing comprises at least one of the following processes: normalization processing, scale scaling processing, denoising processing, window taking processing and contrast enhancement processing.

6. The method of claim 1, wherein the classifying the pulmonary disease of the k segmented tensors to obtain k segmented classification results comprises:

taking the k block tensors as input, and classifying the block tensors based on a lung disease classification model to obtain k block classification results;

the outputting the identification result corresponding to the lung CT data to be identified according to the k blocking classification results, and determining a focus blocking tensor corresponding to the lung CT data to be identified in the k blocking tensors when the identification result is an abnormal result, includes:

and integrating the k block classification results based on an integration model by taking the k block classification results as input so as to output an identification result corresponding to the lung CT data to be identified, and determining a focus block tensor corresponding to the lung CT data to be identified in the k block tensors when the identification result is an abnormal result.

7. The method of claim 6, further comprising:

acquiring a plurality of groups of sample data, and dividing the plurality of groups of sample data into a training set, a verification set and a test set by taking a group as a unit; each group of the sample data comprises a group of block tensors corresponding to lung CT data and identification result labels corresponding to the lung CT data;

constructing a lung disease recognition neural network, and adjusting and determining parameters in the lung disease recognition neural network based on the lung CT data included in the training set, the verification set and the test set to obtain the lung disease classification model and the integration model;

wherein the lung disease recognition neural network comprises a lung disease classification network and an integration network concatenated after the lung disease classification network.

8. A lung disease identification device based on CT data, comprising:

the device comprises a blocking acquisition module, a data acquisition module and a data acquisition module, wherein the blocking acquisition module is used for acquiring a 3D tensor corresponding to lung CT data to be identified and blocking the 3D tensor to obtain k blocking tensors; k is a positive integer greater than or equal to 2;

the classification and identification module is used for performing lung disease classification and identification on the k block tensors to obtain k block classification results;

and the result output module is used for outputting the identification result corresponding to the lung CT data to be identified according to the k block classification results, and determining the focus block tensor corresponding to the lung CT data to be identified in the k block tensors when the identification result is an abnormal result.

9. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.

10. An electronic device, comprising:

a processor; and

a memory for storing executable instructions of the processor;

wherein the processor is configured to perform the method of any of claims 1-7 via execution of the executable instructions.