CN114360718A

CN114360718A - Feature fitting-based PET/CT automatic lung cancer diagnosis and classification system and construction method

Info

Publication number: CN114360718A
Application number: CN202210231670.9A
Authority: CN
Inventors: 朱闻韬; 黄海亮; 金源
Original assignee: Zhejiang Lab
Current assignee: Zhejiang Lab
Priority date: 2022-03-10
Filing date: 2022-03-10
Publication date: 2022-04-15
Anticipated expiration: 2042-03-10
Also published as: CN114360718B

Abstract

The invention discloses a PET/CT automatic lung cancer diagnosis classification system based on feature fitting and a construction method thereof, wherein the system comprises a feature extraction unit, a classification unit and a classification unit, wherein the feature extraction unit is used for extracting and obtaining first features of tumors according to a PET/CT image; the characteristic fitting unit is used for fitting the characteristics in the pathological images corresponding to the PET/CT images according to the PET/CT images to obtain second characteristics of the tumor; and the lung cancer diagnosis and classification unit is used for combining the first characteristic and the second characteristic to carry out lung cancer diagnosis and classification. The invention improves the diagnosis precision obtained by simply using the PET/CT image information characteristics by using the mode of fitting the matched pathological characteristics by using the PET/CT characteristics, is beneficial to the popularization and application of the existing intelligent diagnosis only based on the imaging science, and improves the diagnosis efficiency of clinicians. By the invention, the tumor condition can be better known before the operation of the patient, so that doctors can be better assisted to provide accurate treatment schemes for the patient, and the wound of the patient is reduced.

Description

Feature fitting-based PET/CT automatic lung cancer diagnosis and classification system and construction method

Technical Field

The invention relates to the field of medical images, in particular to a PET/CT automatic lung cancer diagnosis and classification system based on feature fitting and a construction method.

Background

PET/CT has great value in the diagnosis of benign and malignant pulmonary nodules, the staging of lung cancer and the evaluation after treatment of lung cancer in modern clinical medicine. 18F-FDG (18 fluoro labelled glucose analogue) is currently the most widely used tracer in PET/CT scanning, and can maintain cellular energy supply according to abnormal proliferation of malignant tumors and needs to increase glucose uptake and glycolysis, so that different types of tumors show different degrees of glucose uptake on glucose metabolism imaging images. The tracer decays within the patient and annihilates, producing a pair of 511keV gamma photons with emission directions about 180 ° opposite, and the detector collects information about the location and time at which the gamma photons reach the crystal. The acquired information is reconstructed by using an image reconstruction algorithm and post-processed, so that the metabolism and ingestion condition of the reaction tracer in the body of the patient can be obtained. Physicians can reflect the metabolic heterogeneity characteristics in an early and quantitative way through PET imaging, and combine various clinical manifestations of patients to determine further treatment schemes.

Pathological examination is a pathological diagnosis, called biopsy (biopsy), called living body for short, in which a small tissue taken from a diseased part of a patient is prepared into a pathological section, and then the pathological section is stained to determine the nature of the lesion through examination of cell morphology, histopathology and the like under a microscope. That is, a pathomorphological method for examining pathological changes in body organs, tissues or cells is one of the examination methods which has the highest diagnostic accuracy among all examinations and is known as "gold standard". But the way it is checked is invasive.

With the great success of the deep learning technology in the field of computer vision, more and more scholars apply the deep learning technology to medical image analysis, such as classification, detection and segmentation, registration and retrieval, and achieve better effects. Deep learning can automatically learn the salient high-order features of a specific task from a large amount of data so as to complete the specific task, but has certain requirements on the training data volume.

Meanwhile, deep learning also emerges various methods for solving the practical problem, such as multitask learning, knowledge distillation and other methods can be used for reference. Multitask learning (multi task learning): simply speaking, there are multiple objective functions loss to learn simultaneously even though there is multi-task learning. For example, a short video with a big fire at present usually predicts information of multiple dimensions such as how long a user is interested in/not interested in the video, and looks like/not like, and forwards/not forwards the video. The multiple tasks can be learned by establishing a model for each task, and can also be learned by multiple tasks of one model for one-time full learning. Knowledge distillation (knowledge distillation) is a common method for model compression, and is different from pruning and quantification in model compression, and knowledge distillation is to train a small model by constructing a lightweight small model and utilizing supervision information of a large model with better performance so as to achieve better performance and precision. Originally proposed and applied on top of the classification task by Hinton in 2015, this large model was called teacher (teacher model) and the small model was called Student (Student model). The supervisory information from the output of the Teacher model is called knowledge, while the process by which students learn to migrate the supervisory information from Teacher is called Distillation.

For the existing automatic lung cancer diagnosis classification model based on PET/CT, two schemes of a classification model based on image omics and a classification network based on deep learning are mainly provided. Imaging omics is a new emerging research method which is currently emerging, and is proposed by dutch scholars in 2012, and the imaging omics method mainly extracts high-flux image features through images of different modalities (such as CT, MRI, PET and the like), and assists physicians in making the most accurate judgment by means of deeper mining, prediction and analysis of mass image data information, so as to realize the evaluation of tumor heterogeneity and the prognosis evaluation of tumors. In the deep learning method, a convolutional neural network is mostly adopted to construct a specific classification model for tumor classification, such as a Resnet classification network, a DenseNet classification network and the like and a deformation thereof. Compared with biopsy, the two methods can reduce the pain caused by biopsy, improve the working efficiency to a certain extent, reduce the economic burden of patients and provide a healthier and safer way for future patient disease review. Although the existing automatic classification model based on PET/CT for lung cancer diagnosis has achieved better classification accuracy, there is still a long way to go for clinical requirements. Although the existing automatic lung cancer diagnosis classification model based on pathology can obtain higher classification precision, the diagnosis result is slower and traumatic, which is a disadvantage for the rapid diagnosis of lung cancer screening. Therefore, how to combine the advantages of the image and the pathology to shield the disadvantages of the image and the pathology, and introducing the information of the pathological image into the image when the network is trained to be used as prior information during use, so that the accuracy of the image during application is improved, the precision of the PET/CT automatic lung cancer diagnosis is greatly improved, a doctor is better assisted to perform early diagnosis on a patient, and the better help of the patient to make a diagnosis and treatment scheme is also one of the current research hotspots.

Disclosure of Invention

The invention aims to provide a PET/CT automatic lung cancer diagnosis classification system based on feature fitting and a construction method thereof aiming at the defects of the prior art, and the system and the construction method use a feature fitting network to fit out features which are most similar to corresponding pathological features from PET/CT by using multitask learning and distillation learning technology and are superposed on the features extracted by the conventional classification network for assisting the lung cancer diagnosis based on PET/CT images so as to achieve the effects of improving the training efficiency and the upper limit of precision of the PET/CT network.

The purpose of the invention is realized by the following technical scheme:

a PET/CT automatic lung cancer diagnosis and classification system based on feature fitting comprises:

the characteristic extraction unit is used for extracting and obtaining first characteristics of the tumor according to the PET/CT image;

the characteristic fitting unit is used for fitting the characteristics in the pathological images corresponding to the PET/CT images according to the PET/CT images to obtain second characteristics of the tumor;

and the lung cancer diagnosis and classification unit is used for combining the first characteristic and the second characteristic to carry out lung cancer diagnosis and classification.

Further, still include:

the image preprocessing unit is used for removing the non-tumor region in the PET/CT image, operating according to the PET/CT image label of the professional corresponding to the non-tumor region, simultaneously acquiring the patch image of the tumor region, and ensuring that the size of the patch can include the whole tumor region.

Further, the feature fitting unit is a feature fitting convolutional neural network obtained by taking the PET/CT image as input and inputting the feature of the pathological image corresponding to the PET/CT image as a label for training.

Further, the input PET/CT image is extracted by a feature extractor obtained by lung cancer diagnosis classification training based on pathological images. The lung cancer diagnosis classification training based on the pathological images refers to training taking the pathological images as input and lung cancer diagnosis classification as a prediction target.

Further, the extraction method specifically comprises the following steps:

performing operation according to the corresponding pathological image label of the professional, and segmenting the input PET/CT image corresponding to the pathological image to obtain a plurality of pathological patch images containing tumor regions;

and taking each pathology patch image as the input of a feature extractor obtained by lung cancer diagnosis classification training based on the pathology image, extracting to obtain a plurality of features, and selecting one feature from the plurality of features as the feature of the pathology image.

Further, selecting one of the plurality of features as the feature of the pathological image, and setting different screening schemes according to different situations, wherein a specific screening scheme is preferably as follows: and selecting the feature with the minimum sum of Euclidean distances to all other features as the feature of the pathological image.

Further, the tumor coverage rate of the pathology patch image is greater than or equal to 0.8, that is, the area ratio of the area of the region framed according to the pathology image labels of the professionals to the whole pathology patch image is greater than or equal to 0.8.

Further, the feature extraction unit is a feature extractor obtained by training lung cancer diagnosis classification based on PET/CT images. The training of lung cancer diagnosis classification based on the PET/CT image is training taking the PET/CT image as input and lung cancer diagnosis classification as a prediction target.

A construction method of a PET/CT automatic lung cancer diagnosis and classification system based on feature fitting comprises the following steps:

the method comprises the steps of collecting PET/CT images and pathological images of a patient, obtaining corresponding lung cancer diagnosis classification results, and segmenting the pathological images to obtain a plurality of pathological patch images containing tumor regions.

And taking the pathological patch image corresponding to each feature extractor as the input of the feature extractor obtained by the lung cancer diagnosis classification training based on the pathological images, extracting to obtain a plurality of features, and selecting one of the plurality of features as the feature of the pathological image corresponding to the PET/CT image.

Constructing a convolutional neural network, inputting a PET/CT image, inputting the characteristics of a pathological image corresponding to the PET/CT image as a label for training, and using the trained convolutional neural network as a characteristic fitting unit;

and constructing a feature extractor and a classifier, taking the PET/CT image as the input of the feature extractor and the feature fitting unit, superposing the output features of the feature extractor and the feature fitting unit, then taking the superposed output features as the input of the classifier, training the lung cancer diagnosis classification result of the PET/CT image as a label, taking the trained feature extractor as a feature extraction unit, and taking the classifier as a lung cancer diagnosis classification unit.

The method has the advantages that the PET/CT image and the pathology based on the same case have the same characteristics in a certain space, and the classification characteristics of the pathology are more prominently expressed as the golden standard of the existing clinical diagnosis, so that the PET/CT characteristics are used for fitting the corresponding pathological characteristics, and the diagnosis classification accuracy of the PET/CT image is improved in an assisting manner. Therefore, before pathological examination, more accurate diagnosis information can be provided for a clinician through non-invasive PET/CT images compared with the existing classification scheme, and the diagnosis efficiency of the clinician is improved.

Drawings

FIG. 1 is a schematic diagram of the PET/CT automatic lung cancer diagnosis system based on feature fitting according to the present invention;

FIG. 2 is a schematic diagram showing the structure of a feature extractor obtained from the classification training of pathological image-based lung cancer diagnosis and the structural relationship with the PET/CT automatic lung cancer diagnosis system based on feature fitting according to a preferred embodiment of the present invention;

FIG. 3 is a flow chart of the construction of a PET/CT automatic lung cancer diagnosis system based on feature fitting according to a preferred embodiment of the present invention;

FIG. 4 is a diagram of a neural network structure of a PET/CT automatic lung cancer diagnosis system based on feature fitting according to a preferred embodiment of the present invention.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, without departing from

In the context of the present application, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.

The invention provides a PET/CT automatic lung cancer diagnosis and classification system based on feature fitting, as shown in figure 1, comprising:

and the lung cancer diagnosis and classification unit is used for carrying out diagnosis and classification on the lung cancer by combining the first characteristic and the second characteristic, and the general classification comprises lung adenocarcinoma and lung squamous carcinoma.

The invention uses the PET/CT image to fit the characteristics of the corresponding pathological image, and can provide more accurate diagnosis information for a clinician only through the non-invasive PET/CT image, thereby assisting in improving the diagnosis classification precision of the PET/CT image.

Since the actually extracted features of the tumor in the PET/CT image not only increase the calculation amount but also affect the feature extraction accuracy for the non-tumor region in the PET/CT image, as a preferred embodiment, the PET/CT automatic lung cancer diagnosis and classification system based on feature fitting further includes: and the image preprocessing unit is used for removing non-tumor regions in the PET/CT image. Illustratively, the PET/CT image cut patch may be operated on a tumor mask. Acquiring an circumscribed cube according to the tumor, expanding a plurality of pixels outwards, acquiring a tumor region in the PET/CT image according to the cube, and zeroing a part which is 0 in mask data in the tumor region, namely removing a non-tumor region in the PET/CT image, wherein the part is used as the input of the PET/CT automatic lung cancer diagnosis and classification system. Here, in order to facilitate subsequent training of the deep convolutional neural network model, the size of each patch may be selected to be uniform, and it is ensured that the size may completely contain all tumors in the batch of data.

Preferably, the feature fitting unit is a feature fitting convolutional neural network obtained by taking the PET/CT image as input and inputting the feature of the pathological image corresponding to the PET/CT image as a label for training. The input PET/CT image is obtained by extracting the features of the pathological image through a feature extractor obtained by training lung cancer diagnosis classification based on the pathological image, as shown in fig. 2. The extraction method specifically comprises the following steps:

segmenting the input PET/CT image corresponding to the pathological images to obtain a plurality of pathological patch images containing tumor regions;

and taking each pathology patch image as the input of a feature extractor obtained by lung cancer diagnosis classification training based on the pathology image, extracting to obtain a plurality of features, and selecting one feature from the plurality of features as the feature of the pathology image. The feature of the pathological image may be selected randomly or according to a rule, and preferably, the feature in which the sum of euclidean distances to all other features is the smallest is selected as the feature of the pathological image.

Corresponding to the above PET/CT automatic lung cancer diagnosis and classification system based on feature fitting, the present invention further provides a method for constructing the above PET/CT automatic lung cancer diagnosis and classification system based on feature fitting, which is described in detail below according to embodiments and drawings, wherein the structure of the construction method is shown in fig. 2, the construction process is shown in fig. 3, and the method specifically includes:

the method comprises the following steps: constructing a training data set, and collecting one-to-one matched PET/CT image data

And full scan pathology image

. Simultaneously acquiring lung cancer diagnosis classification results corresponding to the lung cancer diagnosis classification results as label data

And

. Here, image labels of the same patient case

And pathological label

Are consistent.

In order to improve the accuracy, as a preferred scheme, the data is preprocessed:

wherein, for

Data, first for all PET/CT image data

Layer thickness level for uniform imaging: interpolating the image data with larger layer thickness to the layer thickness level with smaller layer thickness of the image data;

second according to the tumor mask

Performing a cut-and-patch operation based on the mask data

Acquiring a circumscribed cube, and acquiring image data according to the cube

And in a patch image of

Data of mask

The portion of 0 is zeroed. Obtaining a Patch image of the tumor from which the non-tumor region was removed and recording the Patch image as

As input for subsequent classification of the network. This is achieved byFor convenience in subsequent deep convolutional neural network model training, the size of each patch can be selected to be uniform, and it is ensured that the size can completely contain all tumors in the batch of data. The method comprises the following steps: first, the mask data is obtained according to all the mask data in the data batch

The maximum tumor size is obtained, and then the maximum size is required to be exponentially multiplied by 2 or 7 times 2 according to the length, width and height. In this embodiment, the calculated patch size is

。

For the

Firstly, in order to avoid the influence of color inconsistency caused by pathological staining on subsequent results, the color normalization operation is carried out: selecting a pathological image with better dyeing effect as a target pathological image according to the recommendation of a professional doctor in all pathological data

Normalizing the color of the pathology image of all other cases to the target pathology image using a structure-preserving color normalization (SPCN) technique

The same color level, thereby eliminating the network performance reduction caused by the inconsistency of pathological staining colors;

secondly, because the pathological image of the full scan is very large and is difficult to be directly input into the convolutional neural network for training, the tumor region is selected according to the tumor mask of the pathological image, and then the cut patch operation is performed, specifically:

mask data from full scan pathology image

For pathological image

Performing a cut patch operation, each pathological patch image

Must contain the tumor area, preferably to satisfy the tumor mask data

The coverage of (a) is 80% or more, namely:

（1）

wherein

To represent

The area occupied by the doctor in the doctor for marking the tumor,

representing the area occupied by the pathology patch image. Simultaneous each pathology patch image

Are of the same size, e.g. all

Each pathology patch image

The label selects the corresponding full-scanning pathological image

Tag data of

As a corresponding label

；

Further, in order to ensure the data balance, an overlay-tile strategy (overlay-tile strategy) may be adopted, so that each is enabled to be uniform

The number of cuts remains substantially the same.

The mask can be made by a professional doctor to mark the tumor region, or other conventional mask making methods.

Step two: and training a feature extractor according to a scheme of five-fold cross validation used by a case, and screening to obtain the features of all pathological images. The training of the feature extractor obtained by the pathological image-based lung cancer diagnosis classification training is based on five-fold cross validation of data, five groups of feature extraction parameters are obtained by the five-fold cross validation, and then the features in the five complementary test sets are respectively extracted. And taking the pathological patch image of the test set corresponding to each feature extractor as the input of the feature extractor obtained based on the lung cancer diagnosis classification training of the pathological image, extracting to obtain a plurality of features, and selecting one of the features as the feature of the pathological image corresponding to the PET/CT image. Here, if the number of data sets is small, a scheme such as ten-fold cross validation may be selected for adjustment.

Illustratively, the feature extractor obtained by training lung cancer diagnosis classification based on pathological images refers to a feature extraction structure of a classification network CNN1 obtained by training lung cancer diagnosis classification as a prediction target with the pathological images as input, and the pathological patch images cut in the step one are subjected to

And label therefor

Inputting the data into a classification network CNN1 for training, obtaining a better result and storing a network parameter D of CNN1₁And reserving the feature extraction part to obtain the feature extractor. Because the pathological data are gold standards, a better classification result can be quickly achieved on the lung cancer diagnosis data set, and the classification precision of the test set is required to be more than 95%. Taking ResNet-18 as the main structure of the classification network CNN1 as an example (shown in Table 1 as ResNet-18 network structure), the training specifically comprises the following steps:

TABLE 1 ResNet-18 network architecture

WhereinMNumber representing the diagnostic classification of the tumor, 2 in this example, lung adenocarcinoma and lung squamous carcinoma, respectively;

1. according to the pathological image

Patch image to be cut out

Dividing the data into a training set and a test set, and dividing the data of each fold into the training set accounting for 80 percent and the test set accounting for 20 percent according to a five-fold cross validation strategy, wherein the data of the five folds are complementary, namely the data of the test set of the five folds are not overlapped.

2. Samples in the training set (pathology images)

Excised pathology patch image

) The input is trained in the constructed classification network CNN1 network, and the cross entropy loss function is selected as the loss value of the classification network CNN1 and is recorded as loss _1 for back propagation. The specific calculation method of loss _1 is as follows:

（2）

whereinMRepresenting the number of diagnostic categories of the tumor;y _icis a sign function (0 or 1) ifiTrue class of sample equalscTaking 1, otherwise, taking 0;p _icindicating that the input belongs to the category after passing through the network softmax functioncThe prediction probability of (2).

3. Until the loss function is converged or the iteration times are reached to complete training, a test set is adopted for verification and testing, and then a group of network parameters D with the best effect is stored₁Meanwhile, the classification accuracy of the test set is required to reach more than 95%.

4. Use of network parameter D in a classifying network CNN1₁Keeping the average pooling layer and the network structure before the average pooling layer as a feature extractor, and outputting the features of the pathology patch image by the feature extractor

。

Because the number of cuts for a full-scan pathological image is huge, and the number of cuts for a corresponding PET/CT image is only 1, if feature fitting is directly carried out, an image feature exists during feature fitting

Corresponding to multiple pathological patch image features

Therefore, a screening operation of pathological characteristics is required. Can randomly select a pathological patch image feature

Fitting is performed, but this is not an optimal solution, and preferably, features closest to features of other pathology patch images are selected, and the feature screening rule selected in this example is specifically:

for all acquired pathology patch image features

Calculating Euclidean distance between every two images, and selecting all other pathological patch image features

The characteristic of the pathology patch image with the minimum Euclidean distance sum is taken as the characteristic of the pathology imageF _PI。

Step three: constructing a convolution neural network CNN2 and preprocessing the PET/CT tumor patch image in the step one

As input, the characteristics of the pathological image corresponding to the PET/CT image are inputF _PITraining as a label, fitting the characteristics of the corresponding pathological image by using a PET/CT image, fixing the network parameters after obtaining a better result, and obtaining a characteristic fitting unit; the method specifically comprises the following steps:

1. a convolutional neural network CNN2 based on PET/CT is established, the convolutional neural network CNN1 in the present embodiment adopts a feature extraction part of ResNet-18 as a main feature fitting network, and the specific structure is shown in table 2:

TABLE 2 feature fitting network

2. The tumor patch image of the PET/CT preprocessed in the step one is used

The image is input into a convolutional neural network CNN2 as input, and the PET/CT tumor patch image output by the convolutional neural network CNN2

Is characterized by the fitting of

Inputting the characteristics of the pathological image corresponding to the PET/CT imageF _PIAs a label, a scheme of calculating the cosine similarity as a loss function of the convolutional neural network CNN2, denoted as loss _2, is selected here for back propagation. The specific calculation method is as follows:

（3）

wherein

The dimension representing the output characteristic of the convolutional neural network CNN2,

to represent

The mean value in the dimension is the average value,

to represent

Mean in dimension.

3. Until the loss function converges or the iteration times are reached to complete the training, and after a better result is obtained, the network parameter D is fixed₂And obtaining a feature fitting unit.

Step four: constructing a feature extractor and a classifier, and using the tumor patch image of the PET/CT preprocessed in the step one

The feature extractor and the feature fitting unit are used as input of the classifier, the output features of the feature extractor and the feature fitting unit are mixed and then used as input of the classifier, the lung cancer diagnosis classification result of the PET/CT image is used as a label to be trained, and the trained feature extractor is used for doingThe classifier is used as a lung cancer diagnosis classification unit.

Wherein, the feature extractor and classifier can be an integrated PET/CT-based lung cancer diagnosis classification convolutional neural network CNN3, which operates in conjunction with the PET/CT-based feature fitting convolutional neural network CNN2, namely: after the feature extraction of the CNN3 is completed, the features extracted by the CNN3 and the features extracted by the CNN2 are mixed and then input into a classification network based on a full connection layer in the CNN3 to perform lung cancer diagnosis classification, and the feature extraction and classification network parts in the trained lung cancer diagnosis classification convolutional neural network CNN3 are respectively used as a feature extraction unit and a lung cancer diagnosis classification unit, and the method specifically comprises the following steps:

1. establishing a lung cancer diagnosis classification network CNN3 based on PET/CT, wherein the lung cancer diagnosis classification network CNN3 based on PET/CT in the example adopts a DenseNet-121 structure, and the specific structure is as shown in Table 3:

table 3: DenseNet-121 network architecture

2. The tumor patch image of the PET/CT preprocessed in the step one is used

As input of the PET/CT-based lung cancer diagnosis and classification network CNN3 and the feature fitting unit, and extracting features of the PET/CT-based lung cancer diagnosis and classification network CNN3 behind the average pooling layer of the PET/CT-based lung cancer diagnosis and classification network CNN3 network

Features extracted by the feature fitting unit

Performing superposition operation, then entering a rear full-connection layer classification network, outputting the classification result for lung cancer diagnosis, training the CNN3 network by cooperating with the feature fitting of the feature fitting unit, wherein the loss calculation selects crossThe entropy loss function is consistent with the formula (2) until the loss function converges or the iteration number is reached to finish the training, and the network parameter D is fixed₃Namely, a feature extraction unit and a lung cancer diagnosis classification unit are obtained.

The constructed feature fitting-based PET/CT automatic lung cancer diagnosis and classification system can be used for lung cancer diagnosis and classification, and the classification precision is higher than that of the current system which only uses image data as shown in FIG. 4

The trained network classification precision is high, and experimental results show that the classification precision is at least improved by 2-3%.

It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. This need not be, nor should all embodiments be exhaustive. And obvious variations or modifications of the invention may be made without departing from the scope of the invention.

Claims

1. A PET/CT automatic lung cancer diagnosis and classification system based on feature fitting is characterized by comprising:

2. The system of claim 1, further comprising:

and the image preprocessing unit is used for removing non-tumor regions in the PET/CT image.

3. The system according to claim 1, wherein the feature fitting unit is a feature fitting convolutional neural network obtained by training with the PET/CT image as input and the feature of the pathological image corresponding to the PET/CT image as a label.

4. The system of claim 3, wherein the input PET/CT image is extracted by a feature extractor obtained by training lung cancer diagnosis classification based on pathological images.

5. The system according to claim 4, wherein the extraction method is specifically:

6. The system according to claim 5, characterized in that as a feature of the pathological image is selected from a plurality of features, in particular: and selecting the feature with the minimum sum of Euclidean distances to all other features as the feature of the pathological image.

7. The system of claim 5, wherein a tumor coverage of the pathology patch image is equal to or greater than 0.8.

8. The system according to claim 5, wherein the feature extraction unit is a feature extractor obtained based on a lung cancer diagnosis classification training of PET/CT images.

9. A method for constructing a PET/CT automatic lung cancer diagnosis and classification system based on feature fitting is characterized by comprising the following steps:

acquiring a PET/CT image and a pathological image of a patient, acquiring a corresponding lung cancer diagnosis classification result, and segmenting the pathological image to obtain a plurality of pathological patch images containing tumor regions;

taking each pathology patch image as the input of a feature extractor obtained by lung cancer diagnosis classification training based on pathology images, extracting and obtaining a plurality of features, and selecting one feature from the plurality of features as the feature of the pathology image corresponding to the PET/CT image;