CN109300530B - Pathological picture recognition method and device - Google Patents

Abstract

The invention discloses a pathological picture identification method and a pathological picture identification device, wherein the method comprises the following steps: acquiring a pathological picture to be identified; inputting the pathological picture to be recognized into a plurality of different types of deep neural network models generated by pre-training, recognizing the pathological picture to be recognized, and obtaining a preliminary recognition result by each type of deep neural network model; the method comprises the following steps that a plurality of deep neural network models of different types are generated according to pre-training of a plurality of pathological image samples; and fusing the primary recognition results obtained by the deep neural network models of different types to obtain the final recognition result of the pathological picture to be recognized. According to the technical scheme, the efficiency and the accuracy of pathological picture identification are improved.

Description

Pathological picture recognition method and device

technical field

The present invention relates to the field of medical technology, and in particular, to a method and device for identifying pathological pictures.

Background technique

Lymph node metastasis is the most common way of tumor metastasis. Radical resection of advanced gastric cancer includes complete resection of the primary tumor of gastric cancer, metastatic lymph nodes, and invaded tissues and organs. Postoperative pathological diagnosis of gastric cancer is the gold standard for the diagnosis of gastric cancer and provides an important basis for the staging and treatment of patients. In the pathological diagnosis, the evaluation of whether lymph node metastasis is the key to the diagnosis. Pathologists need to carefully and carefully observe each lymph node one by one. The whole process is not only time-consuming and labor-intensive, but also depends on experience. The accuracy rate is not ideal, and different experience There is a risk that the same pathological picture may have different identification conclusions by doctors.

SUMMARY OF THE INVENTION

An embodiment of the present invention provides a method for identifying a pathological picture to improve the efficiency and accuracy of identifying a pathological picture, and the method includes:

Obtain pathological pictures to be identified;

Input the pathological picture to be identified into multiple different types of deep neural network models generated by pre-training, identify the pathological picture to be identified, and obtain a preliminary identification result for each type of deep neural network model; the multiple different types of deep neural network models The network model is pre-trained and generated according to multiple pathological image samples;

The preliminary identification results obtained by a plurality of different types of deep neural network models are fused to obtain the final identification result of the pathological picture to be identified.

The embodiment of the present invention also provides a pathological picture recognition device to improve the efficiency and accuracy of pathological picture recognition, the device includes:

an acquisition unit for acquiring pathological pictures to be identified;

The identification unit is used to input the pathological pictures to be identified into multiple different types of deep neural network models generated by pre-training, identify the pathological pictures to be identified, and obtain a preliminary identification result for each type of deep neural network model; Different types of deep neural network models are pre-trained and generated based on multiple pathological image samples;

The fusion unit is used to fuse the preliminary recognition results obtained by a plurality of different types of deep neural network models to obtain the final recognition result of the pathological picture to be recognized.

An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the above pathological picture recognition method when the processor executes the computer program.

An embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program for executing the above method for identifying a pathological picture.

The technical solution provided by the embodiment of the present invention is to first obtain the pathological picture to be identified, and then input the to-be-identified pathological picture into a plurality of different types of deep neural network models generated by pre-training, and identify the to-be-identified pathological picture. The network model obtains a preliminary identification result; the multiple different types of deep neural network models are pre-trained and generated according to multiple pathological image samples; finally, the preliminary identification results obtained by the multiple different types of deep neural network models are fused to obtain the The final recognition result of the pathological image to be recognized is described. Since the trained deep neural network model has the function of automatic recognition of pathological images, the pathological image can be input into the deep neural network model, and the area that may have malignant lesions on the pathological image can be identified. For the classification of benign and malignant pathological pictures, the whole process saves time and effort, which not only improves the efficiency of pathological picture recognition, but also does not depend on the personal experience of doctors. The recognition results are obtained by fusion, which greatly improves the accuracy of pathological image recognition.

The technical solutions provided by the embodiments of the present invention can be applied not only to the identification of pathological pictures of gastric lymph node cancer metastasis, but also to the identification of pathological pictures of other cancers.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts. In the attached image:

1 is a schematic flowchart of a method for identifying a pathological picture in an embodiment of the present invention;

2 is a specific example diagram of a method for identifying a pathological picture in an embodiment of the present invention;

3 is a schematic diagram of a multi-GPU parallel training principle in an embodiment of the present invention;

FIG. 4 is a schematic diagram of an apparatus for recognizing a pathological picture in an embodiment of the present invention.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention more clearly understood, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but not to limit the present invention.

Before introducing the embodiments of the present invention, the technical terms involved in the present invention are first introduced.

1. False positive rate: false positive rate, the ratio of the number of samples that are actually negative and predicted by the model to be positive among all negative samples.

2. False negative rate: false negative rate, the ratio of the number of samples that are actually positive and predicted by the model to be negative among all positive samples.

3. Accuracy rate: Accuracy=(number of correct predicted samples)/(total number of samples).

4. Training set: digital pathological images of gastric lymph node cancer metastasis with annotated cancer cell areas (lesion areas) input to the model training.

5. Test set: digital pathological images of gastric lymph node cancer metastasis with annotated cancer cell areas (lesion areas) that are not input to model training.

6. Validation set: digital pathological images of gastric lymph node cancer metastasis without annotated cancer cell area (lesion area)

7. Transfer learning:

Transfer the trained model parameters to the new model to speed up the new model training.

8. scn file: a medical image storage format, which requires special processing when reading.

9. Top5 error rate: imagenet images usually have 1000 possible categories. For each image, 5 category labels can be predicted at the same time. When any one of the predictions is correct, the result is correct. When all 5 are wrong. , it is the prediction error, and the classification error rate at this time is called the top5 error rate.

10. HE staining: hematoxylin-eosin staining, the hematoxylin staining solution is alkaline, which mainly makes the chromatin in the nucleus and the nucleic acid in the cytoplasm to be purple-blue; eosin is Acid dyes, mainly red color components in the cytoplasm and extracellular matrix.

11. Negative samples: samples that do not contain cancer cells, also called negative samples: pathological pictures of normal or benign lesions.

12. Positive samples: samples containing cancer cells, also known as positive samples: pathological pictures of malignant lesions.

The purpose of the present invention is to make a diagnosis of the presence or absence of cancer cells according to the characteristics of the lymph nodes on the pathological tissue slices of the lymph nodes surgically removed from patients with gastric cancer, and to display the location of cancer cells if there are cancer cells.

As the proportion of China's population aged 60 and above to the total population continues to increase, the number of people suffering from cancer will increase rapidly based on the incidence of cancer in the population. This will lead to further strain on medical resources. In the process of cancer diagnosis, pathological diagnosis is the gold standard for final diagnosis. The traditional diagnosis of lymph node cancer metastasis requires a pathologist to observe the lymph nodes repeatedly under a microscope to determine the number of lymph nodes and whether each lymph node has cancer metastasis. Limited by the doctor's experience and the doctor's fatigue state, there will be a certain probability of misdiagnosis and missed diagnosis. The accuracy rate of the invention reaches 99.80% at the patch level, and the false positive rate is lower than 0.06% at the patch level, which effectively assists doctors in diagnosis, reduces the misdiagnosis rate and missed diagnosis rate of doctors, and finally improves the medical treatment experience of patients.

Then, the process of the inventor from discovering the technical problem to proposing the solution according to the embodiment of the present invention will be introduced.

From the 1970s to the present, machine learning technology has continued to develop rapidly, improving human productivity. In the process of machine learning development, hardware performance and the amount of effective data have always restricted the development of machine learning. Around 2010, the substantial improvement of hardware performance and the accumulation of a large amount of high-quality data prompted an important part of machine learning - deep learning, to make great breakthroughs in algorithms and applications. In image processing, deep learning models have made leaps and bounds in tasks such as classification, detection, and segmentation. In the case of a large amount of effective data, using an appropriate deep learning model to model the data, the effect is often due to the effect of traditional machine learning, and it has the ability to transfer learning to different data sets, which significantly reduces the traditional machine learning. The cost of doing feature engineering. Therefore, the inventor mainly adopts the neural network model to model the data in the present invention.

At present, the diagnosis of medical digital pathological images generally adopts the general process of patching/training classification model/prediction. In this process, the cut patch mainly adopts three specifications of 256*256, 512*512 and 1024*1024. The classification model mainly adopts the models with better accuracy of Top5 on the academic dataset ImageNet, such as the Inception model series, the ResNet model series, and the VGG model series. Use a single trained model to get results on new data when making predictions. Since the ImageNet dataset and digital pathology images have many similarities in feature extraction, the model that works well on the ImageNet dataset can also achieve better results when applied to the digital pathology image dataset. During the training/testing process of the model, the existing implementation scheme continuously adjusts the model according to the existing data to achieve better results.

However, the inventor found that the existing solutions have the following technical problems, and proposed corresponding solutions for the discovered technical problems:

A. When determining the model to be used, due to the limitations of code implementation and hardware, only one deep model is selected for training and testing. Limited to the expressive ability of the same depth model, the model unilaterally optimizes a certain performance index while making other performance indicators low. Since the inventor has considered this technical problem, the technical solution proposed is: firstly select 6 models for training, analyze their characteristics, and then use the results of the 3 models to comprehensively process to obtain the final result, at the patch level, control false positives While 0.06%, the false negative rate is guaranteed to be below 0.3%, and the overall accuracy rate is above 99.8%.

B. During model training, the training speed of using a single GPU is slow, and the parallel computing advantages of multiple GPUs are not fully utilized. Considering this technical problem, the inventor proposes to use multiple GPUs for training. With the same amount of training data, the model training time is reduced, the model debugging time in the training phase is shortened, and the developer's time is saved.

C. Model prediction improvement stage. In the existing implementation scheme, more information is mined from the existing data to improve the model. If it encounters a cancer cell category that the model has not processed, the model cannot recognize it, and the direction of the model iteration is not consistent with the actual pathological diagnosis. Considering this technical problem, the inventor proposed a technical solution: by continuously combining medical professional knowledge, supplementing the digital pathological pictures that the model has not processed, so that the model can identify more cancer cell forms, effectively reduce the missed detection rate, and iterate the model. The direction is more in line with the identification needs of pathological pictures, which can also be said to be the actual needs of pathological diagnosis.

The solution proposed by the inventor involves cancer cell detection based on a deep learning classification model (deep neural network model). According to the process, the method may include: 1. Obtaining positive patch samples (positive samples) and negative patch samples (negative samples) from digital pathology scn format files; 2. Comprehensively analyzing the conditions of negative and positive patch samples, and selecting 6 suitable patch samples The deep learning classification model makes full use of the advantages of different models; 3. Use the training set data to train the six deep neural network models determined above in a multi-GPU environment, so that each model has the ability to predict the best patches on each lymph node in the digital pathological image. Malignant ability; 4. Use the test set to test the performance of each model in detail, and test the performance of each model, including false positive rate, false negative rate, and accuracy rate; 5. According to the performance of each model, 3 models are comprehensively selected , fuse the results of the three models on the validation set to improve the accuracy and reduce the false positive rate and false negative rate. 6. Analyze the final prediction results, fully combine the professional knowledge of doctors, supplement new training data, and target iterative models to further improve the recognition accuracy. The identification scheme of the pathological picture is described in detail as follows.

FIG. 1 is a schematic flowchart of a method for identifying a pathological picture in an embodiment of the present invention. As shown in FIG. 1 , the method includes the following steps:

Step 101: Obtain the pathological picture to be identified;

Step 102: Input the pathological picture to be identified into a plurality of different types of deep neural network models generated by pre-training, identify the pathological picture to be identified, and obtain a preliminary identification result for each type of deep neural network model; The deep neural network model is pre-trained and generated according to multiple pathological image samples;

Step 103: Fusion of preliminary identification results obtained by multiple different types of deep neural network models to obtain a final identification result of the pathological picture to be identified.

The technical solution provided by the embodiment of the present invention is to first obtain the pathological picture to be identified, and then input the to-be-identified pathological picture into a plurality of different types of deep neural network models generated by pre-training, and identify the to-be-identified pathological picture. The network model obtains a preliminary identification result; the multiple different types of deep neural network models are pre-trained and generated according to multiple pathological image samples; finally, the preliminary identification results obtained by the multiple different types of deep neural network models are fused to obtain the The final recognition result of the pathological image to be recognized is described. Since the trained deep neural network model has the function of automatic recognition of pathological images, the pathological image can be input into the deep neural network model, and the area that may have malignant lesions on the pathological image can be identified. For the classification of benign and malignant pathological pictures, the whole process saves time and effort, which not only improves the efficiency of pathological picture recognition, but also does not depend on the personal experience of doctors. The recognition results are obtained by fusion, which greatly improves the accuracy of pathological image recognition.

Hereinafter, with reference to FIG. 2 , each step of the method for identifying a pathological picture in the embodiment of the present invention will be described in detail as follows.

First, the process of pre-training to generate multiple different types of deep neural network models is introduced.

In one embodiment, the multiple different types of deep neural network models are pre-trained and generated according to the following method:

Obtaining sample data, the sample data includes positive samples and negative samples, the positive samples are pathological pictures of malignant lesions, the negative samples are pathological pictures of normal or benign lesions, and the lesion area is marked on the pathological pictures of malignant lesions;

dividing the sample data into a training set, a test set and a validation set;

Using the training set to train a plurality of different types of deep neural network models in the first set;

Use the test set to test multiple different types of deep neural network models in the trained first set;

According to the test result, a plurality of deep neural network models are selected from the plurality of different types of deep neural network models in the first set as the second set;

The verification set is used to perform fusion verification on a plurality of different types of deep neural network models in the second set, to obtain a plurality of different types of deep neural network models generated by the pre-training.

In the specific implementation, the process of acquiring training data (sample data), the storage and labeling of digital pathological pictures of gastric lymph node cancer metastasis are introduced first:

According to the standard operation in medical clinic, the lymph node tissue of patients with gastric cancer was first stained with HE staining to make pathological sections. Then use a digital pathology scanner to scan 40 times to obtain a digital pathological picture, store it on a disk medium, and obtain a picture in scn format. The physical size of an scn image stored on disk ranges from 0.4GB to 8GB, with pixels on the order of 10^9–10^10. Pathologists with good professional ability use ImageScope software to annotate digital pathology pictures and delineate cancer areas. The sketched data is saved as an xml tag file in a specific format for the program to read.

Secondly, after the sample data is obtained, the preprocessing process of the sample data is introduced to obtain positive/negative patches (positive/negative samples):

In one embodiment, after obtaining the sample data, the sample data is further preprocessed as follows:

For each negative sample, the following preprocessing is performed:

Convert normal or benign lesions pathological pictures from RGB color format to HSV color format;

Adjust the saturation of normal or benign lesions pathological pictures in HSV color format to a preset threshold;

In the foreground cell area of the normal or benign pathological image after the saturation is adjusted to the preset threshold, extract a plurality of block patch images of preset pixel size;

Determine the first proportion of the foreground contained in the patch picture with the preset pixel size to the entire patch picture, and delete the patch picture with the preset pixel size when the first proportion is less than the first preset proportion value;

For each positive sample, the following preprocessing is performed:

Extracting patch images with multiple preset steps in the lesion area marked on the pathological image of the malignant lesion;

Judging the second proportion of the foreground contained in the patch picture of the preset step size to the entire patch picture, when the second proportion is less than the second preset proportion value, delete the patch picture of the preset step size; Let the foreground contained in the patch image of the stride size be the lesion area.

During specific implementation, the above-mentioned first preset ratio value and second preset ratio value can be flexibly set according to actual work needs, and the two can be the same or different, such as 0.85 mentioned below. The above-mentioned preset pixel size may be 224*224 pixels mentioned below, and of course may be 112*112, 128*128, 256*256, 512*512 pixels or similar sizes. The above-mentioned preset step size can be 112*112 mentioned below, and of course it can also be 128*128 or a similar size.

During the specific implementation, the number of pixels of digital pathological pictures is large, which will lead to the problem of insufficient memory during the subsequent construction and training of the model. In order to solve this problem, the present invention adopts the method of patch-level classification to solve the limitation of insufficient hardware memory. Therefore, this module divides the digital pathological picture into patches of 224*224 pixels. The specific methods are as follows:

For samples whose entire image is negative, first convert the image from RGB color format to HSV color format, and then determine an appropriate threshold at the saturation H layer to distinguish the foreground cells of the image from the blank spaces in the background without cells . In the foreground cell area, from left to right, from top to bottom, take 224*224 pixel patch images without overlapping, to ensure that each foreground area has a corresponding patch. Then determine the proportion of the foreground contained in the patch to the entire patch. If the proportion is less than 0.85, it means that the patch contains a lot of background, so delete this patch. The patch left at the end is the processed data of this pathological image, which is used as the input of the model in the next module. Repeat this for each negative pathology image sample.

For positive pathological image samples, use the program to parse the xml tag file, and when reading the tag, attention should be paid to distinguish between different closed areas and the coordinate points describing the area. At this time, the inside of the sketched area is used as the foreground, and the other parts are used as the background. Since the area of the positive area is generally smaller than the area on the negative image, in order to obtain more positive patches to balance the number of negative/positive samples, in the positive area of the positive pathological image, the method of overlapping 1/2 is adopted, taking A patch of 224*224 pixels, that is, with a step size of 112*112, in the positive area, from left to right, from top to bottom, extract the patches in turn. At the edge of the positive area, determine the proportion of the patch that contains the foreground, and delete the patch if the proportion is less than 0.85. Repeat this operation for each positive pathological image sample to obtain the data required in the next module.

Next, the process of training and generating the multiple different types of deep neural network models after preprocessing the sample data is introduced.

(1) Divide the training/test data and determine 6 models to be trained:

In one embodiment, the multiple different types of deep neural network models in the first set include: inception v3 model, resnet18 model, resnet34 model, resnet50 model, VGG16 model, and VGG19 model.

During specific implementation, the present invention uses 80% of the obtained positive and negative sample patches as a training set and 20% as a test set, and the test set is used to continuously revise the model during the model training process to make the model performance reach an ideal level. Through the investigation of classification models, combined with the characteristics of this data, the present invention selects 6 models (a plurality of different types of deep neural network models in the first set) to train on the same training set, and the selected models are: inception v3 , resnet18, resnet34, resnet50, VGG16 and VGG19. The selected models have been widely used in academic datasets and real-world classification tasks, however, the inventors selected the above 6 models based on extensive experiments. Each model has different characteristics: inceptionv3 mixes different network substructures to improve the versatility of the model; resnet18 has a shallow network, which is better for situations where the amount of data is small or the data difference is not large; resnet34 balances the model description ability and Training speed; compared with the network structure of the same number of layers, resnet50 has a relatively fast training speed; VGG16 and VGG19 models have more parameters, occupy more computing resources, and have rich expression capabilities. Different models have different performances in the data set of the present invention. Through the experimental verification of various models, the model with better effect is further optimized to improve the overall performance index. The six models identified in this module provide prerequisites for subsequent model selection and result fusion, and lay a foundation for subsequent improvement of the recognition accuracy.

During specific implementation, the number of models selected for the model may be other numbers, or may be other types of models.

(2) Model training and testing iterations:

In the example of the present invention, the implementation of the model refers to relevant papers and existing implementation schemes, and selects initial network parameters according to the characteristics of patch data: learning rate and its trend planning, patch size, number of predicted classifications, gradient descent optimization algorithm, initialization Weights. Train and test each model, analyze the current performance of the model according to the test results, and adjust the training parameters to improve the performance of the model. After 5-10 iterations of training and testing, the ability of each model was maximized to obtain the best performance of the model in this dataset.

In one embodiment, using the training set to train a plurality of different types of deep neural network models in the first set includes: performing parallel training on a plurality of different types of deep neural network models in the first set, Among them, each model is trained using 2 graphics processing unit GPUs.

During specific implementation, the embodiment of the present invention utilizes multiple GPU computing resources to speed up the training speed, thereby shortening the model training and testing iteration cycle as a whole. In multi-GPU implementation, it is necessary to fuse the gradients calculated by different GPUs, and the fusion method adopted in the present invention is summation or average value. On this data set, the effect of summation is better, so the present invention selects the method of calculating the gradient summation by multiple GPUs. After a lot of experiments by the inventor, the scheme is proposed: the number of GPUs used for training a single model is determined. For each model, using 2 GPUs can reduce the training time by 80%. If more GPUs are used, due to the synchronization of multiple GPUs The cost is higher, and its acceleration effect is not significantly improved. During the training process, each model uses 2 GPUs for accelerated training, and 6 models are trained at the same time, making full use of the computing resources of 12 GPUs.

(3) The detailed process of parallel training of multiple models is described below.

In one embodiment, using 2 graphics processing units (GPUs) to train each model may include:

Divide the training set data into equally non-overlapping first training data streams and second training data streams;

Obtaining a pathological picture with a preset amount of data in the first training data stream and inputting it to the current model, the model obtains a first loss function value through calculation, and the loss function calculates a partial derivative for each variable to obtain the first gradient value of the variable;

Obtaining a pathological picture with a preset amount of data in the second training data stream and inputting it to the current model, the model obtains a second loss function value through calculation, and the loss function calculates a partial derivative for each variable to obtain the second gradient value of the variable;

The CPU waits for the first GPU (GPU1) and the second GPU (GPU2) to complete the calculation of the gradient values, sums the two gradient values, and then updates the corresponding variables with the obtained gradient values to obtain the new value of the variable, and the CPU stores the updated variable. The value is passed to the first GPU (GPU1) and the second GPU (GPU2), covering the original model (such as the VGG16 model shown in Figure 3) variable values in the first GPU (GPU1) and the second GPU (GPU2), until Training is complete.

In the specific implementation, since the calculation of each model during training is independent of each other, the process of training the VGG16 model on two GPUs is taken as an example. The process is shown in Figure 3:

First, the training data is equally divided into two non-overlapping data streams, namely training data stream 1 (first training data stream) and training data stream 2 (second training data stream). On GPU1, take a fixed number of patches in data stream 1 and input them into the model. In this example, this number is 64 patches. The model obtains the first loss function value by calculating the loss function value, and the loss function calculates the partial derivative of each variable to obtain the gradient value 1 (first gradient value) of the variable. A similar operation is performed on GPU2 to obtain a variable gradient value of 2 (the second gradient value). After that, the control is handed over to the CPU, and the CPU waits for GPU1 and GPU2 to complete the calculation of the gradient value, sums the two gradient values, and then updates the corresponding variable with the obtained gradient value to obtain the new value of the variable. The CPU synchronizes the model variables on GPU1 and GPU2 each time: the CPU transfers the updated variable values to GPU1 and GPU2, overwriting the original VGG16 model variable values in GPU1 and GPU2, so that the model variable values in GPU1 and GPU2 are consistent. . Then the model re-reads the data from the data stream, calculates the loss function, calculates the variable gradient value, and so on. The above mechanism ensures that the data in data streams 1 and 2 will be used up at the same time. At this time, the data stream is expanded again with training data. Similarly, data stream 1 and data stream 2 do not overlap each other and have the same amount of data.

Since the CPU will wait for GPU1 and GPU2 to complete the calculation of the gradient values before performing the next gradient value summation operation, the waiting process causes a waste of time, so using two GPUs cannot increase the training speed by 100%. In the present invention, using 2 GPUs increases the training speed by 80%.

(4) The following describes the prediction process of the three selected models after training multi-model fusion on the validation set.

In one example, the plurality of different types of deep neural network models in the second set include: resnet34 model, VGG16 model and VGG19 model

In the specific implementation, the performance of 6 models is compared, and the resnet18 model, resnet50 model and inceptionv3 model with poor experimental effect are excluded. The present invention selects the resnet34 model with low false negative rate, VGG16 model with low false positive rate and VGG19 model. The fusion method of the results of these three models is: for the case where all three models predict the patch to be negative, the final prediction result of this patch is negative; for the case that all three models predict the patch to be positive, the final prediction result of this patch is positive; for When the prediction results of the three models for the same patch are inconsistent, the final prediction result of this patch is negative.

Use the above method to predict the data of the validation set, and map the patch prediction results to the digital pathological pictures, mark the position of the positive patches predicted by the model on the pathological pictures, and visualize them, so as to prepare for the analysis of the results of the next module. .

Second, after obtaining a number of different types of deep neural network models that have been pre-trained, the process of using the model to make predictions is introduced.

After the step of acquiring the pathological picture to be recognized, for example, the above process of preprocessing the sample data can also be performed, and after the preprocessing of the pathological image to be recognized, the efficiency and accuracy of the recognition can be further improved.

Third, it introduces the process of fusing the preliminary recognition results obtained by multiple different types of deep neural network models. For the process of fusing, please refer to the above-mentioned multi-model training and fusion of the selected three models on the validation set. forecasting process.

During specific implementation, the beneficial technical effects obtained according to a large number of experimental results are: after 3-5 iterations according to the actual situation, the embodiment of the present invention controls the patch level false positives to 0.06% while ensuring that the false negative rate is below 0.3% , the overall patch level accuracy rate is above 99.8%.

Fourth, the steps of model optimization are introduced. In the subsequent use of the model, the process of model optimization may also be included to further improve the accuracy of recognizing pathological pictures. In one embodiment, the process of model optimization may include:

When it is judged that the false positives of the pathological pictures to be identified are high according to the recognition results, the negative samples of the pathological pictures of this type are added to the pathological picture sample database;

When it is judged that the false negative of the pathological image to be recognized is high according to the recognition result, the positive samples of the pathological image of this type are added to the pathological image sample database;

According to the supplemented pathological picture sample database, performing optimization training on multiple different types of deep neural network models generated by the pre-training, to obtain updated multiple different types of deep neural network models;

Input the pathological pictures to be identified into multiple different types of deep neural network models generated by pre-training, and identify the pathological pictures to be identified, including:

The pathological pictures to be identified are input into the updated multiple different types of deep neural network models, and the pathological pictures to be identified are identified.

In the specific implementation, the data characteristics have a non-negligible impact on the model effect. For the above model prediction errors, it can be divided into false positives and false negatives. From the perspective of data analysis, find the commonality of the wrongly predicted data, that is, the error occurs on certain types of patches. If the false positives are high on a certain type of patch, the corresponding type of patch appears less in the negative samples, and this type of pathological pictures with special negative labels needs to be supplemented; if the false negatives are high, it means that the model There is no good study of this type of positive patches, and it is necessary to supplement the delineation of this type of positive pathological pictures. To classify the patches that the model predicts incorrectly, it is necessary to combine the expertise of pathologists and algorithm engineers to jointly analyze and determine what kind of data needs to be supplemented. In addition, due to the diverse morphology of cancer cells, supplementing more diverse pathological pictures can significantly improve the model effect.

Based on the same inventive concept, an embodiment of the present invention also provides an apparatus for identifying a pathological picture, as described in the following embodiments. Since the principle of the device for solving the problem is similar to the method for recognizing pathological pictures, the implementation of the device can refer to the implementation of the method for recognizing pathological pictures, and the repetition will not be repeated.

FIG. 4 is a schematic diagram of an apparatus for identifying a pathological picture in an embodiment of the present invention. As shown in FIG. 4 , the apparatus may include:

Obtaining unit 02, for obtaining pathological pictures to be identified;

The identification unit 04 is used to input the pathological pictures to be identified into multiple different types of deep neural network models generated by pre-training, identify the pathological pictures to be identified, and obtain a preliminary identification result for each type of deep neural network models; Different types of deep neural network models are pre-trained and generated based on multiple pathological image samples;

The fusion unit 06 is configured to fuse the preliminary recognition results obtained by a plurality of different types of deep neural network models to obtain the final recognition result of the pathological picture to be recognized.

In one embodiment, the apparatus for identifying a pathological picture may further include: a training unit, configured to pre-train and generate the multiple different types of deep neural network models according to the following method:

Obtaining sample data, the sample data includes positive samples and negative samples, the positive samples are pathological pictures of malignant lesions, the negative samples are pathological pictures of normal or benign lesions, and the lesion area is marked on the pathological pictures of malignant lesions;

dividing the sample data into a training set, a test set and a validation set;

Using the training set to train a plurality of different types of deep neural network models in the first set;

Use the test set to test multiple different types of deep neural network models in the trained first set;

According to the test result, a plurality of deep neural network models are selected from the plurality of different types of deep neural network models in the first set as the second set;

The verification set is used to perform fusion verification on a plurality of different types of deep neural network models in the second set, to obtain a plurality of different types of deep neural network models generated by the pre-training.

In one embodiment, the above-mentioned pathological image identification device further includes a preprocessing unit, and the preprocessing unit is used for:

For each negative sample, the following preprocessing is performed:

Convert normal or benign lesions pathological pictures from RGB color format to HSV color format;

Adjust the saturation of normal or benign lesions pathological pictures in HSV color format to a preset threshold;

In the foreground cell area of the normal or benign pathological image after the saturation is adjusted to the preset threshold, extract a plurality of block patch images of preset pixel size;

Determine the first proportion of the foreground contained in the patch picture with the preset pixel size to the entire patch picture, and delete the patch picture with the preset pixel size when the first proportion is less than the first preset proportion value;

For each positive sample, the following preprocessing is performed:

Extracting patch images with multiple preset steps in the lesion area marked on the pathological image of the malignant lesion;

Judging the second proportion of the foreground contained in the patch picture of the preset step size to the entire patch picture, when the second proportion is less than the second preset proportion value, delete the patch picture of the preset step size; Let the foreground contained in the patch image of the stride size be the lesion area.

In one embodiment, the multiple different types of deep neural network models in the first set include: inceptionv3 model, resnet18 model, resnet34 model, resnet50 model, VGG16 model and VGG19 model;

The multiple different types of deep neural network models in the second set include: resnet34 model, VGG16 model and VGG19 model.

In one embodiment, using the training set to train multiple different types of deep neural network models in the first set includes: performing parallel training on multiple different types of deep neural network models in the first set, Among them, each model is trained using 2 graphics processing unit GPUs.

In one embodiment, using 2 graphics processing units (GPUs) to train each model may include:

Divide the training set data into equally non-overlapping first training data streams and second training data streams;

Obtaining a pathological picture with a preset amount of data in the first training data stream and inputting it to the current model, the model obtains a first loss function value through calculation, and the loss function calculates a partial derivative for each variable to obtain the first gradient value of the variable;

Obtaining a pathological picture with a preset amount of data in the second training data stream and inputting it to the current model, the model obtains a second loss function value through calculation, and the loss function calculates a partial derivative for each variable to obtain the second gradient value of the variable;

The CPU waits for the first GPU and the second GPU to complete the calculation of the gradient values, sums the two gradient values, and then updates the corresponding variables with the obtained gradient values to obtain the new value of the variable, and the CPU passes the updated variable value to the first GPU. and the second GPU, overwriting the original model variable values in the first GPU and the second GPU until the training is completed.

In one embodiment, the above-mentioned pathological picture identification device may further include an optimization unit, and the optimization unit is used for:

When it is judged that the false positives of the pathological pictures to be identified are high according to the recognition results, the negative samples of the pathological pictures of this type are added to the pathological picture sample database;

When it is judged that the false negative of the pathological image to be recognized is high according to the recognition result, the positive samples of the pathological image of this type are added to the pathological image sample database;

According to the supplemented pathological picture sample database, performing optimization training on multiple different types of deep neural network models generated by the pre-training, to obtain updated multiple different types of deep neural network models;

Input the pathological pictures to be identified into multiple different types of deep neural network models generated by pre-training, and identify the pathological pictures to be identified, including:

The pathological pictures to be identified are input into the updated multiple different types of deep neural network models, and the pathological pictures to be identified are identified.

An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the above pathological picture recognition method when the processor executes the computer program.

An embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program for executing the above method for identifying a pathological picture.

To sum up, today, artificial intelligence technology is on the rise again. With the development of big data, new algorithms, and cloud computing, it has become possible to train deep neural network models. Artificial intelligence will have a profound impact on various industries. Artificial intelligence + medical care is of course also among them. The embodiment of the present invention utilizes the advantages of artificial intelligence in picture classification, and combines with traditional medical treatment, so that it can correctly classify pathological pictures.

The technical solutions provided by the embodiments of the present invention can be applied not only to the identification of pathological pictures of gastric lymph node cancer metastasis, but also to the identification of pathological pictures of other cancers.

The beneficial technical effects achieved by the technical solutions provided in the embodiments of the present invention are:

①In the training phase, 6 models are selected, 3 models are selected, and the results are fused. Select 6 models for training, fully consider the characteristics of each model, and select 3 models according to the false positive rate and false negative rate. The final result is obtained by intersecting the positive prediction results of the three models. The fusion of multiple models effectively exploits the advantages of each model, thereby improving the overall performance.

② During training, a multi-GPU acceleration implementation is adopted. When using multiple GPUs in parallel, the gradient data is allocated to different GPUs for calculation, and then the results of the multiple GPUs are synchronized. The synchronization method in the present invention adopts the summation method. The use of multi-GPU computing improves the training speed, thereby reducing the time to iterate the model as a whole.

③ The analysis model results stage focuses on combining the professional knowledge of doctors and is in line with the reality. In practical applications, it is not only necessary to improve the accuracy of the model, but also to iterate the model by supplementing new data from a medical point of view, making the model more generalizable and more practical. In the present invention, the professional knowledge of doctors and algorithm knowledge are fully combined, and the direction of model iteration is closer to reality.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above-mentioned specific embodiments are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (7)

1. A pathological picture recognition device, comprising:

the acquiring unit is used for acquiring a pathological picture to be identified;

the recognition unit is used for inputting the pathological pictures to be recognized into a plurality of different types of deep neural network models generated by pre-training, recognizing the pathological pictures to be recognized, and obtaining a primary recognition result by each type of deep neural network model; the deep neural network models of different types are generated by pre-training according to a plurality of pathological image samples;

the fusion unit is used for fusing the primary recognition results obtained by the deep neural network models of different types to obtain the final recognition result of the pathological picture to be recognized;

the identification device of the pathological picture further comprises: the training unit is used for generating the plurality of deep neural network models of different types through pre-training according to the following method:

obtaining sample data, wherein the sample data comprises a positive sample and a negative sample, the positive sample is a malignant lesion pathological picture, the negative sample is a normal or benign lesion pathological picture, and a lesion area is marked on the malignant lesion pathological picture;

dividing the sample data into a training set, a test set and a verification set;

training a plurality of different types of deep neural network models in a first set by using the training set;

testing a plurality of different types of deep neural network models in the trained first set by using the test set;

according to the test result, screening out a plurality of deep neural network models from a plurality of different types of deep neural network models in the first set to serve as a second set;

carrying out fusion verification on a plurality of different types of deep neural network models in a second set by using the verification set to obtain a plurality of different types of deep neural network models generated by pre-training;

performing fusion verification on a plurality of different types of deep neural network models in the second set by using the verification set, wherein the fusion verification comprises the following steps: for the condition that the predictions of a plurality of different types of deep neural network models are negative, the final prediction result is negative; for the condition that the predictions of a plurality of different types of deep neural network models are positive, the final prediction result is positive; for the condition that the prediction results of a plurality of different types of deep neural network models are inconsistent, the final prediction result is negative;

training a plurality of different types of deep neural network models in a first set using the training set, including: training a plurality of different types of deep neural network models in the first set in parallel, wherein each model is trained using 2 Graphics Processing Units (GPUs);

training each model using 2 Graphics Processing Units (GPUs) includes:

averagely dividing training set data into a first training data stream and a second training data stream which are not overlapped with each other;

acquiring pathological pictures with preset data volume from a first training data stream, inputting the pathological pictures into a current model, calculating the model to obtain a first loss function value, and calculating a partial derivative of each variable by the loss function to obtain a first gradient value of the variable;

acquiring pathological pictures with preset data volume from a second training data stream, inputting the pathological pictures into a current model, calculating the model to obtain a second loss function value, and calculating a partial derivative of each variable by the loss function to obtain a second gradient value of the variable;

the CPU waits for the first GPU and the second GPU to finish calculating the gradient values, sums the gradient values, updates corresponding variables by using the obtained gradient values to obtain new values of the variables, transmits the updated variable values to the first GPU and the second GPU, covers original model variable values in the first GPU and the second GPU until training is finished;

the identification method of the pathological picture is applied to identification of the gastric lymph node cancer metastasis pathological picture;

the identification device of the pathological picture further comprises a preprocessing unit, wherein the preprocessing unit is used for:

for each negative sample, the following pre-treatments were performed:

converting the normal or benign pathological picture from an RGB color format into an HSV color format;

adjusting the saturation of the normal or benign pathological picture with the HSV color format to a preset threshold;

extracting a plurality of patch pictures with preset pixel sizes from the foreground cell area of the normal or benign pathological picture after the saturation is adjusted to a preset threshold value;

judging a first proportion of a foreground contained in a patch picture with a preset pixel size to the whole patch picture, and deleting the patch picture with the preset pixel size when the first proportion is smaller than a first preset proportion value;

for each positive sample, the following pre-treatments were performed:

extracting a plurality of patch pictures with preset step lengths from a lesion area marked on a pathological picture of a malignant lesion;

judging a second proportion of the foreground contained in the patch picture with the preset step length to the whole patch picture, and deleting the patch picture with the preset step length when the second proportion is smaller than a second preset proportion value; the foreground contained in the patch picture with the preset step length is a lesion area.

2. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements a method for identifying a pathological picture comprising:

acquiring a pathological picture to be identified;

inputting the pathological picture to be recognized into a plurality of different types of deep neural network models generated by pre-training, recognizing the pathological picture to be recognized, and obtaining a preliminary recognition result by each type of deep neural network model; the deep neural network models of different types are generated by pre-training according to a plurality of pathological image samples;

fusing the primary recognition results obtained by a plurality of different types of deep neural network models to obtain a final recognition result of the pathological picture to be recognized;

pre-training and generating the plurality of different types of deep neural network models according to the following method:

obtaining sample data, wherein the sample data comprises a positive sample and a negative sample, the positive sample is a malignant lesion pathological picture, the negative sample is a normal or benign lesion pathological picture, and a lesion area is marked on the malignant lesion pathological picture;

dividing the sample data into a training set, a test set and a verification set;

training a plurality of different types of deep neural network models in a first set by using the training set;

testing a plurality of different types of deep neural network models in the trained first set by using the test set;

according to the test result, screening out a plurality of deep neural network models from a plurality of different types of deep neural network models in the first set to serve as a second set;

carrying out fusion verification on a plurality of different types of deep neural network models in a second set by using the verification set to obtain a plurality of different types of deep neural network models generated by pre-training;

performing fusion verification on a plurality of different types of deep neural network models in the second set by using the verification set, wherein the fusion verification comprises the following steps: for the condition that the predictions of a plurality of different types of deep neural network models are negative, the final prediction result is negative; for the condition that the predictions of a plurality of different types of deep neural network models are positive, the final prediction result is positive; for the condition that the prediction results of a plurality of different types of deep neural network models are inconsistent, the final prediction result is negative;

training a plurality of different types of deep neural network models in a first set using the training set, including: training a plurality of different types of deep neural network models in the first set in parallel, wherein each model is trained using 2 Graphics Processing Units (GPUs);

training each model using 2 Graphics Processing Units (GPUs) includes:

averagely dividing training set data into a first training data stream and a second training data stream which are not overlapped with each other;

acquiring pathological pictures with preset data volume from a first training data stream, inputting the pathological pictures into a current model, calculating the model to obtain a first loss function value, and calculating a partial derivative of each variable by the loss function to obtain a first gradient value of the variable;

acquiring pathological pictures with preset data volume from a second training data stream, inputting the pathological pictures into a current model, calculating the model to obtain a second loss function value, and calculating a partial derivative of each variable by the loss function to obtain a second gradient value of the variable;

the CPU waits for the first GPU and the second GPU to finish calculating the gradient values, sums the gradient values, updates corresponding variables by using the obtained gradient values to obtain new values of the variables, transmits the updated variable values to the first GPU and the second GPU, covers original model variable values in the first GPU and the second GPU until training is finished;

the identification method of the pathological picture is applied to identification of the gastric lymph node cancer metastasis pathological picture;

after obtaining the sample data, the sample data is further preprocessed according to the following mode:

for each negative sample, the following pre-treatments were performed:

converting the normal or benign pathological picture from an RGB color format into an HSV color format;

adjusting the saturation of the normal or benign pathological picture with the HSV color format to a preset threshold;

extracting a plurality of patch pictures with preset pixel sizes from the foreground cell area of the normal or benign pathological picture after the saturation is adjusted to a preset threshold value;

judging a first proportion of a foreground contained in a patch picture with a preset pixel size to the whole patch picture, and deleting the patch picture with the preset pixel size when the first proportion is smaller than a first preset proportion value;

for each positive sample, the following pre-treatments were performed:

extracting a plurality of patch pictures with preset step lengths from a lesion area marked on a pathological picture of a malignant lesion;

judging a second proportion of the foreground contained in the patch picture with the preset step length to the whole patch picture, and deleting the patch picture with the preset step length when the second proportion is smaller than a second preset proportion value; the foreground contained in the patch picture with the preset step length is a lesion area.

3. The computer device of claim 2, wherein the plurality of different types of deep neural network models in the first set comprises: inception v3 model, resnet18 model, resnet34 model, resnet50 model, VGG16 model, and VGG19 model;

the plurality of different types of deep neural network models in the second set includes: the resnet34 model, the VGG16 model, and the VGG19 model.

4. The computer device of claim 2, wherein the identification method of the pathology picture further comprises:

when the false positive of the pathological picture to be identified is judged to be higher according to the identification result, the negative sample of the pathological picture of the type is supplemented to a pathological picture sample database;

when the false negative of the pathological picture to be identified is judged to be higher according to the identification result, supplementing the positive sample of the pathological picture of the type into a pathological picture sample database;

performing optimization training on a plurality of different types of deep neural network models generated by the pre-training according to the supplemented pathological picture sample database to obtain a plurality of updated different types of deep neural network models;

inputting the pathological pictures to be recognized into a plurality of different types of deep neural network models generated by pre-training, and recognizing the pathological pictures to be recognized, wherein the method comprises the following steps:

and inputting the pathological picture to be recognized into the updated deep neural network models of different types, and recognizing the pathological picture to be recognized.

5. A computer-readable storage medium characterized by storing a computer program that executes an identification method of a pathology image:

acquiring a pathological picture to be identified;

inputting the pathological picture to be recognized into a plurality of different types of deep neural network models generated by pre-training, recognizing the pathological picture to be recognized, and obtaining a preliminary recognition result by each type of deep neural network model; the deep neural network models of different types are generated by pre-training according to a plurality of pathological image samples;

fusing the primary recognition results obtained by a plurality of different types of deep neural network models to obtain a final recognition result of the pathological picture to be recognized;

pre-training and generating the plurality of different types of deep neural network models according to the following method:

obtaining sample data, wherein the sample data comprises a positive sample and a negative sample, the positive sample is a malignant lesion pathological picture, the negative sample is a normal or benign lesion pathological picture, and a lesion area is marked on the malignant lesion pathological picture;

dividing the sample data into a training set, a test set and a verification set;

training a plurality of different types of deep neural network models in a first set by using the training set;

testing a plurality of different types of deep neural network models in the trained first set by using the test set;

according to the test result, screening out a plurality of deep neural network models from a plurality of different types of deep neural network models in the first set to serve as a second set;

carrying out fusion verification on a plurality of different types of deep neural network models in a second set by using the verification set to obtain a plurality of different types of deep neural network models generated by pre-training;

performing fusion verification on a plurality of different types of deep neural network models in the second set by using the verification set, wherein the fusion verification comprises the following steps: for the condition that the predictions of a plurality of different types of deep neural network models are negative, the final prediction result is negative; for the condition that the predictions of a plurality of different types of deep neural network models are positive, the final prediction result is positive; for the condition that the prediction results of a plurality of different types of deep neural network models are inconsistent, the final prediction result is negative;

training a plurality of different types of deep neural network models in a first set using the training set, including: training a plurality of different types of deep neural network models in the first set in parallel, wherein each model is trained using 2 Graphics Processing Units (GPUs);

training each model using 2 Graphics Processing Units (GPUs) includes:

averagely dividing training set data into a first training data stream and a second training data stream which are not overlapped with each other;

acquiring pathological pictures with preset data volume from a first training data stream, inputting the pathological pictures into a current model, calculating the model to obtain a first loss function value, and calculating a partial derivative of each variable by the loss function to obtain a first gradient value of the variable;

acquiring pathological pictures with preset data volume from a second training data stream, inputting the pathological pictures into a current model, calculating the model to obtain a second loss function value, and calculating a partial derivative of each variable by the loss function to obtain a second gradient value of the variable;

the CPU waits for the first GPU and the second GPU to finish calculating the gradient values, sums the gradient values, updates corresponding variables by using the obtained gradient values to obtain new values of the variables, transmits the updated variable values to the first GPU and the second GPU, covers original model variable values in the first GPU and the second GPU until training is finished;

the identification method of the pathological picture is applied to identification of the gastric lymph node cancer metastasis pathological picture;

after obtaining the sample data, the sample data is further preprocessed according to the following mode:

for each negative sample, the following pre-treatments were performed:

converting the normal or benign pathological picture from an RGB color format into an HSV color format;

adjusting the saturation of the normal or benign pathological picture with the HSV color format to a preset threshold;

extracting a plurality of patch pictures with preset pixel sizes from the foreground cell area of the normal or benign pathological picture after the saturation is adjusted to a preset threshold value;

judging a first proportion of a foreground contained in a patch picture with a preset pixel size to the whole patch picture, and deleting the patch picture with the preset pixel size when the first proportion is smaller than a first preset proportion value;

for each positive sample, the following pre-treatments were performed:

extracting a plurality of patch pictures with preset step lengths from a lesion area marked on a pathological picture of a malignant lesion;

judging a second proportion of the foreground contained in the patch picture with the preset step length to the whole patch picture, and deleting the patch picture with the preset step length when the second proportion is smaller than a second preset proportion value; the foreground contained in the patch picture with the preset step length is a lesion area.

6. The computer-readable storage medium of claim 5, wherein the plurality of different types of deep neural network models in the first set comprises: inception v3 model, resnet18 model, resnet34 model, resnet50 model, VGG16 model, and VGG19 model;

the plurality of different types of deep neural network models in the second set includes: the resnet34 model, the VGG16 model, and the VGG19 model.

7. The computer-readable storage medium of claim 5, wherein the identification method of the pathology picture further comprises:

when the false positive of the pathological picture to be identified is judged to be higher according to the identification result, the negative sample of the pathological picture of the type is supplemented to a pathological picture sample database;

when the false negative of the pathological picture to be identified is judged to be higher according to the identification result, supplementing the positive sample of the pathological picture of the type into a pathological picture sample database;

performing optimization training on a plurality of different types of deep neural network models generated by the pre-training according to the supplemented pathological picture sample database to obtain a plurality of updated different types of deep neural network models;

inputting the pathological pictures to be recognized into a plurality of different types of deep neural network models generated by pre-training, and recognizing the pathological pictures to be recognized, wherein the method comprises the following steps:

and inputting the pathological picture to be recognized into the updated deep neural network models of different types, and recognizing the pathological picture to be recognized.

Images