CN114387596A - Automatic interpretation system for cytopathology smear - Google Patents
Automatic interpretation system for cytopathology smear Download PDFInfo
- Publication number
- CN114387596A CN114387596A CN202111626061.5A CN202111626061A CN114387596A CN 114387596 A CN114387596 A CN 114387596A CN 202111626061 A CN202111626061 A CN 202111626061A CN 114387596 A CN114387596 A CN 114387596A
- Authority
- CN
- China
- Prior art keywords
- image
- module
- images
- smear
- interpretation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/0052—Optical details of the image generation
- G02B21/006—Optical details of the image generation focusing arrangements; selection of the plane to be imaged
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Optics & Photonics (AREA)
- Investigating Or Analysing Biological Materials (AREA)
Abstract
The invention discloses an automatic interpretation system for cytopathology smear, comprising: an imaging module; an image acquisition module; an image storage and management module; an image preprocessing module; the intelligent interpretation module receives the image output by the image preprocessing module, performs prediction classification on the image on a normal sample and a focus sample, and further performs prediction interpretation on the focus type when the image is the focus sample; and a report composition module which automatically generates an interpretation conclusion text of the sample corresponding to the image. The invention successfully applies the artificial intelligent auxiliary diagnosis technology to the rapid staining cytopathology, can obviously improve the accuracy and consistency of diagnosis and reduce the workload of cytopathologists; the invention solves the uncertain factors introduced by manual sampling by improving the prior convolutional neural network and utilizing the characteristic of a multi-channel attention mechanism, can realize a full scene and multi-classification task with high accuracy, and finally can improve the accuracy of classification interpretation results.
Description
Technical Field
The invention relates to the field of artificial intelligence auxiliary diagnosis and pathology, in particular to an automatic interpretation system for cytopathology smears.
Background
The lung cancer is a disease with the highest morbidity and mortality in the whole world, the disease also has a tendency of rising year by year in China, the disease seriously harms the life health and safety of people, the initial onset of the lung cancer is 12.22/10 ten thousand in 2017 in the whole world, the mortality is 19.88/10 ten thousand, the data of the world cancer foundation show that the incidence of the lung cancer is 35.1/10 ten thousand in 2018 in China, the lung cancer is ranked 16 th in the whole world, and the data published by the national cancer center in 2019 in China show that the morbidity is 57.6/10 ten thousand and the mortality is 45.87/10 ten thousand. Since the early stage of the disease has no special symptoms, the lung cancer patients up to 2/3 are in the middle-late stage when hospitalized, and the patients lose the evidence of surgical treatment and the condition of histopathological diagnosis of surgical excision specimens, the pathological diagnosis of the patients is often determined by the needle aspiration of the lung tumor. Interventional therapy is the main treatment means of patients of the kind, pathological diagnosis directly influences the selection of treatment schemes of the patients, the current lung cancer treatment mainly adopts individualized and accurate treatment, molecular pathological detection is carried out by utilizing cytological specimens or coarse needle pathological specimens, the diagnosis and treatment direction of the patients with the lung cancer at the middle and late stages in the future is the diagnosis and treatment direction of the patients with the lung cancer, and the field judgment is carried out by adopting a rapid cell pathology technology (ROSE technology), so that the diagnosis and treatment method is an internationally developed diagnosis and treatment method, can provide information such as the sufficiency, the initial diagnosis and the next diagnosis and treatment direction of a puncture specimen of an interventional doctor in one interventional diagnosis and treatment process, and can provide the basis for simultaneously carrying out related tumor interventional treatment. Diff-quik staining is the first technique for field judgment because of its rapidity and simplicity. Compared with the traditional method that a clinical interventional doctor judges a specimen by experience, the rapid cytopathology technology can effectively reduce the occurrence probability that a patient needs repeated puncture or misdiagnosis. diff-quick staining is a reduction in the number of cases that cannot be diagnosed by increasing the sufficiency of the specimen. According to the data of the university of Pittsburgh medical school (watching Cai, Adebowaale J Adeniran. Rapid on-site evaluation [ M ]. Springer 2019:8), it was revealed that the mean diagnostically unding rate was about 20% in the cytopathological diagnosis without diff-quick staining, whereas the diagnostically unding rate was 2-10% with diff-quick staining, not only was the probability of re-puncture sampling reduced, but also there was no statistical difference in intraoperative complications and the like compared to the control group without diff-quick technique.
The judgment of rapid cytological diagnosis is generally completed by full-time cytopathology doctors, but the culture of qualified cytologists is long-term, and most primary hospitals do not have full-time cytopathology doctors, so that the development and popularization of interventional medicine are greatly restricted; in addition, the cytologist in China is unfamiliar with the image characteristics of the diff-quick staining, and the difference between the image of the HE staining and the image of the diff-quick staining is large, so that the mastering of the diagnosis technology also costs large manpower and material resources. Since the cytologist's judgment is more subjective, there are diagnostic differences from person to person and it takes a longer time. In large public hospitals in China, the work load of the cytopathologist is far beyond the standard that the work of eight hours per day is less than 100 cervical smears defined in European and American countries, so the work load of the cytopathologist is further increased by on-site rapid cytological judgment.
Artificial intelligence is a branch of computer science, and an important application of the artificial intelligence is to analyze acquired image data, perform specific semantic segmentation or classification identification on images, and the like, so that the development of the artificial intelligence revolutionarily changes the mode of future morphological diagnosis, and is mainly reflected in the diagnosis of histopathology. With the development of artificial intelligence technology, an auxiliary system constructed by using an artificial intelligence algorithm is applied to tissue slice pathological images, medical image images and the like of a plurality of focuses, and automatic and rapid classification diagnosis is performed on the images according to objective and quantized high-dimensional image characteristics, so that the method is a novel clinical diagnosis and treatment technology with high feasibility.
Foreign artificial intelligence has been applied mainly to thyroid gland puncture specimens, urine specimens, breast puncture specimens, hepatobiliary pancreas puncture specimens, hydrothorax specimens and lung specimens in research stages, and has not been commercialized yet in non-gynecological cytology (Landau MS, Pantanowitz L. organic interference in cytology: a review of the performance and overview of commercial landscapin [ J ]. J.Am Soc Cytopathol, 2019; 8(4):230-241.doi:10.1016/j.jasc.2019.03.003.Epub 2019Mar 25), and most of these artificial intelligence pictures are in conventional Papanicolaou staining. In 2017 Teramoto (Teramoto A, Tsukamoto T, Kiriyama Y, et al, automated Classification of Lung Cancer Types from cellular Images Using Deep consistent Neural Networks [ J ]. Biomed Res int, 2017; 2017:4067832.) and the like, adenocarcinoma, squamous cell carcinoma and small cell carcinoma were distinguished in Lung fine needle puncture liquid based cell specimens with an accuracy of 71.1% and non-small cell carcinoma and small cell carcinoma with an accuracy of 85.6%, and the study was limited in that only the main type of Lung Cancer could be distinguished and the specimen could not be distinguished for benign or malignant.
The artificial intelligence in China is in the initial stage in the field of lung cancer diagnosis, the juanjuan and the like (juanjuan, yangzhao, juqiang, and the like; a preliminary application of a cytopathology diagnosis system based on artificial intelligence [ J ]. liberation of medical college reports of military sciences, 2020,41(9):897-900.DOI:10.3969/j.issn.2095-5227.2020.09.012.) takes the lead of using artificial intelligence (artificial neural network algorithm) in the cytopathology diagnosis of lung cancer, mainly analyzes the ploidy of cancer cells according to the DNA ploidy analysis principle, and finds the cancer cells (2c is normal cells) when the ploidy of the cancer cells is more than or equal to 5 c. The lung specimens comprise 101 cases including bronchofiberscope brushes, endoscopic needle aspiration, lung lump needle aspiration, pleural effusion and ascites and the like, and preliminary results are obtained: the artificial intelligence cytopathology diagnosis is prompted to have a coincidence rate with the gold standard of 66.3%, a sensitivity of 67% and a specificity of 60%, and has a great difference with the gold standard pathological method.
Disclosure of Invention
The technical problem to be solved by the invention is to provide an automatic interpretation system for cytopathology smear aiming at the defects in the prior art.
In order to solve the technical problems, the invention adopts the technical scheme that: an automated cytological smear interpretation system comprising:
an imaging module;
the image acquisition module controls the imaging module to image the cytopathology smear so as to realize image acquisition of the cytopathology smear;
the image storage and management module is used for storing the images acquired by the image acquisition module and arranging the images to form an image sample library;
the image preprocessing module is used for preprocessing the image output by the image sample library, removing an uncolored area in the image and outputting the image;
the intelligent interpretation module receives the image output by the image preprocessing module, performs prediction classification on the image on a normal sample and a focus sample, and further performs prediction interpretation on the focus type when the image is the focus sample;
and the report writing module is used for automatically generating an interpretation conclusion text of the sample corresponding to the image according to the result of the intelligent interpretation module.
Preferably, the imaging module comprises a digital microscope, a motorized stage and an upper computer; the electric objective table has X, Y, Z triaxial movement function, the cytopathology smear is placed on the electric objective table, imaging is carried out through the digital microscope, and the obtained image is transmitted to the upper computer;
the automatic interpretation system for the cytopathology smear also comprises an electric objective table control module and an automatic focusing module, wherein the electric objective table control module is used for controlling the movement of the electric objective table; the automatic focusing module is connected with the electric objective table control module and controls the electric objective table to move so as to realize automatic focusing.
Preferably, the image acquisition module is connected with the upper computer to receive the image output by the upper computer, and the work flow of the image acquisition module is as follows:
firstly, the electric objective table is controlled to move to the position that the center of the electric objective table is overlapped with the central axis of an objective lens of the digital microscope, then automatic focusing is completed through the focusing control module, and then the focal length is fixed;
controlling the electric objective table to move, enabling an objective lens of the digital microscope to be aligned to the upper left corner of the electric objective table, then translating a unit distance from the upper left corner in the X direction each time, taking a picture to obtain an image, translating the unit distance in the Y direction when reaching the right side end point, continuing translating the unit distance in the opposite direction of the X direction, translating the unit distance each time to take a picture to obtain an image, and performing snake-shaped advance shooting until all cytopathology smears on the electric objective table are shot;
and acquiring the image of the cytopathology smear from an upper computer, and outputting the image to the image storage and management module.
Preferably, the image storage and management module acquires and numbers all images of the cytopathology smear acquired by the image acquisition module each time, wherein the numbers include the cytopathology smear serial number corresponding to the image and the position number of the image, so that an image sample library consisting of all images of a plurality of cytopathology smears is acquired after multiple acquisitions.
Preferably, the processing method of the image preprocessing module includes the steps of:
removing a gray or white non-stained blood cell area and reserving a blue-violet stained blood cell area according to RGB three-channel color values of the image acquired from the image storage and management module;
and performing binarization operation on the reserved region to enable the reserved region to become a foreground mask, taking the removed region as a background mask, performing closed processing on the foreground mask part, performing edge expansion to obtain a final foreground mask, and performing matting from the initially acquired image by using the foreground mask to obtain a preprocessed image.
The image preprocessing module can also classify and mark the images as different labels, wherein the labels at least comprise positive labels, negative labels and different pathological grading labels.
Preferably, the intelligent interpretation module comprises a two-classification model for distinguishing a normal sample from a focus sample and a multi-classification model for distinguishing a focus type, wherein the two-classification model and the multi-classification model are obtained by respectively training a first training set and a second training set based on the same classification network;
the first training set includes a number of normal sample images and a number of abnormal sample images with corresponding marks, and the second training set includes a number of abnormal sample images with different lesion type marks.
Preferably, the classification network is a ResNet network, a ResNeXt network, or a DenseNet network to which the ecSK _ Unit module is added, and the processing method of the ecSK _ Unit module includes:
after the image matrix is input into the ecSK _ Unit module, firstly, feature extraction is carried out on a convolution layer with convolution kernels of 3 × 3 and 5 × 5 convolution operations to obtain two feature matrices, and the two feature matrices are marked as a feature matrix 1 and a feature matrix 2; then, performing matrix splicing operation on the two characteristic matrixes in the channel dimension through the connection layer, enabling the spliced characteristic matrixes to enter the global average pooling layer, the two full-connection layers and the Sigmoid layer in sequence to establish the relation among the channels, then evaluating the importance degree of image information contained in each channel in the screening and recombining layer to obtain the information weight of each channel, sorting the characteristic graphs of each channel according to the weight to select a half channel characteristic graph with larger weight, and splicing the half channel characteristic graphs in the channel dimension to obtain an information enhancement matrix; and finally, performing self-adaptive fusion on the three obtained feature matrix pairs of the feature matrix 1, the feature matrix 2 and the information enhancement matrix, integrating the three feature matrices in a way of adding element by element, then sequentially obtaining corresponding weights of the three feature matrices through a global average pooling layer, a full connection layer and a Softmax layer, and fusing the three feature matrices in a way of weighted summation to obtain a new feature matrix as output.
Preferably, the interpretation method of the intelligent interpretation module comprises the following steps:
1) acquiring all images of the same cytopathology smear processed by the image preprocessing module;
2) classifying and judging normal images and focus images of all the images by using the two classification models, and extracting all the focus images;
3) classifying and judging all the focus images by using the multi-classification model to obtain the focus type of each focus image;
4) and counting all the images, forming a primary judgment result according to the proportion of the focus images in all the images and outputting the primary judgment result, and forming a secondary judgment result according to the proportion of each focus type in all the focus images and outputting the secondary judgment result.
Preferably, in the step 4), the first-level discrimination result and the second-level discrimination result are formed according to the following rules:
i, for the first-level discrimination result P1According to the proportion K of the focus image to all the images1The value ranges of (a) are classified as follows:
K1＝0，P1is 0 grade; k is more than 01≤10，P1Is grade 1; k is more than 101≤50，P1Is 2 grade; k is more than 501≤70，P1Grade 3; k is more than 701≤100，P1Grade 4;
P1the grade of (a) represents the probability that the sample corresponding to the current cytopathology smear is diseased;
II, for the second-level discrimination result P2According to the proportion K of each type of focus image to all focus images2Value range ofThe following grading is carried out:
K2＝0，P2is 0 grade; k is more than 02≤10，P2Is grade 1; k is more than 102≤50，P2Is 2 grade; k is more than 502≤70，P2Grade 3; k is more than 702≤100，P2Grade 4;
P2the grade of (a) characterizes the probability that the current cytopathology smear is of a particular type of disorder for the case;
III when P1And P2When the absolute value of the difference between the two is more than 2, the second-level discrimination result P corresponding to the current focus category is used2Subtracting 1 from the level of (1) and outputting the result as a final two-stage judgment result.
Preferably, the processing method of the report composition module comprises the following steps:
s1, first obtaining a first-level discrimination result P1When P is1At level 0, according to P1Outputting an interpretation conclusion text with the meaning of 'the current sample is healthy or the current sample is not sick'; when P is present1If not, entering the next step;
s2, obtaining a secondary judgment result P2A 1 is to P2The level 0, the level 1, the level 2, the level 3 and the level 4 are sequentially corresponding to interpretation conclusion texts comprising the following meanings and output: "exclude or not present type of disorder", "small probability present type of disorder", "suspected present type of disorder", "large probability present type of disorder", "confirmed present type of disorder".
The invention has the beneficial effects that: the automatic interpretation system for the cytopathology smear successfully applies the artificial intelligent auxiliary diagnosis technology to the rapid staining cytopathology, can remarkably improve the accuracy and consistency of diagnosis, and reduces the workload of cytopathology doctors; in the intelligent interpretation module provided by the invention, by improving the conventional convolutional neural network and utilizing the characteristics of a multi-channel attention mechanism and the like, the uncertain factors introduced by manual sampling are solved, a full-scene and multi-classification task with high accuracy can be realized, and the accuracy of a classification interpretation result can be finally improved.
Drawings
FIG. 1 is a functional block diagram of an automated cytological smear interpretation system of the present invention;
FIG. 2 is a flow chart of the operation of the automated cytological smear interpretation system of the present invention;
FIG. 3 is a schematic structural diagram of an ecSK _ Unit module according to the present invention;
fig. 4 is a diffquick pathology stain image of lung adenocarcinoma cells in an example of the invention.
Detailed Description
The present invention is further described in detail below with reference to examples so that those skilled in the art can practice the invention with reference to the description.
It will be understood that terms such as "having," "including," and "comprising," as used herein, do not preclude the presence or addition of one or more other elements or groups thereof.
Referring to fig. 1, the invention provides an automatic interpretation system for cytopathology smear, comprising:
an imaging module;
the image acquisition module controls the imaging module to image the cytopathology smear so as to realize image acquisition of the cytopathology smear;
the image storage and management module is used for storing the images acquired by the image acquisition module and arranging the images to form an image sample library;
the image preprocessing module is used for preprocessing the image output by the image sample library, removing an uncolored area in the image and outputting the image;
the intelligent interpretation module receives the image output by the image preprocessing module, performs prediction classification on the image on a normal sample and a focus sample, and further performs prediction interpretation on the focus type when the image is the focus sample;
and the report writing module automatically generates an interpretation conclusion text of the sample corresponding to the image according to the result of the intelligent interpretation module.
The imaging module comprises a digital microscope, an electric objective table and an upper computer; the digital microscope and the electric objective table are both in communication connection with an upper computer, the electric objective table has a X, Y, Z triaxial moving function, the cytopathology smear is placed on the electric objective table and imaged through the digital microscope, and the obtained image is transmitted to the upper computer in real time;
the automatic interpretation system for the cytopathology smear further comprises an electric objective table control module and an automatic focusing module, wherein the electric objective table control module is used for controlling the movement of the electric objective table; the automatic focusing module is connected with the electric objective table control module, and realizes automatic focusing by controlling the electric objective table to move according to factors such as image sharpness and the like, so that the returned image is ensured to be in the clearest state.
Wherein, the host computer is a computer or a smart phone or other intelligent terminals. In this embodiment, the upper computer is a computer, and the image acquisition module, the image storage and management module, the image preprocessing module, the intelligent interpretation module and the report writing module are sequentially connected and all embedded in the computer.
In a preferred embodiment, the image acquisition module is connected with the upper computer to receive the image output by the upper computer, and the work flow of the image acquisition module is as follows:
firstly, controlling the electric objective table to move until the center of the electric objective table is superposed with the central axis of an objective lens of the digital microscope, then completing automatic focusing through a focusing control module, and then fixing the focal length;
controlling the electric objective table to move, enabling an objective lens of the digital microscope to be aligned to the upper left corner of the electric objective table, then translating a unit distance in the X direction from the upper left corner every time, taking a picture to obtain an image, translating the picture in the Y direction by a unit distance when reaching a right side end point, continuing translating the picture in the opposite direction of the X direction, translating the unit distance every time to take the picture to obtain an image, and performing snake-shaped advance shooting until all cytopathology smears on the electric objective table are shot;
and acquiring the image of the cytopathology smear from the upper computer, and outputting the image to the image storage and management module.
In a preferred embodiment, the cytopathology smear is a cytopathology smear stained with diffquick, or a cytopathology smear with another staining method.
In a preferred embodiment, the image storage and management module acquires and numbers all images of the cytopathology smear acquired by the image acquisition module each time, wherein the numbers comprise the cytopathology smear serial number corresponding to the image and the position number of the image, so that an image sample library consisting of all images of a plurality of cytopathology smears is acquired after multiple acquisition.
In a preferred embodiment, referring to fig. 2, the processing method of the image preprocessing module comprises the following steps:
for the image obtained from the image storage and management module, removing the gray or white non-stained blood cell area according to the color values of the RGB three channels, and reserving the blue-violet stained blood cell area;
and performing binarization operation on the reserved region to enable the reserved region to become a foreground mask, taking the removed region as a background mask, performing closed processing on the foreground mask part, performing edge expansion to obtain a final foreground mask, and performing matting from the initially acquired image by using the foreground mask to obtain a preprocessed image.
In a further preferred embodiment, the image preprocessing module can also classify and mark the images, and label the images as different labels, where the labels at least include positive/abnormal labels, negative/normal labels, and different pathological grading labels. And labeling the image through an image preprocessing module to construct a training image.
In a preferred embodiment, the intelligent interpretation module comprises a two-classification model for distinguishing normal samples from lesion samples and a multi-classification model for distinguishing lesion types, wherein the two-classification model and the multi-classification model are obtained by respectively training a first training set and a second training set based on the same classification network; the first training set includes a number of normal sample images and a number of abnormal sample images with corresponding marks, and the second training set includes a number of abnormal sample images with different lesion type marks.
Marking a plurality of acquired preprocessed images through an image preprocessing module, and constructing a first training set comprising a plurality of normal sample images and a plurality of abnormal sample images and a second training set comprising a plurality of abnormal sample images with different lesion type marks; then, training a classification network by using a first training set to obtain a two-classification model; and training the other classification network by using the second training set to obtain a multi-classification model.
In a further preferred embodiment, the classification network is a ResNet network, a ResNeXt network or a densneet network that is added with the ecSK _ Unit module, and attention modules such as SENet and SKNet can also be added to the classification network.
In a further preferred embodiment, the structure of the ecSK _ Unit module is shown in fig. 3, and the processing method of the ecSK _ Unit module includes:
after the image matrix is input into the ecSK _ Unit module, firstly, feature extraction is carried out on a convolution layer with convolution kernels of 3 × 3 and 5 × 5 convolution operations to obtain two feature matrices, and the two feature matrices are marked as a feature matrix 1 and a feature matrix 2; then, performing matrix splicing operation on the two characteristic matrixes in the channel dimension through the connection layer, enabling the spliced characteristic matrixes to enter the global average pooling layer, the two full-connection layers and the Sigmoid layer in sequence to establish the relation among the channels, then evaluating the importance degree of image information contained in each channel in the screening and recombining layer to obtain the information weight of each channel, sorting the characteristic graphs of each channel according to the weight to select a half channel characteristic graph with larger weight, and splicing the half channel characteristic graphs in the channel dimension to obtain an information enhancement matrix; and finally, performing self-adaptive fusion on the three obtained feature matrix pairs of the feature matrix 1, the feature matrix 2 and the information enhancement matrix, integrating the three feature matrices in a way of adding element by element, then sequentially obtaining corresponding weights of the three feature matrices through a global average pooling layer, a full connection layer and a Softmax layer, and fusing the three feature matrices in a way of weighted summation to obtain a new feature matrix as output.
In a preferred embodiment, the interpretation method of the intelligent interpretation module comprises the following steps:
1) acquiring all images of the same cytopathology smear processed by the image preprocessing module;
2) classifying and judging normal images and focus images by using a secondary classification model, and extracting all focus images;
3) classifying and judging all focus images by using a multi-classification model to obtain the focus type of each focus image;
4) and counting all the images, forming a primary judgment result according to the proportion of the focus images in all the images and outputting the primary judgment result, and forming a secondary judgment result according to the proportion of each focus type in all the focus images and outputting the secondary judgment result.
In a further embodiment, the first-level discrimination result and the second-level discrimination result are formed in step 4) according to the following rules:
i, for the first-level discrimination result P1According to the proportion K of the focus image to all the images1The value ranges of (a) are classified as follows:
K1＝0，P1is 0 grade; k is more than 01≤10，P1Is grade 1; k is more than 101≤50，P1Is 2 grade; k is more than 501≤70，P1Grade 3; k is more than 701≤100，P1Grade 4;
P1the grade of (a) represents the probability that the sample corresponding to the current cytopathology smear is diseased;
II, for the second-level discrimination result P2According to the proportion K of each type of focus image to all focus images2The value ranges of (a) are classified as follows:
K2＝0，P2is 0 grade; k is more than 02≤10，P2Is grade 1; k is more than 102≤50，P2Is 2 grade; k is more than 502≤70，P2Grade 3; k is more than 702≤100，P2Grade 4;
P2the grade of (a) characterizes the probability that the current cytopathology smear is of a particular type of disorder for the case;
III when P1And P2When the absolute value of the difference between the two is more than 2, the second-level discrimination result P corresponding to the current focus category is used2Subtracting 1 from the level of (1) and outputting the result as a final two-stage judgment result.
In a further embodiment, the method of processing the report composition module comprises the steps of:
s1, first obtaining a first-level discrimination result P1When P is1At level 0, according to P1The result of (a) outputting an interpretation conclusion text of "the current sample is healthy or the current sample is not a symptom" or an interpretation conclusion text having the same meaning, for example, "the sample is not a symptom XX"; when P is present1If not, entering the next step;
s2, obtaining a secondary judgment result P2A 1 is to P2The level 0, the level 1, the level 2, the level 3 and the level 4 are sequentially corresponding to the following interpretation conclusion texts or the interpretation conclusion texts including the following meanings and output: "exclude or not present type of disorder", "small probability present type of disorder", "suspected present type of disorder", "large probability present type of disorder", "diagnosed present type of disorder"; for example, "the sample has a high probability of being a XX disorder, is suspected of being a XX disorder, has a low probability of being a XX disorder, excludes a XX disorder.
The foregoing is a general idea of the present invention, and detailed examples are provided below to further explain the present invention.
Example 1
In this embodiment, the digital microscope has a maximum 100-fold optical zoom function, and the upper computer is a computer with an X86 architecture.
1. Obtaining a training set
1-1, collecting healthy cells (negative and atypical) and lung cancer cells, placing the cells on a glass slide, staining the cells by using a diffquick technology, and making a cytopathology smear, wherein the focus types are further divided into adenocarcinoma, squamous carcinoma, small cell carcinoma and large cell neuroendocrine carcinoma. Referring to fig. 4, adenocarcinoma cells (blue) are shown by the arrows.
1-2, placing the cytopathology smear on an electric objective table, and enabling an imaging module and an image acquisition module to work:
firstly, controlling the electric objective table to move until the center of the electric objective table is superposed with the central axis of an objective lens of the digital microscope, then completing automatic focusing through a focusing control module, and then fixing the focal length; and controlling the electric objective table to move so that the objective lens of the digital microscope is aligned with the upper left corner of the electric objective table. The corner detection can be performed using an edge detection algorithm based on the canny operator and a right angle detection algorithm based on template matching, and the stage stops moving when the upper left corner edge appears in the field of view.
Starting from the upper left corner, translating 5mm each time, and shooting according to snake-shaped advancing; shooting all cytopathology smears on the electric objective table; and acquiring the image of the cytopathology smear from the upper computer, and outputting the image to the image storage and management module. The image storage and management module acquires and numbers all images of the cytopathology smear acquired by the image acquisition module each time, wherein the numbers comprise the cytopathology smear serial number corresponding to the image and the position number of the image, such as the form of 'sample number-position number'.
And 1-3, preprocessing by an image preprocessing module.
1-4, labeling through an image preprocessing module, and labeling the images with 2 types of labels of normal (negative, atypical) and abnormal (tumor) to form a first training set; 4 types of marking of adenocarcinoma, squamous carcinoma, small cell carcinoma and large cell neuroendocrine carcinoma are carried out on the copy of the abnormal sample according to the type of the focus to which the copy belongs, and a second training set is formed; and respectively using a first training set and a second training set to train the ResNet network integrated with the ecSK _ Unit, performing 8-time amplification on the sample by using orthogonal twice-turning and histogram equalization during training, and obtaining a two-class model and a multi-class model by using a loss function in the training as a cross entropy.
Secondly, automatically interpreting the sample to be detected
1. Firstly, obtaining a cytopathology smear image of a sample to be detected according to the steps from 1-1 to 1-3;
2. using a second classification model to judge the normal and abnormal conditions of all images, if the images are judged to be normal, outputting a text 'the sample is negative' through a report writing module, and ending the process; if the judgment result is abnormal, the next step is carried out;
3. classifying and judging all abnormal sample focus images by using a multi-classification model to obtain specific classifications of samples, wherein 4 classes of adenocarcinoma, squamous carcinoma, small cell carcinoma and large cell neuroendocrine carcinoma are respectively and correspondingly marked as c1, c2, c3 and c 4;
4. counting the proportion of c 1-4 in all test samples, wherein the number of the test samples is 30, 15 abnormal samples account for 50%, namely the primary judgment result P1 is 2 level;
5. counting the proportion of each type in c 1-4 in the abnormal samples, wherein the proportion is c1: 10%, and the corresponding second-level discrimination result P2 is level 1; c2: 35%, corresponding P2 is grade 2; c3: 55%, corresponding P2 is grade 3; c4: 0%, corresponding P2 is grade 0;
according to the statistical result, P1 is grade 2, the two-stage discrimination results of each abnormal class are respectively grade c1:1, grade c2:2, grade c3:3 and grade c4:0, and all the two-stage discrimination results do not need to be corrected;
6. the report writing module obtains the interpretation result of the intelligent interpretation module, and outputs an interpretation conclusion text 'the case has high probability of small cell carcinoma, suspected squamous carcinoma and low probability of adenocarcinoma and excludes large cell neuroendocrine carcinoma' according to the corresponding case library.
While embodiments of the invention have been disclosed above, it is not limited to the applications listed in the description and the embodiments, which are fully applicable in all kinds of fields of application of the invention, and further modifications may readily be effected by those skilled in the art, so that the invention is not limited to the specific details without departing from the general concept defined by the claims and the scope of equivalents.
Claims (10)
1. An automatic cytological smear interpretation system, comprising:
an imaging module;
the image acquisition module controls the imaging module to image the cytopathology smear so as to realize image acquisition of the cytopathology smear;
the image storage and management module is used for storing the images acquired by the image acquisition module and arranging the images to form an image sample library;
the image preprocessing module is used for preprocessing the image output by the image sample library, removing an uncolored area in the image and outputting the image;
the intelligent interpretation module receives the image output by the image preprocessing module, performs prediction classification on the image on a normal sample and a focus sample, and further performs prediction interpretation on the focus type when the image is the focus sample;
and the report writing module is used for automatically generating an interpretation conclusion text of the sample corresponding to the image according to the result of the intelligent interpretation module.
2. The automated cytological smear interpretation system of claim 1, wherein the imaging module comprises a digital microscope, a motorized stage and a host computer; the electric objective table has X, Y, Z triaxial movement function, the cytopathology smear is placed on the electric objective table, imaging is carried out through the digital microscope, and the obtained image is transmitted to the upper computer;
the automatic interpretation system for the cytopathology smear also comprises an electric objective table control module and an automatic focusing module, wherein the electric objective table control module is used for controlling the movement of the electric objective table; the automatic focusing module is connected with the electric objective table control module and controls the electric objective table to move so as to realize automatic focusing.
3. The cytopathology smear automatic interpretation system according to claim 2, wherein the image acquisition module is connected with the upper computer to receive the image outputted by the upper computer, and the work flow of the image acquisition module is as follows:
firstly, the electric objective table is controlled to move to the position that the center of the electric objective table is overlapped with the central axis of an objective lens of the digital microscope, then automatic focusing is completed through the focusing control module, and then the focal length is fixed;
controlling the electric objective table to move, enabling an objective lens of the digital microscope to be aligned to the upper left corner of the electric objective table, then translating a unit distance from the upper left corner in the X direction each time, taking a picture to obtain an image, translating the unit distance in the Y direction when reaching the right side end point, continuing translating the unit distance in the opposite direction of the X direction, translating the unit distance each time to take a picture to obtain an image, and performing snake-shaped advance shooting until all cytopathology smears on the electric objective table are shot;
and acquiring the image of the cytopathology smear from an upper computer, and outputting the image to the image storage and management module.
4. The automated cytological smear interpretation system according to claim 3, wherein the image storage and management module acquires and numbers all images of the cytological smear acquired by the image acquisition module each time, the numbers include the serial number of the cytological smear corresponding to the image and the position number of the image, so as to obtain an image sample library consisting of all images of a plurality of cytological smear after a plurality of acquisitions.
5. The automated cytological smear interpretation system according to claim 4, wherein the processing method of the image pre-processing module comprises the following steps:
removing a gray or white non-stained blood cell area and reserving a blue-violet stained blood cell area according to RGB three-channel color values of the image acquired from the image storage and management module;
and performing binarization operation on the reserved region to enable the reserved region to become a foreground mask, taking the removed region as a background mask, performing closed processing on the foreground mask part, performing edge expansion to obtain a final foreground mask, and performing matting from the initially acquired image by using the foreground mask to obtain a preprocessed image.
6. The automated cytological smear interpretation system according to claim 5, wherein the intelligent interpretation module comprises a two-classification model for distinguishing normal samples from lesion samples and a multi-classification model for distinguishing types of lesions, the two-classification model and the multi-classification model are trained respectively by the first training set and the second training set based on the same classification network;
the first training set includes a number of normal sample images and a number of abnormal sample images with corresponding marks, and the second training set includes a number of abnormal sample images with different lesion type marks.
7. The automated cytological smear interpretation system according to claim 6, wherein the classification network is a ResNet network, a ResNeXt network or a DenseNet network added with an ecSK _ Unit module, and the processing method of the ecSK _ Unit module comprises:
after the image matrix is input into the ecSK _ Unit module, firstly, feature extraction is carried out on a convolution layer with convolution kernels of 3 × 3 and 5 × 5 convolution operations to obtain two feature matrices, and the two feature matrices are marked as a feature matrix 1 and a feature matrix 2; then, performing matrix splicing operation on the two characteristic matrixes in the channel dimension through the connection layer, enabling the spliced characteristic matrixes to enter the global average pooling layer, the two full-connection layers and the Sigmoid layer in sequence to establish the relation among the channels, then evaluating the importance degree of image information contained in each channel in the screening and recombining layer to obtain the information weight of each channel, sorting the characteristic graphs of each channel according to the weight to select a half channel characteristic graph with larger weight, and splicing the half channel characteristic graphs in the channel dimension to obtain an information enhancement matrix; and finally, performing self-adaptive fusion on the three obtained feature matrix pairs of the feature matrix 1, the feature matrix 2 and the information enhancement matrix, integrating the three feature matrices in a way of adding element by element, then sequentially obtaining corresponding weights of the three feature matrices through a global average pooling layer, a full connection layer and a Softmax layer, and fusing the three feature matrices in a way of weighted summation to obtain a new feature matrix as output.
8. The automated cytological smear interpretation system according to claim 7, wherein the interpretation method of the intelligent interpretation module comprises the steps of:
1) acquiring all images of the same cytopathology smear processed by the image preprocessing module;
2) classifying and judging normal images and focus images of all the images by using the two classification models, and extracting all the focus images;
3) classifying and judging all the focus images by using the multi-classification model to obtain the focus type of each focus image;
4) and counting all the images, forming a primary judgment result according to the proportion of the focus images in all the images and outputting the primary judgment result, and forming a secondary judgment result according to the proportion of each focus type in all the focus images and outputting the secondary judgment result.
9. The automated cytological smear interpretation system according to claim 8, wherein the step 4) forms the primary and secondary discrimination results according to the following rules:
i, for the first-level discrimination result P1According to the proportion K of the focus image to all the images1The value ranges of (a) are classified as follows:
K1＝0，P1is 0 grade; k is more than 01≤10，P1Is grade 1; k is more than 101≤50，P1Is 2 grade; k is more than 501≤70，P1Grade 3; k is more than 701≤100，P1Grade 4;
P1the grade of (a) represents the probability that the sample corresponding to the current cytopathology smear is diseased;
II, for the second-level discrimination result P2According to the proportion K of each type of focus image to all focus images2The value ranges of (a) are classified as follows:
K2＝0，P2is 0 grade; k is more than 02≤10，P2Is grade 1; k is more than 102≤50，P2Is 2 grade; k is more than 502≤70，P2Grade 3; k is more than 702≤100，P2Grade 4;
P2the grade of (a) characterizes the probability that the current cytopathology smear is of a particular type of disorder for the case;
III when P1And P2When the absolute value of the difference between the two is more than 2, the second-level discrimination result P corresponding to the current focus category is used2Subtracting 1 from the level of (1) and outputting the result as a final two-stage judgment result.
10. The automated cytological smear interpretation system according to claim 9, wherein the processing method of the report composition module comprises the steps of:
s1, first obtaining a first-level discrimination result P1When P is1At level 0, according to P1Outputting an interpretation conclusion text with the meaning of 'the current sample is healthy or the current sample is not sick'; when P is present1If not, entering the next step;
s2, obtaining a secondary judgment result P2A 1 is to P2The level 0, the level 1, the level 2, the level 3 and the level 4 are sequentially corresponding to interpretation conclusion texts comprising the following meanings and output: "exclude or not present type of disorder", "small probability present type of disorder", "suspected present type of disorder", "large probability present type of disorder", "confirmed present type of disorder".
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111626061.5A CN114387596A (en) | 2021-12-28 | 2021-12-28 | Automatic interpretation system for cytopathology smear |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111626061.5A CN114387596A (en) | 2021-12-28 | 2021-12-28 | Automatic interpretation system for cytopathology smear |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114387596A true CN114387596A (en) | 2022-04-22 |
Family
ID=81197660
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111626061.5A Pending CN114387596A (en) | 2021-12-28 | 2021-12-28 | Automatic interpretation system for cytopathology smear |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114387596A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116230214A (en) * | 2023-05-08 | 2023-06-06 | 浙江大学滨江研究院 | HCC and VETC auxiliary diagnosis device and equipment |
| CN116259053A (en) * | 2023-05-15 | 2023-06-13 | 西南科技大学 | Medical microscopic image imaging focus prediction method based on convolutional neural network |
-
2021
- 2021-12-28 CN CN202111626061.5A patent/CN114387596A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116230214A (en) * | 2023-05-08 | 2023-06-06 | 浙江大学滨江研究院 | HCC and VETC auxiliary diagnosis device and equipment |
| CN116259053A (en) * | 2023-05-15 | 2023-06-13 | 西南科技大学 | Medical microscopic image imaging focus prediction method based on convolutional neural network |
| CN116259053B (en) * | 2023-05-15 | 2023-07-21 | 西南科技大学 | Medical microscopic image imaging focus prediction method based on convolutional neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| DK2973397T3 (en) | Tissue-object-based machine learning system for automated assessment of digital whole-slide glass | |
| CN109903284B (en) | HER2 immunohistochemical image automatic discrimination method and system | |
| US20220351860A1 (en) | Federated learning system for training machine learning algorithms and maintaining patient privacy | |
| Veta et al. | Detecting mitotic figures in breast cancer histopathology images | |
| US20220351379A1 (en) | Non-tumor segmentation to support tumor detection and analysis | |
| CN114387596A (en) | Automatic interpretation system for cytopathology smear | |
| JP2010540931A (en) | Method and system using multiple wavelengths for processing biological specimens | |
| CN112784767A (en) | Cell example segmentation algorithm based on leukocyte microscopic image | |
| Hossain et al. | Tissue artifact segmentation and severity assessment for automatic analysis using wsi | |
| CN113628199A (en) | Pathological picture stained tissue area detection method, pathological picture stained tissue area detection system and prognosis state analysis system | |
| CN113902669A (en) | Method and system for reading urine exfoliative cell fluid-based smear | |
| CN115206495A (en) | Renal cancer pathological image analysis method and system based on CoAtNet deep learning and intelligent microscopic device | |
| CN115471838A (en) | Cervical squamous lesion cell detection method based on depth self-adaptive feature extraction | |
| CN116757998A (en) | Screening method and device for CTC cells and CTC-like cells based on AI | |
| CN113222928B (en) | Urine cytology artificial intelligence urothelial cancer identification system | |
| CN114898866A (en) | Thyroid cell auxiliary diagnosis method, equipment and storage medium | |
| CN111783571A (en) | Cervical cell automatic classification model establishment and cervical cell automatic classification method | |
| CN117496276B (en) | Lung cancer cell morphology analysis and identification method and computer readable storage medium | |
| CN114821046B (en) | Method and system for cell detection and cell nucleus segmentation based on cell image | |
| Koss | The application of PAPNET to diagnostic cytology | |
| CN113344868B (en) | Label-free cell classification screening system based on mixed transfer learning | |
| Shimomoto et al. | Cell Nucleus Detection in Oral Cytology Using Artificial Intelligence | |
| Sabata et al. | Automated analysis of pin-4 stained prostate needle biopsies | |
| Ben Taieb | Analyzing cancers in digitized histopathology images | |
| EP4246468A1 (en) | Method for training an artificial intelligence system, method for recognizing and classifying cells for pathological cell and tissue examination, classifying system, computer-readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |