CN107274386B - artificial intelligent auxiliary cervical cell fluid-based smear reading system - Google Patents
artificial intelligent auxiliary cervical cell fluid-based smear reading system Download PDFInfo
- Publication number
- CN107274386B CN107274386B CN201710351064.XA CN201710351064A CN107274386B CN 107274386 B CN107274386 B CN 107274386B CN 201710351064 A CN201710351064 A CN 201710351064A CN 107274386 B CN107274386 B CN 107274386B
- Authority
- CN
- China
- Prior art keywords
- cell
- image
- cervical
- neural network
- cells
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20028—Bilateral filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
Abstract
the invention discloses an artificial intelligent auxiliary film reading system for a cervical cell fluid-based smear, which comprises: the cell image detection and segmentation module is used for automatically detecting different cell components of the image cell and automatically segmenting cell nucleus, cytoplasm and background in the same cell component; fine adjustment and optimization are carried out on the segmentation result; the cell rapid grading identification module is used for identifying the segmented image and distinguishing the segmented image into single cells or cell clusters; adopting a double-current convolutional neural network in the additional knowledge field and a constructed cell knowledge map to respectively carry out hierarchical identification on single cells to respectively obtain two hierarchical results; identifying the non-separable cell clusters by adopting a double-current convolution neural network model of the cell clusters; and the interpretation and post-processing module is used for carrying out combined interpretation on the two grading results of the single cell and carrying out conflict processing to obtain the grading result of the single cell.
Description
Technical Field
The invention relates to the field of medical cell image processing, in particular to an artificial intelligent auxiliary cervical cell fluid-based smear reading system.
background
Cervical cancer is a malignant tumor occurring in the cervical part of the uterus, and is closely related to high-risk HPV infection. Early detection, diagnosis and treatment of cervical precancerous lesions are one of the strategies for reducing the incidence and mortality of cervical cancer. The cervical cancer screening method is simple, convenient and economical, and screening means are adopted by many countries to control the incidence and mortality of cervical cancer and reduce the disease burden.
In 2008 to 2016, Beijing city can complete free screening of more than 200 cervical cancer, more than 4000 cases of cervical cancer and precancerous lesions are detected, the positive detection rate of cervical cytology is 2.3%, according to foreign data and research data of partial areas of China, experts speculate that the positive detection rate of cervical cytology in normal people of China is approximately between 5 and 7%, and the detection rate of Beijing has a larger difference from the standard at present.
The artificial intelligence assisted cervical cell liquid-based smear reading has positive effects on reducing the labor intensity of a film reading worker, improving the film reading accuracy and work efficiency, being applied to large-batch cervical cancer cytology general investigation and the like, follows the development rules and requirements of the medical market, accords with the research layout of national health medical big data, and has important significance in the medical big data industry in image informatization and image intelligent diagnosis.
some well-known research institutes in the united states and europe have started to perform quantitative cervical cell analysis and related research for auxiliary diagnosis, such as the slide scanning image analysis system of the american heroe company, which can interpret cervical cytosol-based smears, on one hand, the system cannot give final grading diagnosis results, and its function is to select 22 fields from smears for the physician to interpret, and the physician can not interpret other fields to improve the efficiency of the interpretation; on the other hand, the system can only judge the liquid-based smear produced by the company, and can not judge the liquid-based smear of other companies, so that the universality of the film reading system is caused. The Wuhanlan Ding medicine in China is focused on cervical cancer screening, and an automatic cervical cancer cell detection system based on the DNA quantitative analysis of images is developed, but the automatic cervical cancer cell detection system is lack of characteristic extraction of various cells in the aspect of data analysis, and the interpretability of a judgment result cannot be given.
The existing cervical cell quantitative analysis and auxiliary diagnosis method still has obvious defects, the research of the cervical cell analysis technology is still in a starting stage, in order to further deepen the clinical and research needs in the field of cervical cytology, the cervical cell morphological structure information, the spatial neighborhood information and the color distribution information are fully utilized to establish accurate cell image segmentation, the cervical cell quantitative analysis and intelligent auxiliary diagnosis frame is fully established by utilizing the cervical cell database and the cervical cell clinical diagnosis rule, and a software platform is provided for the artificial intelligent auxiliary film reading system based on the cervical cell liquid-based smear.
disclosure of Invention
the present invention aims to overcome the above problems of the prior cervical cell quantitative analysis and auxiliary diagnosis; based on the knowledge in the field of the existing cervical cytology, the method learns and extracts the key characteristics of the cervical cells through a deep learning technology, automatically segments and identifies the cancerous area and type of the cells on the liquid-based smear, finally achieves the purposes of shortening the smear reading time and reducing the missed diagnosis misdiagnosis rate, provides a solution for the artificial intelligent auxiliary smear reading technology of the cervical cell liquid-based smear, and finally realizes the methodological breakthrough of quantitative evaluation and auxiliary intelligent diagnosis of the cervical cells.
in order to achieve the above object, the present invention provides an artificial intelligence auxiliary smear system for cervical cell fluid-based smear, which comprises:
the cell image acquisition module is used for scanning and storing cell images in an overlapped mode by using the automatic smear scanner;
The cell image preprocessing module is used for preprocessing the cell image;
The cell image detection and segmentation module is used for automatically detecting different cell components of the image cell and automatically segmenting a cell nucleus, a cell cytoplasm and a background in the same cell component; an improved active contour model and a fast regional convolutional neural network are adopted for detection, and algorithms such as cell characteristics and multi-scale levels are used for fine adjustment and optimization of segmentation results;
The cell rapid grading identification module is used for identifying the segmented image and distinguishing the segmented image into single cells or cell clusters; respectively carrying out hierarchical identification on single cells by adopting a double-current convolutional neural network in the additional knowledge field and a constructed cell knowledge map to respectively obtain a first hierarchical result and a second hierarchical result; identifying the non-separable cell clusters by adopting a double-current convolution neural network model of the cell clusters; and
The interpretation and post-processing module is used for carrying out combined interpretation on the first grading result and the second grading result of the single cell and carrying out conflict processing to obtain the grading result of the single cell; the conflict processing is used for solving the problem that when various characteristics point to different interpretation results, various factors are integrated, conflicts are eliminated, and clear and reliable interpretation is made; and then, readability of the cervical cell identification process and interpretability of the cervical cell identification result are realized by using the knowledge map and the activity-like mapping.
In the above technical solution, the cell image acquisition module is implemented by: the scanning range and the range of the liquid-based smear cells are completely covered by adopting the forty-fold magnification of an ocular lens, the scanning path is rectangular, and the scanning mode is overlapped scanning.
In the above technical solution, the preprocessing includes: the image is denoised by adopting a bilateral filter, wherein the filter consists of two functions: one function determines the filter coefficients from the geometric spatial distance and the other determines the filter coefficients from the pixel difference; the edges of the image are then patched using morphological processing, filling the holes and removing fine junctions, and finally histogram equalization is used to increase the contrast of the nucleus to the cytoplasm.
in the above technical solution, the cell image detection and segmentation module specifically comprises the following steps:
step S1), carrying out foreground and background rough segmentation on the preprocessed cell image, and extracting the region to which the cell belongs;
Step S2) detecting and segmenting cell components of the roughly segmented cell image, and segmenting cells of different types by using a fast regional convolution neural network;
Step S3) detecting and segmenting the cervical cell nuclei;
Step S4) screening cell nucleuses according to the characteristic parameters of the cell nucleuses to obtain final candidate cell nucleuses;
Step S5), judging whether the cell type obtained in the step S2) is a cell cluster, if not, using the active contour model and the prior template to segment the cytoplasm region; otherwise, go to step S6);
Step S6) the segmentation results of the cell nucleus and the cytoplasm and the domain knowledge are integrated for post-processing, and the effective segmentation of the whole cervical cell is completed.
in the above technical solution, the fast regional convolutional neural network adopts a network structure of VGG16 of convolutional neural network, the size of the input image is 515 × 512, and the detection categories of the final cell components are classified into 5 categories: squamous cells, glandular cells, cervical cells, metaplastic cells, and background diathesis, the cellular components excluding squamous cells and background diathesis are defined as non-separable cell clusters.
In the above technical solution, the specific process of segmenting the cytoplasmic region by using the active contour model and the prior template in step S5) includes:
An improved active contour model is adopted, an energy function and shape prior information are added, contour optimization is carried out iteratively, and accurate boundaries of cytoplasm are obtained;
the energy function E (u) is:
E(u)＝λ1Es(u)+R(u)
Wherein Es(u) is the shape prior, R (u) is a regularization term to ensure smoothness of the segmentation boundary, λ1is a learnable parameter; shape prior Es(u) is:
wherein H is a blackplug matrix.
In the above technical solution, the cell image detection and segmentation module specifically comprises the following steps:
Step 1) preprocessing the segmented cervical cell image;
Step 2) judging whether the cell image after pretreatment is a single cell, if so, switching to step 3), otherwise, switching to step 6 if the image is an inseparable cell cluster;
step 3) determining a calculable cell parameter, and then calculating cell parameter characteristics;
Step 4) establishing a cell knowledge map inference judgment model, and inputting cell parameter characteristics into the model to obtain a first grading result of a single cell;
step 5) constructing a double-current convolutional neural network model with additional domain knowledge, and inputting cell parameter characteristics and cell images into the double-current convolutional neural network model to obtain a second grading result of a single cell;
And 6) constructing a double-current convolutional neural network model of the cell clusters, and carrying out hierarchical identification on the cell clusters of the undifferentiated cell clusters by using the model to obtain a hierarchical result of the cell clusters.
in the technical scheme, one input of the double-current convolutional neural network with additional domain knowledge is the cell parameter characteristics obtained in the step 3), the other input is a single cell image, the size of the single cell image is unified and normalized to 256 × 256 pixel values, and the characteristics of the cell image are implicitly extracted through 5 cascaded convolutional pooling combined modules; the convolution kernel size of the most important convolution operation is 7 × 7, the step size is selected to be 1, the number of feature maps is selected to be 96, and the convolution operation is as follows:
in the above formula, M represents a set of selected input feature maps, wijrepresents a weight, bjan additional bias is added to each feature map output, and the extracted 1096-dimensional features are then spliced together with 20-dimensional features that can be computed from cell domain knowledge and input to the fully-connected and classified layers of the dual-flow convolutional neural network.
in the above technical solution, one input of the double-current convolutional neural network of the cell clusters is: the method comprises the following steps that (1) the cell nuclei are regularly arranged, the other path of input is cervical cells of cell clusters corresponding to cell parameters, the input sizes of the cervical cells are uniformly normalized to 512 x 512 pixel values, and the cell image characteristics are implicitly extracted through 8 cascaded convolution pooling combined modules; the convolution kernel size of the most important convolution operation is 5 × 5, the step size is selected to be 2, and the number of feature maps is selected to be 108.
The invention has the beneficial effects that:
1. the system of the invention has high sensitivity for pathological cervical cell systems and high specificity for normal cervical cells, the whole auxiliary film reading system does not need manual participation, and the labor intensity of film reading workers is greatly reduced.
Drawings
FIG. 1 is a diagram of the artificial intelligence-aided cervical fluid-based smear reading system according to the present invention;
fig. 2 is a schematic diagram of a cervical cell image detection segmentation module of the present invention;
fig. 3 is a schematic diagram of the rapid cervical cell grading identification module of the present invention.
Detailed Description
the following detailed description of the preferred embodiments of the present invention, taken in conjunction with the accompanying drawings, will make the advantages and features of the invention easier to understand by those skilled in the art, and thus will clearly and clearly define the scope of the invention.
as shown in fig. 1, an artificial intelligence auxiliary smear system for cervical fluid-based smear, the system comprises:
the cell image acquisition module is used for scanning and storing cell images in an overlapped mode by using the automatic smear scanner; the scanning device adopts the four-fold magnification of an ocular lens, the scanning path is rectangular, and the scanning mode is overlapped scanning, so that the scanning range and the range of the liquid-based smear cells can be completely covered;
for example, 20 scanned images of cervical cells can be obtained for one image of 2 ten thousand by 2 ten thousand pixels.
the cell image preprocessing module is used for preprocessing the cell image and comprises: the image is denoised by adopting a bilateral filter, wherein the filter consists of two functions: one function determines the filter coefficients from the geometric spatial distance and the other determines the filter coefficients from the pixel difference; the edges of the image are then patched using morphological processing, filling the holes and removing fine junctions, and finally histogram equalization is used to increase the contrast of the nucleus to the cytoplasm.
the cell image detection and segmentation module is used for automatically detecting different cell components of the image cell and automatically segmenting a cell nucleus, a cell cytoplasm and a background in the same cell component; an improved active contour model and a fast regional convolutional neural network are adopted for detection, and algorithms such as cell characteristics and multi-scale levels are used for fine adjustment and optimization of segmentation results;
As shown in fig. 2, the specific implementation steps are as follows:
step S1), carrying out foreground and background rough segmentation on the preprocessed cell image, and extracting the region to which the cell belongs;
Removing noise of the whole cell image by adopting 9 threshold values; then, SIFT edge detection and a multi-scale watershed algorithm are adopted to obtain a foreground region, the SIFT keeps invariance to rotation, scale scaling and brightness change, and also keeps certain stability to view angle change, affine transformation and noise; and finally, carrying out optimization fine adjustment on the segmented regions, and merging adjacent pixels by adopting a K-means clustering algorithm, wherein the cluster number of the clustering algorithm is selected to be 3, namely, the clustering algorithm is divided into three categories of cytoplasm, nucleus and background.
Step S2), detecting and segmenting cell components of the roughly segmented cell image, and segmenting different types of cells and background qualities by using a fast regional convolution neural network;
Firstly, cleaning and sorting data aiming at a training sample, and then detecting and segmenting cell components by adopting a fast regional convolution neural network in a deep learning algorithm. The training stage selects an end-to-end training mode, the model adopts a network structure of VGG16 of a convolutional neural network, the size of an input image is 515 × 512, and finally the detection categories of cell components are classified into 5 types: squamous cells, glandular cells, cervical cells, metaplastic cells, and background diathesis, four classes of cellular components other than squamous cells are defined herein as non-separable cell clusters. Considering the artificial intelligence assisted reading and subsequent grading identification steps, the inseparable cell cluster region is divided into four categories of glandular cells, cervical cells, metaplastic cells and background diathesis in detail, wherein the background diathesis does not belong to the cell category and does not participate in the subsequent grading identification.
step S3) detecting and segmenting the cervical cell nuclei;
an improved random forest algorithm is adopted, 5 characteristics of cell nucleuses are extracted to carry out segmentation on cell nucleus areas, the number of trees in the forest is selected to be 20, the characteristic number is selected to be logN when the optimal characteristic is selected each time, and the minimum number of leaf nodes is selected to be 3;
in order to prevent the cell nucleus region from being missed and detect the cell nucleus by a multi-scale watershed algorithm, 5 different parameters are selected for combination, an image to be segmented is divided into cell images with different scales and high combination degree and low combination degree, and the detected results of the cell images and the cell images are combined to serve as candidate regions of the cell nucleus.
Step S4) screening cell nucleuses according to the characteristic parameters of the cell nucleuses to obtain final candidate cell nucleuses;
the characteristics can be calculated according to characteristic parameters of the cell nucleus, including the size of the cell nucleus, the circularity of the cell nucleus and the depth of the cell nucleus. A parameter such as the size of the nucleus is represented by directly calculating the sum of the pixels within the boundary of the nucleus region, as shown in equation 1.
wherein f (x, y) is the pixel value of a certain point (x, y) on the binary image, the pixel point belongs to the target area when the value is 1, the pixel point belongs to the background area when the value is 0, and the area is the number of pixels of which f (x, y) is 1.
Step S5), judging whether the cell type obtained in the step S2) is a cell cluster, if not, using the active contour model and the prior template to segment the cytoplasm region; otherwise, go to step S6);
an improved active contour model is adopted, an energy function and shape prior information are added, contour optimization is carried out in an iterative mode, and accurate boundaries of cytoplasm are obtained;
the energy function E (u) is:
E(u)＝λ1Es(u)+R(u)
wherein Es(u) is the shape prior proposed for use by the invention, R (u) is a regularization term that ensures smoothness of the segmentation boundary, λ1Is a learnable parameter; shape prior Es(u) is:
Where H is the blackplug Matrix (Hessian Matrix).
Step S6) the segmentation results of the cell nucleus and the cytoplasm and the domain knowledge are integrated for post-processing, and the effective segmentation of the whole cervical cell is completed.
And (2) carrying out post-processing aiming at the segmentation result of the step, mainly using morphological operation in medical image processing to repair the edge of the image, filling the cavity and removing fine connection, mainly adopting opening operation and closing operation, wherein the parameter of the template is set to be the size of [ 33 ], and smoothing the boundary by using methods such as filtering and denoising, wherein the parameter is set to be w to be 2, and the variance sigma is set to be [ 20.1 ], so as to obtain final accurate boundary information.
the cell rapid grading identification module is used for identifying the segmented image and distinguishing the segmented image into single cells or cell clusters; carrying out hierarchical identification on single cells by adopting a double-current convolutional neural network in the additional knowledge field and a constructed cell knowledge map; identifying the non-separable cell clusters by adopting a double-current convolution neural network model of the cell clusters;
As shown in fig. 3, the specific implementation steps are as follows:
Step 1) preprocessing the segmented cervical cell image;
and scanning the segmented cervical cell region, filling cell boundary pixel values, filling pixel values outside the cell boundary to be 0, and then uniformly normalizing the cell image filled with the pixel values to 256 × 256 pixel values.
step 2) judging whether the cell image after pretreatment is a single cell, if so, switching to step 3), otherwise, switching to step 7 if the image is an inseparable cell cluster;
and (3) realizing by using a watershed algorithm in image processing, and judging that the cell is a single cell if the number of cell nucleuses in the cell image is 1.
Step 3) determining calculable cell parameters including the size, depth and shape of cell nucleus, the size and shape of cytoplasm and nuclear plasma ratio, and then calculating the characteristics of the cell parameters;
taking the size of the cell nucleus as an example, the size parameter of the cell nucleus is represented by directly calculating the sum of pixels within the boundary of the cell nucleus region:
Wherein f (x, y) is the pixel value of a certain point (x, y) on the binary image, the pixel point belongs to the target area when the value is 1, the pixel point belongs to the background area when the value is 0, and the area is the number of pixels of which f (x, y) is 1.
Step 4) establishing a cell knowledge map inference judgment model, and inputting cell parameter characteristics into the model to obtain a first grading result of cells;
step 5) constructing a double-current convolutional neural network model with additional domain knowledge, and inputting cell parameter characteristics and cell images into the double-current convolutional neural network model to obtain a second grading result of the cells;
As shown in fig. 2, one input of the dual-flow convolutional neural network is the cell parameter characteristics obtained in step 3), the other input is a single cell image, the size of the single cell image is uniformly normalized to 256 × 256 pixel values, and the characteristics of the cell image are implicitly extracted through 5 cascaded convolutional pooling modules. The convolution kernel size of the most important convolution operation is 7 × 7, the step size is selected to be 1, the number of feature maps is selected to be 96, and the convolution operation is as follows:
m represents a set of selected input feature maps, wijrepresents a weight, bjAn additional bias is added to each feature map output, and the extracted 1096-dimensional features are then spliced together with 20-dimensional features that can be computed from cell domain knowledge and input to the fully-connected and classified layers of the dual-flow convolutional neural network. Based on TBS criteria diagnosis, the fractional identification of different cells was combined together, totaling 9 classes.
As shown in fig. 3, the cytological features are based solely on the TBS criteria, and the extraction is of the language that uses the criteria. Cells can be classified in different abstract levels, and from the level of existence of pathological changes, the cells are mainly divided into two categories, namely normal cells and abnormal cells, wherein the normal cells comprise columnar cells, middle-layer cells and surface-layer cells; abnormal cells include mild squamous intraepithelial lesion cells, moderate squamous intraepithelial lesion cells, severe squamous intraepithelial lesion cells, squamous cell carcinoma cells.
the interpretation rule base is also completely based on the interpretation process and thought of the reader, such as the mapping of the color characteristics of cytoplasm, wherein the colors having interpretation significance for cervical lesions are blue, pink and orange, and the cytological terms are usually called basophilic, eosinophilic and orange, i.e. basophilic cytoplasm is blue, eosinophilic cytoplasm is pink and orange.
step 6), constructing a double-current convolution neural network model of the cell clusters, and carrying out hierarchical identification on cluster cells of the undifferentiated cell clusters by using the model;
one input of the double-current convolutional neural network of the cell clusters is as follows: the invention unifies and normalizes the input size of the cervical cells into 512 by 512 pixel values, and implicitly extracts the characteristics of the cell image through 8 cascaded convolution pooling combination modules. The convolution kernel size of the most important convolution operation is 5 × 5, the step size is selected to be 2, and the number of feature maps is selected to be 108.
the interpretation and post-processing module is used for carrying out combined interpretation on the first grading result and the second grading result of the single cell and carrying out conflict processing to obtain the grading result of the single cell; readability of the cervical cell identification process and interpretability of the cervical cell identification result are achieved by using a knowledge map and a CAM (class activity Mapping) method.
for example, if the first and second classification results for a cell are both the result of squamous cell carcinoma, then the cell is interpreted as squamous cell carcinoma.
The conflict processing mainly solves the problem that when various characteristics point to different interpretation results, various factors are integrated, conflicts are eliminated, and clear and reliable interpretation is made. For example, a cell may be classified as squamous cell carcinoma by a first classification (a dual-flow convolutional neural network model) and as squamous intraepithelial lesion of a lower classification by a second classification (a knowledge mapping model), which is a conflict of results.
the above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are merely exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (7)
1. an artificial intelligence assisted slide system for a cervical fluid-based slide, the system comprising:
the cell image acquisition module is used for scanning and storing cell images in an overlapped mode by using the automatic smear scanner;
The cell image preprocessing module is used for preprocessing the cell image;
The cell image detection and segmentation module is used for automatically detecting different cell components of the image cell and automatically segmenting a cell nucleus, a cell cytoplasm and a background in the same cell component; adopting an improved active contour model and a fast regional convolution neural network for detection, and assisting with cell characteristics and a multi-scale horizontal algorithm to finely adjust and optimize the segmentation result;
the cell rapid grading identification module is used for identifying the segmented image and distinguishing the segmented image into single cells or cell clusters; respectively carrying out hierarchical identification on single cells by adopting a double-current convolutional neural network with additional domain knowledge and a constructed cell knowledge graph to respectively obtain a first hierarchical result and a second hierarchical result; identifying the non-separable cell clusters by adopting a double-current convolution neural network model of the cell clusters; and
The interpretation and post-processing module is used for carrying out combined interpretation on the first grading result and the second grading result of the single cell and carrying out conflict processing to obtain the grading result of the single cell; the conflict processing is used for solving the problem that when various characteristics point to different interpretation results, various factors are integrated, conflicts are eliminated, and clear and reliable interpretation is made; then, readability of the cervical cell identification process and interpretability of the cervical cell identification result are achieved by means of the knowledge map and the class activity mapping;
One path of the double-current convolutional neural network with additional domain knowledge inputs cell parameter characteristics, the other path of the double-current convolutional neural network inputs single cell images, the cell images are unified and normalized to 256 × 256 pixel values in size, and the cell image characteristics are implicitly extracted through 5 cascaded convolutional pooling combined modules; the convolution kernel size of the most important convolution operation is 7 × 7, the step size is selected to be 1, the number of feature maps is selected to be 96, and the convolution operation is as follows:
In the above formula, M represents a set of selected input feature maps, wijRepresents a weight, bjoutputting an additional bias added to each feature map, splicing the extracted 1096-dimensional features and 20-dimensional features which can be calculated by cell domain knowledge, and inputting the spliced 20-dimensional features to a full connection layer and a classification layer of the double-current convolutional neural network;
One input of the double-current convolution neural network of the cell clusters is as follows: the method comprises the following steps that (1) the cell nuclei are regularly arranged, the other path of input is cervical cells of cell clusters corresponding to cell parameters, the input sizes of the cervical cells are uniformly normalized to 512 x 512 pixel values, and the cell image characteristics are implicitly extracted through 8 cascaded convolution pooling combined modules; the convolution kernel size of the most important convolution operation is 5 × 5, the step size is 2, and the number of feature maps is 108.
2. the artificial intelligent auxiliary cervical cell liquid-based smear reading system according to claim 1, wherein the cell image acquisition module adopts an eyepiece for forty times magnification, the scanning path is rectangular, and the scanning mode is overlapped scanning, so that the scanning range and the range of the liquid-based smear cells are completely covered.
3. The cervical cytosol-based smear artificial intelligence assisted interpretation system of claim 1, wherein said preprocessing module comprises: the image is denoised by adopting a bilateral filter, wherein the filter consists of two functions: one function determines the filter coefficients from the geometric spatial distance and the other determines the filter coefficients from the pixel difference; the edges of the image are then patched using morphological processing, filling the holes and removing fine junctions, and finally histogram equalization is used to increase the contrast of the nucleus to the cytoplasm.
4. The system for auxiliary reading of the cervical cell fluid-based smear through artificial intelligence according to claim 1, wherein the cell image detection and segmentation module is implemented by the following steps:
step S1), carrying out foreground and background rough segmentation on the preprocessed cell image, and extracting the region to which the cell belongs;
Step S2) detecting and segmenting cell components of the roughly segmented cell image, and segmenting cells of different types by using a fast regional convolution neural network;
Step S3) detecting and segmenting the cervical cell nuclei;
step S4) screening cell nucleuses according to the characteristic parameters of the cell nucleuses to obtain final candidate cell nucleuses;
Step S5), judging whether the cell type obtained in the step S2) is a cell cluster, if not, using the active contour model and the prior template to segment the cytoplasm region; otherwise, go to step S6);
Step S6) the segmentation results of the cell nucleus and the cytoplasm and the domain knowledge are integrated for post-processing, and the effective segmentation of the whole cervical cell is completed.
5. the artificial intelligence-aided cervical cytosol-based smear interpretation system according to claim 4, wherein the fast area convolutional neural network adopts a network structure of VGG16 of convolutional neural network, the size of the input image is 515 × 512, and the detection categories of the final cell components are classified into 5 categories: squamous cells, glandular cells, cervical cells, metaplastic cells, and background diathesis, the cellular components excluding squamous cells and background diathesis are defined as non-separable cell clusters.
6. The artificial intelligence assisted interpretation system for cervical fluid-based smear according to claim 4, wherein the specific process of segmenting the cytoplasmic region by using the active contour model and the prior template in the step S5) is as follows:
an improved active contour model is adopted, an energy function and shape prior information are added, contour optimization is carried out iteratively, and accurate boundaries of cytoplasm are obtained;
the energy function E (u) is:
E(u)＝λ1Es(u)+R(u)
wherein Es(u) is the shape prior, R (u) is a regularization term to ensure smoothness of the segmentation boundary, λ1is a learnable parameter; shape prior Es(u) is:
Wherein H is a blackplug matrix.
7. the system for auxiliary reading of the cervical cell fluid-based smear through artificial intelligence according to claim 1, wherein the rapid cell grading identification module is implemented by the following steps:
step 1) preprocessing the segmented cervical cell image;
Step 2) judging whether the cell image after pretreatment is a single cell, if so, switching to step 3), otherwise, switching to step 6 if the image is an inseparable cell cluster;
Step 3) determining a calculable cell parameter, and then calculating cell parameter characteristics;
Step 4) establishing a cell knowledge map inference judgment model, and inputting cell parameter characteristics into the model to obtain a first grading result of a single cell;
Step 5) constructing a double-current convolutional neural network model with additional domain knowledge, and inputting cell parameter characteristics and cell images into the double-current convolutional neural network model to obtain a second grading result of a single cell;
And 6) constructing a double-current convolutional neural network model of the cell clusters, and carrying out hierarchical identification on the cell clusters of the undifferentiated cell clusters by using the model to obtain a hierarchical result of the cell clusters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710351064.XA CN107274386B (en) | 2017-05-18 | 2017-05-18 | artificial intelligent auxiliary cervical cell fluid-based smear reading system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710351064.XA CN107274386B (en) | 2017-05-18 | 2017-05-18 | artificial intelligent auxiliary cervical cell fluid-based smear reading system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107274386A CN107274386A (en) | 2017-10-20 |
| CN107274386B true CN107274386B (en) | 2019-12-17 |
Family
ID=60064694
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710351064.XA Active CN107274386B (en) | 2017-05-18 | 2017-05-18 | artificial intelligent auxiliary cervical cell fluid-based smear reading system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107274386B (en) |
Families Citing this family (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107995521A (en) * | 2017-12-07 | 2018-05-04 | 南京熊猫电子制造有限公司 | A kind of medical display all-in-one machine control system for being used for intelligent diagosis |
| CN108389198A (en) * | 2018-02-27 | 2018-08-10 | 深思考人工智能机器人科技（北京）有限公司 | The recognition methods of atypia exception gland cell in a kind of Cervical smear |
| CN108334860A (en) * | 2018-03-01 | 2018-07-27 | 北京航空航天大学 | The treating method and apparatus of cell image |
| CN109690562B (en) * | 2018-05-18 | 2022-09-13 | 香港应用科技研究院有限公司 | Image pre-processing to accelerate cytological image classification by full convolution neural network |
| US10586336B2 (en) | 2018-05-18 | 2020-03-10 | Hong Kong Applied Science and Technology Research Institute Company Limited | Image pre-processing for accelerating cytological image classification by fully convolutional neural networks |
| CN109190441B (en) * | 2018-06-21 | 2022-11-08 | 丁彦青 | Intelligent classification method, diagnostic instrument and storage medium for female genital tract cell pathology |
| CN108982500B (en) * | 2018-07-03 | 2020-07-14 | 怀光智能科技（武汉）有限公司 | Intelligent auxiliary cervical fluid-based cytology reading method and system |
| GB201812050D0 (en) | 2018-07-24 | 2018-09-05 | Dysis Medical Ltd | Computer classification of biological tissue |
| CN109272492B (en) * | 2018-08-24 | 2022-02-15 | 深思考人工智能机器人科技（北京）有限公司 | Method and system for processing cytopathology smear |
| CN109815888B (en) * | 2019-01-21 | 2021-02-09 | 武汉兰丁智能医学股份有限公司 | Novel Pasteur staining method-based abnormal cervical cell automatic identification method |
| CN110189310B (en) * | 2019-05-24 | 2022-03-25 | 上海联影智能医疗科技有限公司 | Image characteristic value acquisition method, computer device and storage medium |
| CN110231259B (en) * | 2019-05-28 | 2022-02-18 | 怀光智能科技（武汉）有限公司 | Digital diagnosis system for cervical cell slide |
| CN110736747B (en) * | 2019-09-03 | 2022-08-19 | 深思考人工智能机器人科技（北京）有限公司 | Method and system for positioning under cell liquid-based smear mirror |
| CN112257681A (en) * | 2019-12-06 | 2021-01-22 | 珠海圣美生物诊断技术有限公司 | Cell interpretation method and system |
| CN111429761B (en) * | 2020-02-28 | 2022-10-21 | 中国人民解放军陆军军医大学第二附属医院 | Artificial intelligent simulation teaching system and method for bone marrow cell morphology |
| CN111785364A (en) * | 2020-06-15 | 2020-10-16 | 杭州思柏信息技术有限公司 | Internet and cervical image intelligent auxiliary film reading method and auxiliary film reading system |
| CN112102277A (en) * | 2020-09-10 | 2020-12-18 | 深圳市森盈生物科技有限公司 | Device and method for detecting tumor cells in pleural fluid fluorescence image |
| CN112396583B (en) * | 2020-11-18 | 2024-01-26 | 深思考人工智能机器人科技（北京）有限公司 | DNA ploidy quantitative analysis method and system based on Papanicolaou staining mode |
| CN112884663B (en) * | 2021-01-18 | 2023-11-21 | 北京晶科瑞医学检验实验室有限公司 | Method for identifying and dividing cell boundaries aiming at tissue mass spectrum imaging result |
| CN113052806B (en) * | 2021-03-15 | 2023-02-28 | 黑龙江机智通智能科技有限公司 | Canceration degree grading system |
| CN113111926A (en) * | 2021-03-31 | 2021-07-13 | 南京华晟医学检验实验室有限公司 | Abnormal cervical blood cell screening method based on TCT (TCT) slide |
| CN113222911A (en) * | 2021-04-26 | 2021-08-06 | 清华大学深圳国际研究生院 | Cervical cell image screening method, cervical cell image screening system, computer equipment and storage medium |
| CN113256577B (en) * | 2021-05-18 | 2022-06-28 | 湖南医药学院 | Cancer auxiliary analysis system and device based on HE staining pathological image |
| CN113420653A (en) * | 2021-06-22 | 2021-09-21 | 深圳跃美生物医学科技有限公司 | Method, device and readable medium for comprehensive detection and analysis of bacterial cell morphology |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1106246A (en) * | 1993-11-01 | 1995-08-09 | 波拉技术有限公司 | Method and apparatus for tissue type recognition |
| WO2007059119A2 (en) * | 2005-11-11 | 2007-05-24 | Ivan Borozan | Systems and methods for identifying diagnostic indicators |
| CN101460090A (en) * | 2006-06-05 | 2009-06-17 | 福思光子学有限公司 | Methods for characterizing tissues |
| CN105894490A (en) * | 2015-11-05 | 2016-08-24 | 广西师范大学 | Fuzzy integration multiple classifier integration-based uterine neck cell image identification method and device |
-
2017
- 2017-05-18 CN CN201710351064.XA patent/CN107274386B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1106246A (en) * | 1993-11-01 | 1995-08-09 | 波拉技术有限公司 | Method and apparatus for tissue type recognition |
| WO2007059119A2 (en) * | 2005-11-11 | 2007-05-24 | Ivan Borozan | Systems and methods for identifying diagnostic indicators |
| CN101460090A (en) * | 2006-06-05 | 2009-06-17 | 福思光子学有限公司 | Methods for characterizing tissues |
| CN105894490A (en) * | 2015-11-05 | 2016-08-24 | 广西师范大学 | Fuzzy integration multiple classifier integration-based uterine neck cell image identification method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107274386A (en) | 2017-10-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107274386B (en) | artificial intelligent auxiliary cervical cell fluid-based smear reading system | |
| Salvi et al. | The impact of pre-and post-image processing techniques on deep learning frameworks: A comprehensive review for digital pathology image analysis | |
| CN107256558B (en) | Unsupervised automatic cervical cell image segmentation method and system | |
| CN108364288B (en) | Segmentation method and device for breast cancer pathological image | |
| Pan et al. | Accurate segmentation of nuclei in pathological images via sparse reconstruction and deep convolutional networks | |
| EP3455782B1 (en) | System and method for detecting plant diseases | |
| Hayakawa et al. | Computational nuclei segmentation methods in digital pathology: a survey | |
| Kowal et al. | Cell nuclei segmentation in cytological images using convolutional neural network and seeded watershed algorithm | |
| Valkonen et al. | Metastasis detection from whole slide images using local features and random forests | |
| Shahin et al. | A novel white blood cells segmentation algorithm based on adaptive neutrosophic similarity score | |
| Jia et al. | Multi-layer segmentation framework for cell nuclei using improved GVF Snake model, Watershed, and ellipse fitting | |
| Pan et al. | Cell detection in pathology and microscopy images with multi-scale fully convolutional neural networks | |
| Sajeena et al. | Automated cervical cancer detection through RGVF segmentation and SVM classification | |
| Wang et al. | A hybrid method for the segmentation of a ferrograph image using marker-controlled watershed and grey clustering | |
| He et al. | A review: The detection of cancer cells in histopathology based on machine vision | |
| Zheng et al. | A novel algorithm based on visual saliency attention for localization and segmentation in rapidly-stained leukocyte images | |
| Wang et al. | Nucleus segmentation of cervical cytology images based on depth information | |
| Salvi et al. | Automatic discrimination of neoplastic epithelium and stromal response in breast carcinoma | |
| Somasundaram et al. | Automatic segmentation of nuclei from pap smear cell images: A step toward cervical cancer screening | |
| Han et al. | Segmenting images with complex textures by using hybrid algorithm | |
| WO2014006421A1 (en) | Identification of mitotic cells within a tumor region | |
| He et al. | Multiphase level set model with local K-means energy for histology image segmentation | |
| US11790673B2 (en) | Method for detection of cells in a cytological sample having at least one anomaly | |
| He et al. | Progress of machine vision in the detection of cancer cells in histopathology | |
| Saxena et al. | Study of Computerized Segmentation & Classification Techniques: An Application to Histopathological Imagery |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20171020 Assignee: Suzhou deep thinking Artificial Intelligence Technology Co.,Ltd. Assignor: IDEEPWISE ARTIFICIAL INTELLIGENCE ROBOT TECHNOLOGY (BEIJING) CO.,LTD. Contract record no.: X2022980003658 Denomination of invention: An artificial intelligence assisted film reading system for cervical cell liquid based smear Granted publication date: 20191217 License type: Common License Record date: 20220401 |
|
| EE01 | Entry into force of recordation of patent licensing contract |