CN112036298A - Cell detection method based on double-segment block convolutional neural network - Google Patents
Cell detection method based on double-segment block convolutional neural network Download PDFInfo
- Publication number
- CN112036298A CN112036298A CN202010889738.3A CN202010889738A CN112036298A CN 112036298 A CN112036298 A CN 112036298A CN 202010889738 A CN202010889738 A CN 202010889738A CN 112036298 A CN112036298 A CN 112036298A
- Authority
- CN
- China
- Prior art keywords
- network
- block
- segment
- representing
- roi
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
- G06V20/698—Matching; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a cell detection method based on a two-section block convolutional neural network, which comprises the steps of constructing a convolutional neural network model based on deep learning; extracting the representation of the input data in a high-dimensional nonlinear space by using the neural network model, and inputting the representation into a two-segment block network; constructing an area block proposal network by using the two-section block network, and classifying and screening a section of block; learning two segments of blocks of the two-segment block network based on the one segment of block classification and fitting, and adjusting by using an attention mechanism; and outputting the cell detection result with high confidence by using a high-performance classifier according to the adjustment result to finish the detection. The invention can efficiently realize cell detection, combine the cell detection with the gradually developed artificial intelligence technology, and can effectively identify and improve the detection precision.
Description
Technical Field
The invention relates to the technical field of cell detection, in particular to a cell detection method based on a two-segment block convolutional neural network.
Background
In recent years, with the development of the field of artificial intelligence in intelligent image processing, intelligent medical image processing has been rapidly and effectively developed in recent years. Since medical images have higher specialization than traditional natural images, models and methods that directly utilize natural image processing often fail to achieve effective recognition and detection accuracy.
The medical image processing objects are medical images of various imaging mechanisms, and the clinical widely used medical imaging categories mainly include four categories of X-ray imaging (X-CT), Magnetic Resonance Imaging (MRI), Nuclear Medicine Imaging (NMI) and Ultrasonic Imaging (UI). In current medical imaging diagnosis, the pathological changes are mainly discovered by observing a group of two-dimensional slice images, which often needs to be determined by the experience of doctors. The two-dimensional slice image is analyzed and processed by using a computer image processing technology, so that segmentation extraction, three-dimensional reconstruction and three-dimensional display of human organs, soft tissues and pathological variants are realized, and qualitative and even quantitative analysis of pathological change bodies and other interested areas can be assisted by doctors, so that the accuracy and reliability of medical diagnosis are greatly improved; can also play an important auxiliary role in medical teaching, operation planning, operation simulation and various medical researches [1,2 ]. At present, medical image processing mainly focuses on four aspects of lesion detection, image segmentation, image registration and image fusion.
Data analysis by a deep learning method shows a rapid growth trend, which is called one of 10 breakthrough technologies in 2013. Deep learning is an improvement of artificial neural networks, consisting of more layers, allowing higher layers to contain more abstract information for data prediction. To date, it has become a leading machine learning tool in the field of computer vision, the deep neural network learning of intermediate and high level abstract features automatically obtained from raw data (images). Recent results show that the information extracted from CNN is very effective in target recognition and localization in natural images. Medical image processing institutions around the world have rapidly entered the field and applied CNN and other deep learning methods to various medical image analyses.
Disclosure of Invention
This section is for the purpose of summarizing some aspects of embodiments of the invention and to briefly introduce some preferred embodiments. In this section, as well as in the abstract and the title of the invention of this application, simplifications or omissions may be made to avoid obscuring the purpose of the section, the abstract and the title, and such simplifications or omissions are not intended to limit the scope of the invention.
The present invention has been made in view of the above-mentioned problem that the existing models and methods that directly use natural image processing often cannot obtain effective recognition and detection accuracy.
Therefore, the technical problem solved by the invention is as follows: the traditional natural image detection method considers the natural attributes of the object as class information, so that the effect in cell detection application is poor.
In order to solve the technical problems, the invention provides the following technical scheme: constructing a convolutional neural network model based on deep learning; extracting the representation of the input data in a high-dimensional nonlinear space by using the neural network model, and inputting the representation into a two-segment block network; constructing an area block proposal network by using the two-section block network, and classifying and screening a section of block; learning two segments of blocks of the two-segment block network based on the one segment of block classification and fitting, and adjusting by using an attention mechanism; and outputting the cell detection result with high confidence by using a high-performance classifier according to the adjustment result to finish the detection.
As a preferable embodiment of the cell detection method based on the two-segment block convolutional neural network of the present invention, wherein: the convolutional neural network model comprises a model of,
{I1,I2,I3,I4}＝Aug(I)
wherein, I represents an input image of RGB three channels, and Aug represents a data expansion method based on cell data;
x＝fbackbone(I)
wherein f isbackboneRepresenting the feature extraction network used.
As a preferable embodiment of the cell detection method based on the two-segment block convolutional neural network of the present invention, wherein: the data expansion method of the cell data comprises horizontal folding, vertical folding, in-plane rotation and color enhancement/weakening.
As a preferable embodiment of the cell detection method based on the two-segment block convolutional neural network of the present invention, wherein: the feature extraction network comprises a plurality of convolutional layers, an activation function, a normalization layer and a pooling layer.
As a preferable embodiment of the cell detection method based on the two-segment block convolutional neural network of the present invention, wherein: the area block proposal network comprises a network of area blocks,
zPRN＝fRPN(x)
where x denotes the input multi-channel feature map, fRPNRepresenting a block proposal network, zRPNRepresenting the corresponding network output.
As a preferable embodiment of the cell detection method based on the two-segment block convolutional neural network of the present invention, wherein: the classifying and fitting learning two-stage block comprises,
zROI＝fROI(zRPN)
wherein z isRPNRepresenting the filtered block feature input, fROIRepresenting a two-segment block classification and fitting network, zROIAnd representing the output candidate feature map.
As a preferable embodiment of the cell detection method based on the two-segment block convolutional neural network of the present invention, wherein: the high-performance classifier includes a plurality of classifiers,
SROI,PredROI＝fFinal(zROI)
wherein z isROICandidate feature graph representing an input, fFinalRepresenting fully connected network classifiers, SROIClass score, Pred, representing the classifier outputROIA rectangular box prediction representing the detected position.
As a preferable embodiment of the cell detection method based on the two-segment block convolutional neural network of the present invention, wherein: the network learning stage comprises defining classification errors and positioning errors corresponding to the one-segment and two-segment block convolutional networks, namely a crossed entropy function and a position four-variable-based l1Norm error.
As a preferable embodiment of the cell detection method based on the two-segment block convolutional neural network of the present invention, wherein: the cell detection loss function includes a function of,
wherein the content of the first and second substances,
indicating the classification error of a segment of the area proposal network,
representing the regression error of a segment of the regional proposal network,
representing the classification error of the two-segment network,
representing the regression error of the two-segment network.
As a preferable embodiment of the cell detection method based on the two-segment block convolutional neural network of the present invention, wherein: the training model comprises a database model which comprises training data, test data and data outside the database; the detection algorithm model comprises a training model and a testing model.
The invention has the beneficial effects that: cell detection is efficiently realized and combined with the gradually developed artificial intelligence technology, and effective identification and detection precision improvement can be achieved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise. Wherein:
FIG. 1 is a basic flowchart of a cell detection method based on a two-segment block convolutional neural network according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a cell detection network of a cell detection method based on a two-segment block convolutional neural network according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a convolutional neural network feature extraction network of a cell detection method based on a two-segment block convolutional neural network according to an embodiment of the present invention;
fig. 4 is a schematic sub-network diagram of a convolutional neural network feature extraction network Conv1 of a cell detection method based on a two-segment block convolutional neural network according to an embodiment of the present invention;
fig. 5 is a schematic sub-network diagram of a convolutional neural network feature extraction network Conv2 of a cell detection method based on a two-segment block convolutional neural network according to an embodiment of the present invention;
fig. 6 is a schematic sub-network diagram of a convolutional neural network feature extraction network Conv3 of a cell detection method based on a two-segment block convolutional neural network according to an embodiment of the present invention;
fig. 7 is a schematic sub-network diagram of a convolutional neural network feature extraction network Conv4 of a cell detection method based on a two-segment block convolutional neural network according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of an area block proposal network of a cell detection method based on a two-segment block convolutional neural network according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of two-segment block classification and fitting learning of a cell detection method based on a two-segment block convolutional neural network according to an embodiment of the present invention;
FIG. 10 is a diagram illustrating a high-performance classifier of a cell detection method based on a two-segment block convolutional neural network according to an embodiment of the present invention;
fig. 11 is a schematic diagram illustrating a cell detection method result of a cell detection method based on a two-segment block convolutional neural network according to an embodiment of the present invention;
FIG. 12 is a schematic diagram of an anchor point of a cell detection method based on a dual-segment block convolutional neural network according to an embodiment of the present invention;
FIG. 13 is a schematic diagram of a rectangular frame regression of a cell detection method based on a two-segment block convolutional neural network according to an embodiment of the present invention;
FIG. 14 is a schematic diagram illustrating ROI pooling of a fusion attention mechanism of a cell detection method based on a two-segment block convolutional neural network according to an embodiment of the present invention;
fig. 15 is a schematic diagram of a database model and a detection algorithm model of a cell detection method based on a two-segment block convolutional neural network according to an embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, specific embodiments accompanied with figures are described in detail below, and it is apparent that the described embodiments are a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making creative efforts based on the embodiments of the present invention, shall fall within the protection scope of the present invention.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways than those specifically described and will be readily apparent to those of ordinary skill in the art without departing from the spirit of the present invention, and therefore the present invention is not limited to the specific embodiments disclosed below.
Furthermore, reference herein to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one implementation of the invention. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments.
The present invention will be described in detail with reference to the drawings, wherein the cross-sectional views illustrating the structure of the device are not enlarged partially in general scale for convenience of illustration, and the drawings are only exemplary and should not be construed as limiting the scope of the present invention. In addition, the three-dimensional dimensions of length, width and depth should be included in the actual fabrication.
Meanwhile, in the description of the present invention, it should be noted that the terms "upper, lower, inner and outer" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of describing the present invention and simplifying the description, but do not indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation and operate, and thus, cannot be construed as limiting the present invention. Furthermore, the terms first, second, or third are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
The terms "mounted, connected and connected" in the present invention are to be understood broadly, unless otherwise explicitly specified or limited, for example: can be fixedly connected, detachably connected or integrally connected; they may be mechanically, electrically, or directly connected, or indirectly connected through intervening media, or may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
Example 1
In medical imaging, accurate diagnosis and assessment of disease depends on the acquisition of medical images and image interpretation. In recent years, image acquisition has improved significantly, with devices acquiring data at faster rates and higher resolutions. However, image interpretation processes have only recently begun to benefit from computer technology. Medical image interpretation is mostly performed by physicians, however, medical image interpretation is limited by physician subjectivity, large differential cognition by physicians, and fatigue. The cell detection technology plays a fundamental diagnosis role in the current medical image processing, but how to efficiently realize cell detection and combine the cell detection with the gradually-developed artificial intelligence technology is the focus of research at the present stage.
Referring to fig. 1 to 15, a basic flow of a cell detection method based on a two-segment block convolutional neural network is provided as a first embodiment of the present invention, and includes:
s1: the convolutional neural network model is constructed based on deep learning, and it is to be noted that the convolutional neural network model comprises,
{I1,I2,I3,I4}＝Aug(I)
wherein, I represents an input image of RGB three channels, and Aug represents a data expansion method based on cell data;
x＝fbackbone(I)
wherein f isbackboneRepresenting the feature extraction network used.
Specific data expansion methods for cellular data include horizontal folding, vertical folding, in-plane rotation, and color enhancement/reduction.
The feature extraction network comprises a plurality of convolutional layers, an activation function, a normalization layer and a pooling layer.
S2: extracting the representation of the input data in a high-dimensional nonlinear space by using a neural network model, inputting the representation into a two-segment block network, wherein the area block proposal network comprises,
zPRN＝fRPN(x)
where x denotes the input multi-channel feature map, fRPNRepresenting a block proposal network, zRPNRepresenting the corresponding network output.
S3: a two-segment block network is used to construct an area block proposal network, classify and screen a segment of block, wherein the specific steps comprise,
FIG. 2 shows a two-segment block-based convolutional neural network architecture for cell detection, which first scales an arbitrary-sized color image to a fixed size of 800 × 600, and then feeds the scaled image into the network; as shown in fig. 3, the feature extraction neural network includes a plurality of conv modules (as shown in fig. 4 to 7, original image information is mapped to a high-dimensional nonlinear feature space by using a deep learning technique; as shown in fig. 8, a region proposal network is firstly convolved by 3x3, then offset of an effective anchor point and a corresponding position frame body are respectively generated, then an effective proposal is calculated, as shown in fig. 9, the effective proposal is projected to a representation form with the same spatial resolution by using a ROI pooling method, and then final classification and estimation of the position frame body are realized by using a multilayer fully-connected network and a softmax layer, and the result is shown in fig. 11.
The classic detection method consumes excessive calculation amount in generating a detection frame, such as a sliding window and an image pyramid used by adaboost to generate the detection frame; for example, the R-CNN method uses an SS (selective search) method to generate the detection frame, whereas the two-segment block neural network abandons the conventional sliding window and SS methods, and directly uses the anchor point and the proposal network to generate the detection frame, which is also a great advantage for the cell detection task with complex diversity scenarios, and can greatly increase the generation speed of the detection frame.
To avoid the computational burden of sliding windows, anchor points are used to describe the potential windows in the whole image region, so as to implement multi-scale setup, as shown in fig. 12, it is proposed to scofflate the cell location of each feature space with 9 anchor points, corresponding to 3 different scales and 3 different aspect ratios respectively:
scale＝[8，16，32]
ratio＝[0.5,1,2]
after the original image is scaled to 800 × 600, the feature extraction network down-samples the original pixel size to 50 × 38 via the receptive field, so that the total number of anchor points generated is 17100.
As shown in fig. 13, in the actual cell detection task, red is the true position (Ground Truth) of the cell, and green is the extracted anchor box, but because the green anchor box is not accurately located, it is necessary to use a regression method to further predict the offset from the green anchor to the true red value, and the box is generally represented by using a four-dimensional vector: (x, y, w, h) respectively corresponding to the center horizontal and vertical coordinates, the width and the length of the frame body. Given the coordinate representation of the anchor point and the true value, respectively, (x)a,ya,wa,ha) And (x)g,yg,wg,hg) The task of the fitting is to find a transformation fREGSo that
Wherein
When the input anchor point is slightly different from the real value, the transformation can be regarded as a linear transformation, and then the window can be subjected to fine adjustment by modeling through linear regression.
And (3) respectively obtaining corresponding foreground and background classification results and frame position deviation results by all anchor points through a regional proposal network, equivalently preliminarily extracting results of the detection target candidate region, tracking foreground probability to obtain 6000 proposals, and entering two-stage detection classification and fitting learning.
S4: learning two-segment blocks of the two-segment block network based on one-segment block classification and fitting, and adjusting by using an attention mechanism, it is to be noted that the classification and fitting learning two-segment blocks comprise,
zROI＝fROI(zRPN)
wherein z isRPNRepresenting the filtered block feature input, fROIRepresenting a two-segment block classification and fitting network, zROIAnd representing the output candidate feature map.
Specifically, the network learning stage includes defining classification errors and positioning errors corresponding to one-segment and two-segment block convolutional networks, namely a cross entropy function and l based on position four variables1Norm error.
It should be noted that the training model of the cell detection method based on the two-segment block convolutional neural network is mainly divided into two parts: the database model and the detection algorithm model (as shown in fig. 15) are arranged in the database after preprocessing cell data collected and labeled by a professional doctor, the cell data are input into a training model of the detection algorithm, and the classification and the position of related cells are judged through an advanced two-stage detection positioning algorithm.
Specifically, the database model is divided into training data, test data and extralibrary data; the training data and the test data are from received pictures with labeled cells, wherein the training data is used for training a detection algorithm model, the test data is used for evaluating the trained model, the training data and the test data are generated and grouped in a non-overlapping mode, the out-of-library data is a new picture generated in an actual application environment, and the classified detection of the cells is realized through established detection codes.
The detection algorithm model is divided into a training model and a testing model. The training model is a module for realizing algorithm functions through a deep neural network based on a specific cell detection classification task, a loss function associated with data and parameter design in a learning stage, and the testing model is a module consisting of the trained deep neural network based on the specific cell detection classification task, data preprocessing and result presentation.
In the training phase, the cell detection loss function constructed according to the network structure is as follows:
wherein
Representing a classification error of a section of regional proposal network, namely judging whether an image block corresponding to an anchor point is a foreground region;
expressing the regression error of a section of regional proposal network, namely measuring the difference between the anchor point and the real target frame position;
the classification error of the two-segment network is represented, which is different from the one-segment classification task, and the specific category of the current cell needs to be further judged;
representing the regression error of the two-segment network. The above classification error and regression error are respectively utilized with a cross entropy function and l1Norm is defined and trained for back propagation.
In this stage, it is necessary to determine whether the target is a foreground (cell) and to further obtain a corresponding classification result, so that it is necessary to perform a dimension fixing operation before each proposal is input into the two-segment network, and in combination with an attention mechanism, as shown in fig. 14, channel attention weighting and spatial attention weighting are performed on the high-dimensional feature representation of the proposal part, so as to highlight the apparent correlation of a specific depth channel to a specific cell type, and further enhance the learning of the difference between multiple types of cells. The spatial resolution of each proposal in the original feature input x is scaled to 7 × 7 by ROI pooling, and then max boosting is performed on each grid in 7 × 7 to obtain the final proposal representation zROI。
S5: outputting a cell detection result with high confidence by using a high-performance classifier according to the adjustment result to finish the detection, wherein the high-performance classifier comprises,
SROI,PredROI＝fFinal(zROI)
wherein z isROICandidate feature graph representing an input, fFinalRepresenting fully connected network classifiers, SROIClass score, Pred, representing the classifier outputROIA rectangular box prediction representing the detected position.
Specifically, the obtained proposal is represented by zROIAfter being input into the high-performance classifier, the classification is performed by full connection and softmax, which is already a category actually recognized as shown in fig. 10, and the frame position is corrected again for the proposal to acquire a frame position with higher accuracy.
Example 2
In order to verify the effectiveness of the present invention, in this embodiment, a verification experiment is performed on the constructed data set, first a brief description is made on the constructed data set, and then an experiment result of the experiment is displayed.
(1) A cell assay dataset.
The collected cell examination data set contains 364 image sets, each image set corresponding to the cell display of one case. A total of 8 types of cells were discussed according to medical definition and physician labeling, respectively: lymphocytes, neutrophils, abnormal lymphocytes, tumor cells, monocytes, live lymphocytes, degenerated cells, live single cells, and the remaining cells not involved in the assay. The statistics for each cell type are shown in table 1.
Table 1 survey cell data set statistics table.
| Cell name | Number of cells | Cell label | Cell name | Number of cells | Cell label |
| Lymphocytes | 60369 | LB | Live lymphocytes | 1376 | HL |
| Neutrophils | 14839 | ZXL | Degenerated |
1750 | TH |
| Abnormal lymph | 7378 | YCLB | Viable single cell | 1007 | HD |
| Tumor cells | 7354 | ZL | Others | 2561 | QT |
| Monocyte cell | 8123 | DH | Total of | 101862 | - |
(2) And (4) preprocessing data.
For color picture input, 3-channel RGB patterns [0-255] are used for representation. Scaling the size of each image to be no more than 800 pixels and 600 pixels in length and height respectively when the network is actually input; at the same time, the input image is averaged by subtracting 122.7717, 115.9465, and 102.9801 from the RGB channels, respectively. The training process of the network adopts a random gradient descent optimizer, and the learning rate is 1 e-3.
(3) An evaluation method.
The cell detection experiment uses a standard mean Average Precision (mAP) of detection evaluation indexes. The mAP counts the mathematical expectation of the Average Precision (AP) when the Recall (Recall) is slid directly from 0 to 1. First, the precision is defined as the ratio of the number of true positive samples to the number of all positive samples: precision is TP/(TP + FP), and recall is defined as the number of true positive samples and the ratio of the number of true positive samples to the number of false negative samples: recall is TP/(TP + FN).
(4) And (5) experimental results.
17365 images were selected as training set and 1078 images were selected as verification set by randomization on the whole volume data set, and the experimental results on the verification set are shown in table 2.
Table 2 table of cell assay results.
| Type (B) | mAP |
| Lymphocyte (LB) | 0.904 |
| Neutrophils (ZXL) | 0.905 |
| Abnormal lymph (YCLB) | 0.904 |
| Tumor cells (ZL) | 0.909 |
| Mononuclear cells (DH) | 0.883 |
| Active lymphocyte (HL) | 0.717 |
| Degenerated cell (TH) | 0.655 |
| Single cell alive (HD) | 0.886 |
| General of | 0.900 |
Wherein the mAP detected on lymphocytes, neutrophils, abnormal lymphocytes and tumor cells reaches more than 90 percent, and the other four types of cells (monocytes, live lymphocytes, degenerated cells and live single cells) respectively obtain 88 percent, 72 percent, 66 percent and 89 percent of mAP due to the limitation of the number of training samples.
(5) And (4) experimental analysis.
Different module ablation comparison experiments are carried out on the same divided data sets.
As shown in Table 3, the cell data expansion method used in the present optimization protocol was compared with the detection accuracy under the condition without data expansion. Compared with a network method without data expansion, the data expansion method based on cell data in the optimization scheme can improve the cell detection precision mAP from 0.717 to 0.900, and improves the cell detection precision mAP by 18.3%.
Table 3 cell assay data expansion ablation experimental results table.
| Attention mechanism | Network scheme without data expansion | Optimization scheme incorporating cellular data expansion |
| Lymphocyte (LB) | 0.877 | 0.904 |
| Neutrophils (ZXL) | 0.882 | 0.905 |
| Abnormal lymph (YCLB) | 0.881 | 0.904 |
| Tumor cells (ZL) | 0.863 | 0.909 |
| Mononuclear cells (DH) | 0.751 | 0.883 |
| Active lymphocyte (HL) | 0.405 | 0.717 |
| Degenerated cell (TH) | 0.412 | 0.655 |
| Single cell alive (HD) | 0.661 | 0.886 |
| General of | 0.717 | 0.900 |
As shown in table 4, the spatial and channel attention based two-segment block detection network used in the present optimization scheme was compared with the detection accuracy under the single attention mechanism and the non-attention mechanism conditions. Compared with a network without a attention mechanism, the spatial attention mechanism can improve the detection accuracy mAP from 0.680 to 0.705; the channel attention mechanism can further promote the mAP to 0.718; the method for combining the space and channel attention mechanism selected in the optimization scheme can achieve the detection precision of 0.900, and compared with a network without the attention mechanism, a space attention mechanism network and a channel attention mechanism network, the detection precision is respectively improved by 6.1%, 3.6% and 18.2%.
Table 4 table of cell detection attention mechanism ablation experiment results.
(6) And (5) experimental conclusion.
The invention provides a cell detection method based on a two-segment block convolutional neural network by combining space dimension information and channel dimension information. And then, depth feature representation layer, and the foreground judgment, type judgment and space positioning of the cell target are realized by using a two-segment block network, the expression strength of the high-dimensional features in a space domain is enhanced by a space attention mechanism, and the discriminativity of different channel attributes is improved by a channel attention mechanism. Experimental results show that the method is superior to the current mainstream scheme in detection precision in comparison with a non-data expansion algorithm and in comparison with other single attention mechanism algorithms and non-attention mechanism algorithms.
Through an experimental part, the established cell detection database is preprocessed, the data of the sample is further expanded, and meanwhile, comparison based on different attention mechanism models is carried out. The present embodiment contributions can therefore be summarized as follows: a network model combining a space attention mechanism and a channel attention mechanism is innovatively used; a data expansion method suitable for a cell detection task is provided; the cell detection method based on the two-section block convolutional neural network is integrated and realized.
It should be recognized that embodiments of the present invention can be realized and implemented by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer readable memory. The methods may be implemented in a computer program using standard programming techniques, including a non-transitory computer-readable storage medium configured with the computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner, according to the methods and figures described in the detailed description. Each program may be implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Furthermore, the program can be run on a programmed application specific integrated circuit for this purpose.
Further, the operations of processes described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and/or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) collectively executed on one or more processors, by hardware, or combinations thereof. The computer program includes a plurality of instructions executable by one or more processors.
Further, the method may be implemented in any type of computing platform operatively connected to a suitable interface, including but not limited to a personal computer, mini computer, mainframe, workstation, networked or distributed computing environment, separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, and the like. Aspects of the invention may be embodied in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optically read and/or write storage medium, RAM, ROM, or the like, such that it may be read by a programmable computer, which when read by the storage medium or device, is operative to configure and operate the computer to perform the procedures described herein. Further, the machine-readable code, or portions thereof, may be transmitted over a wired or wireless network. The invention described herein includes these and other different types of non-transitory computer-readable storage media when such media include instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. The invention also includes the computer itself when programmed according to the methods and techniques described herein. A computer program can be applied to input data to perform the functions described herein to transform the input data to generate output data that is stored to non-volatile memory. The output information may also be applied to one or more output devices, such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including particular visual depictions of physical and tangible objects produced on a display.
As used in this application, the terms "component," "module," "system," and the like are intended to refer to a computer-related entity, either hardware, firmware, a combination of hardware and software, or software in execution. For example, a component may be, but is not limited to being: a process running on a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of example, both an application running on a computing device and the computing device can be a component. One or more components can reside within a process and/or thread of execution and a component can be localized on one computer and/or distributed between two or more computers. In addition, these components can execute from various computer readable media having various data structures thereon. The components may communicate by way of local and/or remote processes such as in accordance with a signal having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and/or across a network such as the internet with other systems by way of the signal).
It should be noted that the above-mentioned embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, which should be covered by the claims of the present invention.
Claims (10)
1. A cell detection method based on a two-segment block convolutional neural network is characterized by comprising the following steps:
constructing a convolutional neural network model based on deep learning;
extracting the representation of the input data in a high-dimensional nonlinear space by using the neural network model, and inputting the representation into a two-segment block network;
constructing an area block proposal network by using the two-section block network, and classifying and screening a section of block;
learning two segments of blocks of the two-segment block network based on the one segment of block classification and fitting, and adjusting by using an attention mechanism;
and outputting the cell detection result with high confidence by using a high-performance classifier according to the adjustment result to finish the detection.
2. The method of claim 1, wherein the method comprises: the convolutional neural network model comprises a model of,
{I1,I2,I3,I4}＝Aug(I)
wherein, I represents an input image of RGB three channels, and Aug represents a data expansion method based on cell data;
x＝fbackbone(I)
wherein f isbackboneRepresenting the feature extraction network used.
3. The cell detection method based on the two-segment block convolutional neural network of claim 1 or 2, wherein: the data expansion method of the cell data comprises the following steps,
horizontal folding, vertical folding, in-plane rotation, and color enhancement, weakening.
4. The method of claim 3, wherein the method comprises: the feature extraction network comprises a network of feature extractions,
a plurality of convolutional layers, an activation function, a normalization layer, and a pooling layer.
5. The method of claim 4, wherein the method comprises: the area block proposal network comprises a network of area blocks,
zPRN＝fRPN(x)
where x denotes the input multi-channel feature map, fRPNRepresenting a block proposal network, zRPNRepresenting the corresponding network output.
6. The method of claim 5, wherein the method comprises: the classifying and fitting learning two-stage block comprises,
zROI＝fROI(zRPN)
wherein z isRPNRepresenting the filtered block feature input, fROIRepresenting a two-segment block classification and fitting network, zROIAnd representing the output candidate feature map.
7. The method of claim 6, wherein the method comprises: the high-performance classifier includes a plurality of classifiers,
SROI,PredROI＝fFinal(zROI)
wherein z isROICandidate feature graph representing an input, fFinalRepresenting fully connected network classifiers, SROIClass score, Pred, representing the classifier outputROIA rectangular box prediction representing the detected position.
8. The method of claim 7, wherein the method comprises: the network learning phase comprises the steps of,
defining classification error and positioning error corresponding to the one-segment and two-segment block convolution network, namely cross entropy function and l based on position four-variable1Norm error.
9. The method of claim 8, wherein the method comprises: the cell detection loss function includes a function of,
wherein the content of the first and second substances,
indicating the classification error of a segment of the area proposal network,
representing the regression error of a segment of the regional proposal network,
representing the classification error of the two-segment network,
representing the regression error of the two-segment network.
10. The method of claim 1, wherein the method comprises: the training model comprises a training model which comprises,
the database model comprises training data, testing data and extralibrary data;
the detection algorithm model comprises a training model and a testing model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010889738.3A CN112036298A (en) | 2020-08-28 | 2020-08-28 | Cell detection method based on double-segment block convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010889738.3A CN112036298A (en) | 2020-08-28 | 2020-08-28 | Cell detection method based on double-segment block convolutional neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112036298A true CN112036298A (en) | 2020-12-04 |
Family
ID=73587173
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010889738.3A Pending CN112036298A (en) | 2020-08-28 | 2020-08-28 | Cell detection method based on double-segment block convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112036298A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112613505A (en) * | 2020-12-18 | 2021-04-06 | 安徽丹姆斯生物科技有限公司 | Cell micronucleus identification, positioning and counting method based on deep learning |
| CN112950585A (en) * | 2021-03-01 | 2021-06-11 | 中国人民解放军陆军军医大学 | Cervical cancer cell intelligent detection method based on liquid-based thin-layer cell detection technology TCT |
| CN114708484A (en) * | 2022-03-14 | 2022-07-05 | 中铁电气化局集团有限公司 | Pattern analysis method suitable for identifying defects |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109598224A (en) * | 2018-11-27 | 2019-04-09 | 微医云(杭州)控股有限公司 | Recommend white blood cell detection method in the Sections of Bone Marrow of convolutional neural networks based on region |
| CN110110719A (en) * | 2019-03-27 | 2019-08-09 | 浙江工业大学 | A kind of object detection method based on attention layer region convolutional neural networks |
-
2020
- 2020-08-28 CN CN202010889738.3A patent/CN112036298A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109598224A (en) * | 2018-11-27 | 2019-04-09 | 微医云(杭州)控股有限公司 | Recommend white blood cell detection method in the Sections of Bone Marrow of convolutional neural networks based on region |
| CN110110719A (en) * | 2019-03-27 | 2019-08-09 | 浙江工业大学 | A kind of object detection method based on attention layer region convolutional neural networks |
Non-Patent Citations (2)
| Title |
|---|
| CHAO YANG ET AL.: "CONSTRAINED R-CNN: A GENERAL IMAGE MANIPULATION DETECTION MODEL", 《2020 ICME》 * |
| MUYI SUN ET AL.: "Efficient Nucleus Detection in Digital Pathology Images using Multi-task and Multi-scale Instance Segmentation Network", 《PROCEEDINGS OF THE 2019 INTERNATIONAL CONFERENCE ON ADVANCED MECHATRONIC SYSTEMS》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112613505A (en) * | 2020-12-18 | 2021-04-06 | 安徽丹姆斯生物科技有限公司 | Cell micronucleus identification, positioning and counting method based on deep learning |
| CN112950585A (en) * | 2021-03-01 | 2021-06-11 | 中国人民解放军陆军军医大学 | Cervical cancer cell intelligent detection method based on liquid-based thin-layer cell detection technology TCT |
| CN112950585B (en) * | 2021-03-01 | 2022-11-29 | 中国人民解放军陆军军医大学 | Cervical cancer cell intelligent detection method based on liquid-based thin-layer cell detection technology |
| CN114708484A (en) * | 2022-03-14 | 2022-07-05 | 中铁电气化局集团有限公司 | Pattern analysis method suitable for identifying defects |
| CN114708484B (en) * | 2022-03-14 | 2023-04-07 | 中铁电气化局集团有限公司 | Pattern analysis method suitable for identifying defects |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108364006B (en) | Medical image classification device based on multi-mode deep learning and construction method thereof | |
| Huang et al. | Time-efficient sparse analysis of histopathological whole slide images | |
| CN112101451B (en) | Breast cancer tissue pathological type classification method based on generation of antagonism network screening image block | |
| Kong et al. | Multi-modal data Alzheimer’s disease detection based on 3D convolution | |
| Hashemi et al. | Mass detection in lung CT images using region growing segmentation and decision making based on fuzzy inference system and artificial neural network | |
| CN112036298A (en) | Cell detection method based on double-segment block convolutional neural network | |
| Ypsilantis et al. | Recurrent convolutional networks for pulmonary nodule detection in CT imaging | |
| Pan et al. | Mitosis detection techniques in H&E stained breast cancer pathological images: A comprehensive review | |
| Yao et al. | Pneumonia Detection Using an Improved Algorithm Based on Faster R‐CNN | |
| Rajee et al. | Gender classification on digital dental x-ray images using deep convolutional neural network | |
| Ikromjanov et al. | Whole slide image analysis and detection of prostate cancer using vision transformers | |
| CN113782184A (en) | Cerebral apoplexy auxiliary evaluation system based on facial key point and feature pre-learning | |
| Banerjee et al. | A CADe system for gliomas in brain MRI using convolutional neural networks | |
| Nie et al. | Recent advances in diagnosis of skin lesions using dermoscopic images based on deep learning | |
| Hou et al. | Mass segmentation for whole mammograms via attentive multi-task learning framework | |
| Tiwari et al. | Optimized Ensemble of Hybrid RNN-GAN Models for Accurate and Automated Lung Tumour Detection from CT Images | |
| Kumaraswamy et al. | Automatic prostate segmentation of magnetic resonance imaging using Res-Net | |
| CN112488996A (en) | Inhomogeneous three-dimensional esophageal cancer energy spectrum CT (computed tomography) weak supervision automatic labeling method and system | |
| CN117036288A (en) | Tumor subtype diagnosis method for full-slice pathological image | |
| Khan et al. | A Computer-Aided Diagnostic System to Identify Diabetic Retinopathy, Utilizing a Modified Compact Convolutional Transformer and Low-Resolution Images to Reduce Computation Time. Biomedicines. 2023. No. 11. Art. 1566 | |
| Vinisha et al. | A Novel Framework for Brain Tumor Segmentation using Neuro Trypetidae Fruit Fly-Based UNet | |
| Vinod et al. | Ensemble Technique for Brain Tumour Patient Survival Prediction | |
| Santos et al. | A new approach for fundus lesions instance segmentation based on mask r-cnn x101-fpn pre-trained architecture | |
| Manju et al. | Early diagnosis of lung tumors for extending patients’ life using deep neural networks | |
| Xia et al. | A new hybrid brain MR image segmentation algorithm with super-resolution, spatial constraint-based clustering and fine tuning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |