CN110853005A - Immunohistochemical membrane staining section diagnosis method and device - Google Patents
Immunohistochemical membrane staining section diagnosis method and device Download PDFInfo
- Publication number
- CN110853005A CN110853005A CN201911075417.3A CN201911075417A CN110853005A CN 110853005 A CN110853005 A CN 110853005A CN 201911075417 A CN201911075417 A CN 201911075417A CN 110853005 A CN110853005 A CN 110853005A
- Authority
- CN
- China
- Prior art keywords
- immunohistochemical
- image
- region
- membrane staining
- quality evaluation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002055 immunohistochemical effect Effects 0.000 title claims abstract description 67
- 238000010186 staining Methods 0.000 title claims abstract description 66
- 239000012528 membrane Substances 0.000 title claims abstract description 63
- 238000000034 method Methods 0.000 title claims abstract description 30
- 238000003745 diagnosis Methods 0.000 title claims abstract description 19
- 238000013441 quality evaluation Methods 0.000 claims abstract description 50
- 230000011218 segmentation Effects 0.000 claims abstract description 17
- 230000001575 pathological effect Effects 0.000 claims abstract description 11
- 210000004027 cell Anatomy 0.000 claims description 65
- 210000004881 tumor cell Anatomy 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 4
- 238000002405 diagnostic procedure Methods 0.000 claims 2
- 238000012360 testing method Methods 0.000 abstract description 2
- 238000012549 training Methods 0.000 abstract description 2
- 238000001514 detection method Methods 0.000 description 17
- 238000013528 artificial neural network Methods 0.000 description 16
- 230000006870 function Effects 0.000 description 10
- 230000008569 process Effects 0.000 description 9
- 238000011176 pooling Methods 0.000 description 8
- 230000008439 repair process Effects 0.000 description 7
- 238000013527 convolutional neural network Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 230000009467 reduction Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000004913 activation Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000006835 compression Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 230000007170 pathology Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 206010028980 Neoplasm Diseases 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 238000005315 distribution function Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 229940079593 drug Drugs 0.000 description 1
- 238000004043 dyeing Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 230000001744 histochemical effect Effects 0.000 description 1
- 230000001900 immune effect Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000007762 localization of cell Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 210000004180 plasmocyte Anatomy 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 238000005067 remediation Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20092—Interactive image processing based on input by user
- G06T2207/20104—Interactive definition of region of interest [ROI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
Abstract
The invention relates to a method for diagnosing immunohistochemical membrane staining sections, which comprises the following steps: s10, receiving a digital slice full-field image of the immunohistochemical membrane staining pathological section to be diagnosed; s20, performing quality evaluation on the region of interest on the digital slice full field image, and performing image restoration on the region with unqualified quality evaluation to obtain the region of interest with qualified quality evaluation; s30, performing cell positioning, classification and segmentation on the region of interest qualified by the quality evaluation, marking the position and the type of the cell on the region of interest, and calculating and marking an immunohistochemical index; and S40, displaying the marked region of interest. The present application also relates to an immunohistochemical membrane staining section diagnosis device and a computer readable medium. Because the contextual information of the focus area has important significance for interpretation of the immunohistochemical membrane staining section, training and testing are carried out by collecting eight-connected patch images of immunohistochemical membrane staining pathological sections with time sequence, and the accuracy of interpretation is guaranteed.
Description
Technical Field
The invention relates to the field of section diagnosis, in particular to a method and a device for diagnosing an immunohistochemical membrane staining section.
Background
The immunohistochemical technology is taken as a key technology of histopathology or cytopathology, and is taken as one of key technologies in the process of judging, diagnosing and treating tumors at present, so that the immunohistochemical technology has great influence. The immunohistochemical or immunocytochemical techniques refer to the qualitative, localized, or quantitative studies of certain chemical components in tissue sections or cell specimens using immunological and histochemical principles. The immunohistochemical histopathology technology comprises immunohistochemical membrane staining and plasma cell staining, and for immunohistochemical membrane staining sections, the staining process is manually finished by staining technicians, because the staining technicians control different amounts of staining chemical drugs, the imaging quality of the immunohistochemical membrane staining sections is easy to be different, and pathologists are easy to generate large statistical result deviation when observing and counting tumor cells on the immunohistochemical membrane staining sections through naked eyes, so that the accurate section diagnosis of the doctors is influenced.
Disclosure of Invention
The invention aims to solve the technical problems that in the prior art, the staining effect of an immunohistochemical membrane staining section is different, and the counting of the number of cells and the distinguishing of cell types are easily influenced.
The invention realizes the purpose through the following technical scheme: an immunohistochemical membrane staining section diagnosis method comprising: s10, receiving a digital slice full-field image of the immunohistochemical membrane staining pathological section to be diagnosed; s20, performing quality evaluation on the region of interest on the digital slice full field image, and performing image restoration on the region with unqualified quality evaluation to obtain the region of interest with qualified quality evaluation; s30, performing cell positioning, classification and segmentation on the region of interest qualified by the quality evaluation, marking the position and the type of the cell on the region of interest, and calculating and marking an immunohistochemical index; and S40, displaying the marked region of interest.
Further, S20 specifically includes: s21, intercepting a plurality of continuous image blocks with the same size on the region of interest; s22, performing quality evaluation on each image block, and judging whether the quality evaluation of the image block is qualified; and performing image restoration on the image blocks with unqualified quality evaluation to obtain the region of interest with qualified quality evaluation after restoration.
Further, S21, sequentially truncating 9 image blocks with the same size on the region of interest according to the upper left, the upper right, the middle left, the middle right, the lower left and the lower right.
Further, the quality evaluation of the image block includes: calculating the peak signal-to-noise ratio m of the image block, and judging whether the peak signal-to-noise ratio m is within a standard peak signal-to-noise ratio range [ m1, m2], wherein if yes, the quality evaluation of the image block is qualified; otherwise, the image block quality evaluation is unqualified; the standard peak signal-to-noise ratio range [ m1, m2] is the peak signal-to-noise ratio range of the staining standard immunohistochemical membrane staining slice image, wherein m1 is the lowest peak signal-to-noise ratio value of the standard immunohistochemical membrane staining slice image, and m2 is the highest peak signal-to-noise ratio value of the standard immunohistochemical membrane staining slice image.
Further, S22 performs image restoration on the image block with the quality evaluation being unqualified, including: and adopting a generator in the generation countermeasure network to carry out image restoration on the image block with unqualified quality evaluation, then carrying out quality judgment through a discriminator in the countermeasure generation network, and if the peak signal-to-noise ratio m of the restored image block is not within the standard peak signal-to-noise ratio range [ m1, m2], carrying out iterative restoration until the peak signal-to-noise ratio m of the image block reaches the standard peak signal-to-noise ratio range [ m1, m2 ].
Further, S30, includes: determining the position of membrane stained cells in the region of interest and performing position marking;
classifying the membrane staining cells into positive tumor cells, positive normal cells, negative tumor cells and negative normal cells, marking the category of each membrane staining cell, counting the number of each cell type, and calculating and marking corresponding immunohistochemical indexes; the specific contour of the membrane stained cells was segmented.
The present application also provides an immunohistochemical membrane staining section diagnosis device, comprising: one or more processors; a memory;
and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, the programs comprising instructions for performing a method of immunohistochemical membrane staining slice diagnosis.
Further, the system also comprises a high-throughput scanner, which is used for converting the immunohistochemical membrane staining pathological section to be diagnosed into a digital section full-field image and is connected with the processor; and a display coupled to the processor for receiving and displaying the annotated region of interest from the processor.
The present application also provides a computer-readable medium storing a computer program for electronic data exchange, wherein the computer program causes a computer to execute an immunohistochemical membrane staining section diagnosis method.
Compared with the prior art, the invention has the following beneficial effects: (1) and (3) accurate analysis: the contextual information of the focus area has important significance for judging the immunohistochemical membrane staining slice, and compared with the existing method, the system trains and tests by acquiring eight-connected patch images of immunohistochemical membrane staining pathological sections with time sequence, so that the accuracy of the judgment is guaranteed. (2) The robustness is better: compared with the prior art, the system firstly repairs the digitized immunohistochemical membrane staining pathological section, repairs the immunohistochemical membrane staining section image with unqualified stained inferior image quality evaluation to the peak signal-to-noise ratio distribution range of the full-field image of the standard digital section, and then carries out accurate analysis. (3) The system is fully automatic, digital imaging is carried out by a high-throughput scanner, and finally intelligent tumor grading interpretation evaluation is carried out as one-stop service, so that the process is convenient to operate; (4) by adopting the immunohistochemical membrane staining section diagnosis method and system provided by the invention, a pathologist only needs to place the immunohistochemical stained section in a high-throughput scanner, and after click scanning, the system automatically performs the processes of digitization, intelligent repair, interpretation analysis and the like, and finally provides the cell position, the cell type, the number of various cells and corresponding immunohistochemical indexes of an interested area for the doctor.
Drawings
FIG. 1 is a flow chart of a method for diagnosing an immunohistochemical membrane stained section (1);
FIG. 2 is a flow chart of a method for diagnosing an immunohistochemical membrane stained section (2);
FIG. 3 is a schematic diagram of a generative confrontation network generator architecture employed for image quality remediation;
FIG. 4 is a schematic diagram of a generation countermeasure network discriminator employed for image quality restoration;
FIG. 5 is a schematic diagram of a convolutional neural network used in a cell localization segmentation, segmentation model.
Detailed Description
The invention will be further described with reference to the accompanying drawings in which:
an immunohistochemical membrane staining section diagnosis device comprises a high-throughput scanner, a processor and a display. The processor is in communication connection with the high-flux scanner and is connected with the display. The immunohistochemical membrane staining pathological section is placed under a high-flux scanner, the high-flux scanner scans the immunohistochemical membrane staining pathological section into a digital section full-field image, the digital section full-field image is sent to a processor to be processed, and the digital section full-field image processed by the processor is output and displayed on a display end. The high throughput scanner used in this example was used to scan immunohistochemically stained sections for digital visualization of the sections, including 4 x, 10 x and 40x magnification.
As shown in fig. 1 and 2, the process of performing digital full-field image processing of slices by using the immunohistochemical membrane staining slice diagnosis device comprises the following steps:
s10, receiving a digital slice full-field image of the immunohistochemical membrane staining pathological section to be diagnosed;
s20, performing quality evaluation on the region of interest on the digital slice full field image, performing image restoration on the region with unqualified quality evaluation, and obtaining the region of interest with qualified quality evaluation, which specifically comprises:
s21, intercepting 9 continuous image blocks with the same size on the region of interest;
the operator delineates the region of interest on the digital slice full field map, which may be a part of the entire digital slice full field map or the entire digital slice full field map, according to the slice position desired to be observed. The selected region of interest is mapped to a 40X magnification (enough to distinguish cell outlines and cell categories), image blocks are intercepted on the region of interest according to the upper left, the upper right, the middle left, the middle right, the lower left, the lower right and the lower left in sequence, 9 image blocks with the same size are obtained, and if the region of interest is not acquired in sequence according to time and space, the integrity of a cell tissue region on the digital pathology full-field map is damaged. Before the image block is intercepted in the region of interest, firstly, judging whether the length and the width of the region of interest are matched with a preset value, if the length and width of the region of interest do not match the preset values, the length and width of the region of interest are changed, for example filled, then intercepting the image blocks according to an eight-communication mode to obtain 9 continuous image blocks, wherein the size of each image block is consistent with that of an input picture of an artificial neural network model for cell detection, since the received digital slice full field map can clearly separate the cell outline and the cell type, when the operator divides the region of interest, the cutting of a cell into two parts, i.e. the division of the region of interest, can be actively avoided, which can avoid the destruction of the integrity of the cells at the border of the region. The image blocks acquired according to the eight-communication mode sequentially according to the time sequence and the space sequence have time sequence information, and the time sequence information is added in the network training fitting process by combining a circulating neural network in the artificial neural network, so that the accuracy of final cell detection, classification and segmentation can be ensured. The reason that 9 image blocks are to be acquired in an eight-pass manner is that in practice, if more than 9 image blocks are processed, for example, a 16-grid division manner is adopted to obtain 16 continuous image blocks, the processing speed of the CPU is limited, and if less than 9 image blocks are processed, for example, a 4-grid division manner is adopted to obtain 4 continuous images, the time sequence among the image blocks is slightly worse, and the 9 continuous image blocks can both take account of the processing speed of the CPU and the time sequence of the images, and are an optimal division manner.
S22, performing quality evaluation on each image block, and judging whether the quality evaluation of the image block is qualified; and performing image restoration on the image blocks with unqualified quality evaluation to obtain the region of interest with qualified quality evaluation after restoration.
The reason why the quality evaluation and repair of the region of interest are required is that the slice staining quality is easily uneven due to the influence of various factors during the slice making of the immunohistochemical membrane staining slice, the white balance, hue, lightness and chroma of the digital slice full field image collected by the high-throughput scanner are inconsistent, and these influence factors can cause certain influence on the subsequent cell detection, resulting in the consequences of inaccurate detection and the like, so that it is very necessary to ensure that the accuracy of the subsequent cell detection cannot be influenced as much as possible through the quality repair link.
Before image block quality evaluation is carried out, firstly, a digital slice full-field image of an immunohistochemical membrane staining slice with a staining standard is obtained and used as a standard digital slice full-field image, peak signal-to-noise ratio statistics is carried out on the standard digital slice full-field image, m1 is set as the lowest peak signal-to-noise ratio value of the standard digital slice full-field image, m2 is set as the highest peak signal-to-noise ratio value of the standard digital slice full-field image, then [ m1, m2] is used as the standard peak signal-to-noise ratio range of the standard digital slice full-field image, and multiple experiments show that the standard value of m1 is 47.6 decibels, the error range is 5.04, the standard value of m2 is 78.4 decibels. And evaluating the quality of the image block, which comprises the following steps: calculating the peak signal-to-noise ratio m of each image block, judging whether the peak signal-to-noise ratio m is within a standard peak signal-to-noise ratio range [ m1, m2], and if so, judging that the quality evaluation of the image block is qualified; otherwise, the image block quality evaluation is unqualified.
When the image blocks with unqualified quality evaluation are repaired, firstly, the image blocks with unqualified quality evaluation are repaired through the generation countermeasure network, then the quality judgment is carried out through a discriminator in the countermeasure generation network, if the peak signal-to-noise ratio m of the repaired image blocks is not within the standard peak signal-to-noise ratio range [ m1, m2], iterative repair is carried out again until the peak signal-to-noise ratio m of the image blocks reaches the standard peak signal-to-noise ratio range [ m1, m2 ]. And (4) completing the image block repairing of all the image blocks with unqualified quality evaluation, namely completing the image repairing of the region of interest, and obtaining the repaired region of interest with qualified quality evaluation.
The generator for generating the countermeasure network is, as shown in fig. 3, a U-shaped full convolution neural network (U-Net), as shown in fig. 3, composed of a series of convolution layers, pooling layers, up-sampling layers, and jump connection layers, generates an image normalization layer by using an activation function (Softmax), and finally converts the image normalization layer into a standard RGB image, i.e., a finally repaired image. The convolutional layer is mainly used for extracting local features of an image, the pooling layer performs dimensionality reduction compression on an extracted feature map, then transmits the feature map subjected to dimensionality reduction to the next convolutional layer for feature extraction, after a series of convolution and pooling operations, the extracted feature map is tiled into an array to be used as input of an upsampling layer, the upsampling layer enables each feature point of the obtained high-level feature map to be subjected to linear interpolation to obtain a new feature map with the size 2 times that of the original feature map, the skip connection layer performs feature channel superposition on a coding layer and a decoding layer in the same level, performs dimensionality reduction compression on the superposed feature map and then performs upsampling, the feature map is continuously upsampled to the size of an original image, an image data distribution Inceposition Score is used as a loss function, and Adam is an optimization method and is trained until the loss function does not change obviously any more. The image input by the convolutional neural network is 10000 patch images which are dyed in an abnormal manner and normalized, the size of the patch images is consistent with that of the image blocks, and the size of the output images is consistent with that of the input images. In the loss function calculation process, 10000 normalized patch images of the dyeing standard are taken, and image data distribution function IncepotionScore is calculated with the images output by the network each time to obtain a loss function value, so that the gradient is calculated and updated.
The arbiter in the generative countermeasure network is also a structure of a convolutional neural network, as shown in fig. 4, and is composed of a series of convolutional layers, pooling layers, and full-link layers, as shown in fig. 4, and finally classifies the images by using an activation function (Softmax). The convolution layer is mainly used for extracting local features of images, the pooling layer conducts dimensionality reduction compression on the extracted feature graph, then transmits the feature graph after dimensionality reduction into the next convolution layer to conduct feature extraction, after a series of convolution and pooling operations, the extracted feature graph is tiled into an array to serve as the input of the full connection layer, the full connection layer further extracts features, the features are classified, and therefore the input images are classified. The final classification process not only judges the authenticity of the generated image, but also calculates the physical distance between the peak signal-to-noise ratio of the generated image and the standard peak signal-to-noise ratio range as a loss function. The convolutional neural network takes the repaired image block as input, obtains the peak signal-to-noise ratio of the coded characteristic diagram of the repaired image staining block through the convolutional neural network, compares the peak signal-to-noise ratio with the standard peak signal-to-noise ratio range, and further determines whether the repair is finished. And executing S30 after all the image blocks with unqualified quality evaluation are repaired.
S30, performing cell detection, classification and segmentation on the region of interest qualified by the quality evaluation, marking the position and the type of the cell on the region of interest, and calculating and marking an immunohistochemical index; the method comprises the following steps:
determining the position of membrane stained cells in the region of interest and performing position marking; for the region of interest qualified by the quality evaluation, adopting an artificial neural network model for cell detection and adopting an artificial neural network model for cell detection; classifying each membrane staining cell by adopting an artificial neural network model for cell classification, classifying the membrane staining cells into positive tumor cells, positive normal cells, negative tumor cells and negative normal cells, marking the category of each membrane staining cell, counting the number of each cell, and calculating and marking corresponding immunohistochemical indexes; the specific outline of the membrane stained cells is segmented by an artificial neural network model for cell segmentation.
And S40, displaying the marked region of interest.
The artificial neural network model for cell detection, the artificial neural network model for cell classification and the artificial neural network model for cell segmentation form an end-to-end cascade model, the deep neural network for cell detection, classification and segmentation is formed, as shown in fig. 5, the deep neural network mainly comprises a series of convolution layers, pooling layers, up-sampling layers and jump connection layers, finally, the detected cells are classified by using an activation function (Softmax), a cross-over-comparison function (Iou) is used for judging whether the detected cells are real current cell types or not to control the false positive rate, and a cross entropy function is used for evaluating the cell segmentation result.
In the deep neural network for cell detection, classification and segmentation adopted in the embodiment, a detection network and a segmentation network are fused, firstly, a coding network is used for extracting multi-scale semantic features, then, a spatial pyramid network (RPN) is used for obtaining a cell detection candidate region frame anchor box, approximately N rois are reserved after non-maximum suppression operation, wherein N is a hyperparameter and can be autonomously defined according to requirements, and because the step sizes are different in the process of convolution feature extraction, roiign operation is separately and respectively performed on the step sizes corresponding to feature maps of four different scales, so-called roiign refers to traversing each candidate region, keeping floating point boundaries from being quantized, segmenting the candidate region into k × k units, and not quantizing the boundaries of each unit. Fixed four coordinate positions are calculated in each cell, the values of the four positions are calculated by a bilinear interpolation method, and then the maximum pooling operation is performed. The purpose is to map the candidate region box obtained for each scale on the input original image. And then the candidate areas mapped on the original image generated by the RoIAlign operation are connected by Concat, and then the network is divided into three parts: full connected prediction class, full connected prediction rectangle box, full convolution prediction pixel partition mask.
Meanwhile, in order to ensure the time sequence among image blocks, the deep neural network for cell detection, classification and segmentation combines the time sequence characteristics of the cyclic neural network, the number of input images is set to be 9, 9 cascaded long-short term memory units (LSTM) are designed for eight-connected patch images, and the long-short term memory units balance final output under the condition of combining the last output result and the current output result. The purpose of this is to ensure that there is context timing information between the image blocks of the digital slice full field map.
Finally, according to the obtained classification result, the processor counts the number of each cell type such as positive tumor cells, positive normal cells, negative tumor cells, negative normal cells and the like, calculates the immunohistochemical index of the current region of interest and the final immunohistochemical index of the whole piece, finally displays the detection and segmentation result on the original region of interest and returns the detection and segmentation result to the display, and the display displays the detection and quantification result of the tumor cells on the region of interest and the corresponding immunohistochemical index.
When the device is actually used, the immunohistochemical section images are converted into a digital pathology full-field image through the high-throughput scanner and transmitted to the processor, the processor displays the digital pathology full-field image on a first display interface (or a first display area) at the display end, an operator selects an interested area in the first display interface, and the processor displays the labeled interested area image on a second display interface (or a second display area) after performing cell positioning, classification, segmentation and labeling on the selected interested area. If the operator wants to view another region of interest, the selection can be made again on the first display interface (or first display area), and the selected region of interest is processed by the processor and displayed on the second display interface (or second display area).
The foregoing illustrates and describes the principles, general features, and advantages of the present invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed.
Claims (9)
1. An immunohistochemical membrane staining section diagnosis method, comprising: s10, receiving a digital slice full-field image of the immunohistochemical membrane staining pathological section to be diagnosed; s20, performing quality evaluation on the region of interest on the digital slice full field image, and performing image restoration on the region with unqualified quality evaluation to obtain the region of interest with qualified quality evaluation; s30, performing cell positioning, classification and segmentation on the region of interest qualified by the quality evaluation, marking the position and the type of the cell on the region of interest, and calculating and marking an immunohistochemical index; and S40, displaying the marked region of interest.
2. The method of claim 1, wherein S20 comprises: s21, intercepting a plurality of continuous image blocks with the same size on the region of interest; s22, performing quality evaluation on each image block, and judging whether the quality evaluation of the image block is qualified; and performing image restoration on the image blocks with unqualified quality evaluation to obtain the region of interest with qualified quality evaluation after restoration.
3. The immunohistochemical membrane staining diagnosis method of claim 2, wherein S21, on the region of interest, sequentially cuts out 9 image blocks with the same size according to the upper left, upper right, middle left, middle right, lower left, lower right.
4. The immunohistochemical membrane staining section diagnosis method according to claim 2 or 3, wherein the quality evaluation of the image blocks comprises: calculating the peak signal-to-noise ratio m of the image block, and judging whether the peak signal-to-noise ratio m is within a standard peak signal-to-noise ratio range [ m1, m2], wherein if yes, the quality evaluation of the image block is qualified; otherwise, the standard peak signal-to-noise ratio range [ m1, m2] within which the image block quality evaluation is unqualified is the peak signal-to-noise ratio range of the immunohistochemical membrane staining slice image with the staining standard, wherein m1 is the lowest peak signal-to-noise ratio value of the standard immunohistochemical membrane staining slice image, and m2 is the highest peak signal-to-noise ratio value of the standard immunohistochemical membrane staining slice image.
5. The immunohistochemical membrane staining section diagnosis method according to claim 4, wherein the S22 image repairing the image blocks with unqualified quality evaluation comprises: and adopting a generator in the generation countermeasure network to carry out image restoration on the image block with unqualified quality evaluation, then carrying out quality judgment through a discriminator in the countermeasure generation network, and if the peak signal-to-noise ratio m of the restored image block is not within the standard peak signal-to-noise ratio range [ m1, m2], carrying out iterative restoration until the peak signal-to-noise ratio m of the image block reaches the standard peak signal-to-noise ratio range [ m1, m2 ].
6. The immunohistochemical membrane staining section diagnosis method according to claim 1, wherein S30 comprises: determining the position of membrane stained cells in the region of interest qualified by the quality evaluation, and marking the position; classifying the membrane staining cells into positive tumor cells, positive normal cells, negative tumor cells and negative normal cells, marking the category of each membrane staining cell, counting the number of each cell type, and calculating and marking corresponding immunohistochemical indexes; the specific contour of the membrane stained cells was segmented.
7. An immunohistochemical membrane staining section diagnosis device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, the programs comprising instructions for performing the immunohistochemical membrane staining slice diagnostic method of claims 1-6.
8. The immunohistochemical membrane staining section diagnosis device according to claim 7, further comprising a high-throughput scanner for converting the immunohistochemical membrane staining pathological section to be diagnosed into a digital section full-field image, connected to the input end of the processor; and the display is connected with the output end of the processor and used for receiving and displaying the marked region of interest from the processor.
9. A computer-readable medium storing a computer program for electronic data exchange, wherein the computer program causes a computer to execute the immunohistochemical membrane staining slice diagnostic method of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911075417.3A CN110853005A (en) | 2019-11-06 | 2019-11-06 | Immunohistochemical membrane staining section diagnosis method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911075417.3A CN110853005A (en) | 2019-11-06 | 2019-11-06 | Immunohistochemical membrane staining section diagnosis method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110853005A true CN110853005A (en) | 2020-02-28 |
Family
ID=69599449
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911075417.3A Pending CN110853005A (en) | 2019-11-06 | 2019-11-06 | Immunohistochemical membrane staining section diagnosis method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110853005A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111383429A (en) * | 2020-03-04 | 2020-07-07 | 西安咏圣达电子科技有限公司 | Method, system, device and storage medium for detecting dress of workers in construction site |
| CN111430011A (en) * | 2020-03-31 | 2020-07-17 | 杭州依图医疗技术有限公司 | Cell stain image display method, cell stain image processing method, and storage medium |
| CN111751371A (en) * | 2020-06-24 | 2020-10-09 | 武汉中纪生物科技有限公司 | Immunohistochemical digital slide reading system and method |
| CN112102323A (en) * | 2020-09-17 | 2020-12-18 | 陕西师范大学 | Adherent nucleus segmentation method based on generation of countermeasure network and Caps-Unet network |
| CN112270684A (en) * | 2020-12-25 | 2021-01-26 | 宁波兰茜生物科技有限公司 | Microscopic image immunohistochemical virtual multiple labeling and analyzing method and system |
| CN112330625A (en) * | 2020-11-03 | 2021-02-05 | 杭州迪英加科技有限公司 | Immunohistochemical nuclear staining section cell positioning multi-domain co-adaptation training method |
| CN112580748A (en) * | 2020-12-30 | 2021-03-30 | 电子科技大学 | Method for counting cancer cells of Ki67 stained image |
| CN113030088A (en) * | 2021-03-10 | 2021-06-25 | 浙江大学 | Cell rapid statistical evaluation method for solid lignification degree of loquat fruit |
| CN113033287A (en) * | 2021-01-29 | 2021-06-25 | 杭州依图医疗技术有限公司 | Pathological image display method and device |
| CN114511559A (en) * | 2022-04-18 | 2022-05-17 | 杭州迪英加科技有限公司 | Multidimensional evaluation method, system and medium for quality of stained nasal polyp pathological section |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106846310A (en) * | 2017-01-19 | 2017-06-13 | 宁波江丰生物信息技术有限公司 | A kind of pathology aided analysis method based on immunohistochemistry technique |
| CN108492258A (en) * | 2018-01-17 | 2018-09-04 | 天津大学 | A kind of radar image denoising method based on generation confrontation network |
| CN109903284A (en) * | 2019-03-04 | 2019-06-18 | 武汉大学 | A kind of other method and system of HER2 immunohistochemistry image automatic judging |
| CN110189278A (en) * | 2019-06-06 | 2019-08-30 | 上海大学 | A kind of binocular scene image repair method based on generation confrontation network |
| CN111383429A (en) * | 2020-03-04 | 2020-07-07 | 西安咏圣达电子科技有限公司 | Method, system, device and storage medium for detecting dress of workers in construction site |
-
2019
- 2019-11-06 CN CN201911075417.3A patent/CN110853005A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106846310A (en) * | 2017-01-19 | 2017-06-13 | 宁波江丰生物信息技术有限公司 | A kind of pathology aided analysis method based on immunohistochemistry technique |
| CN108492258A (en) * | 2018-01-17 | 2018-09-04 | 天津大学 | A kind of radar image denoising method based on generation confrontation network |
| CN109903284A (en) * | 2019-03-04 | 2019-06-18 | 武汉大学 | A kind of other method and system of HER2 immunohistochemistry image automatic judging |
| CN110189278A (en) * | 2019-06-06 | 2019-08-30 | 上海大学 | A kind of binocular scene image repair method based on generation confrontation network |
| CN111383429A (en) * | 2020-03-04 | 2020-07-07 | 西安咏圣达电子科技有限公司 | Method, system, device and storage medium for detecting dress of workers in construction site |
Non-Patent Citations (3)
| Title |
|---|
| NESLIHAN BAYRAMOGLU 等: "Towards Virtual H&E Staining of Hyperspectral Lung Histology Images Using Conditional Generative Adversarial Networks", 《2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS》 * |
| 李雪瑾 等: "基于生成对抗网络的数字图像修复技术", 《电子测量与仪器学报》 * |
| 沈湘萍 等: "45 例胸腔积液细胞蜡块结合免疫组化在细胞学诊断中的应用", 《浙江实用医学》 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111383429A (en) * | 2020-03-04 | 2020-07-07 | 西安咏圣达电子科技有限公司 | Method, system, device and storage medium for detecting dress of workers in construction site |
| CN111430011A (en) * | 2020-03-31 | 2020-07-17 | 杭州依图医疗技术有限公司 | Cell stain image display method, cell stain image processing method, and storage medium |
| CN111430011B (en) * | 2020-03-31 | 2023-08-29 | 杭州依图医疗技术有限公司 | Display method, processing method and storage medium for cell staining image |
| CN111751371A (en) * | 2020-06-24 | 2020-10-09 | 武汉中纪生物科技有限公司 | Immunohistochemical digital slide reading system and method |
| CN111751371B (en) * | 2020-06-24 | 2021-01-08 | 武汉中纪生物科技有限公司 | Immunohistochemical digital slide reading system and method |
| CN112102323B (en) * | 2020-09-17 | 2023-07-07 | 陕西师范大学 | Adhesion cell nucleus segmentation method based on generation of countermeasure network and Caps-Unet network |
| CN112102323A (en) * | 2020-09-17 | 2020-12-18 | 陕西师范大学 | Adherent nucleus segmentation method based on generation of countermeasure network and Caps-Unet network |
| CN112330625A (en) * | 2020-11-03 | 2021-02-05 | 杭州迪英加科技有限公司 | Immunohistochemical nuclear staining section cell positioning multi-domain co-adaptation training method |
| CN112270684A (en) * | 2020-12-25 | 2021-01-26 | 宁波兰茜生物科技有限公司 | Microscopic image immunohistochemical virtual multiple labeling and analyzing method and system |
| CN112580748B (en) * | 2020-12-30 | 2022-10-14 | 电子科技大学 | Method for counting classified cells of stain image |
| CN112580748A (en) * | 2020-12-30 | 2021-03-30 | 电子科技大学 | Method for counting cancer cells of Ki67 stained image |
| CN113033287A (en) * | 2021-01-29 | 2021-06-25 | 杭州依图医疗技术有限公司 | Pathological image display method and device |
| CN113030088A (en) * | 2021-03-10 | 2021-06-25 | 浙江大学 | Cell rapid statistical evaluation method for solid lignification degree of loquat fruit |
| CN114511559A (en) * | 2022-04-18 | 2022-05-17 | 杭州迪英加科技有限公司 | Multidimensional evaluation method, system and medium for quality of stained nasal polyp pathological section |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110853005A (en) | Immunohistochemical membrane staining section diagnosis method and device | |
| JP7422825B2 (en) | Focus-weighted machine learning classifier error prediction for microscope slide images | |
| US11842556B2 (en) | Image analysis method, apparatus, program, and learned deep learning algorithm | |
| CN110428432B (en) | Deep neural network algorithm for automatically segmenting colon gland image | |
| CN111784671B (en) | Pathological image focus region detection method based on multi-scale deep learning | |
| CN111696094B (en) | Immunohistochemical PD-L1 membrane staining pathological section image processing method, device and equipment | |
| CN111814741B (en) | Method for detecting embryo-sheltered pronucleus and blastomere based on attention mechanism | |
| WO2021139258A1 (en) | Image recognition based cell recognition and counting method and apparatus, and computer device | |
| US11010883B2 (en) | Automated analysis of petrographic thin section images using advanced machine learning techniques | |
| KR20080016847A (en) | Methods of chromogen separation-based image analysis | |
| CN110736748A (en) | Immunohistochemical nuclear plasma staining section diagnosis method and system | |
| CN112819821B (en) | Cell nucleus image detection method | |
| CN109726644A (en) | A kind of nucleus dividing method based on generation confrontation network | |
| CN108305253A (en) | A kind of pathology full slice diagnostic method based on more multiplying power deep learnings | |
| CN112132166A (en) | Intelligent analysis method, system and device for digital cytopathology image | |
| CN112215217B (en) | Digital image recognition method and device for simulating doctor to read film | |
| CN110838094B (en) | Pathological section dyeing style conversion method and electronic equipment | |
| CN115909006B (en) | Mammary tissue image classification method and system based on convolution transducer | |
| CN110796661B (en) | Fungal microscopic image segmentation detection method and system based on convolutional neural network | |
| CN112990214A (en) | Medical image feature recognition prediction model | |
| CN115546605A (en) | Training method and device based on image labeling and segmentation model | |
| CN111656393A (en) | Histological image analysis | |
| CN114612669B (en) | Method and device for calculating ratio of inflammation to necrosis of medical image | |
| CN115564750A (en) | Intraoperative frozen slice image identification method, intraoperative frozen slice image identification device, intraoperative frozen slice image identification equipment and intraoperative frozen slice image storage medium | |
| CN115222649A (en) | System, apparatus and method for detecting and classifying patterns of heatmaps |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200228 |