CN114937045A - Hepatocellular carcinoma pathological image segmentation system - Google Patents
Hepatocellular carcinoma pathological image segmentation system Download PDFInfo
- Publication number
- CN114937045A CN114937045A CN202210698119.5A CN202210698119A CN114937045A CN 114937045 A CN114937045 A CN 114937045A CN 202210698119 A CN202210698119 A CN 202210698119A CN 114937045 A CN114937045 A CN 114937045A
- Authority
- CN
- China
- Prior art keywords
- hepatocellular carcinoma
- segmentation
- pathological image
- image segmentation
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000001575 pathological effect Effects 0.000 title claims abstract description 113
- 206010073071 hepatocellular carcinoma Diseases 0.000 title claims abstract description 105
- 231100000844 hepatocellular carcinoma Toxicity 0.000 title claims abstract description 103
- 238000003709 image segmentation Methods 0.000 title claims abstract description 67
- 230000011218 segmentation Effects 0.000 claims abstract description 52
- 230000007170 pathology Effects 0.000 claims abstract description 30
- 238000012549 training Methods 0.000 claims abstract description 19
- 230000002452 interceptive effect Effects 0.000 claims abstract description 13
- 238000007781 pre-processing Methods 0.000 claims abstract description 11
- 230000001133 acceleration Effects 0.000 claims abstract description 7
- 238000012800 visualization Methods 0.000 claims description 22
- 230000003993 interaction Effects 0.000 claims description 14
- 230000007246 mechanism Effects 0.000 claims description 11
- 230000003321 amplification Effects 0.000 claims description 7
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 6
- 230000002093 peripheral effect Effects 0.000 claims description 6
- 238000000354 decomposition reaction Methods 0.000 claims description 4
- 238000004043 dyeing Methods 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 238000002360 preparation method Methods 0.000 claims description 3
- 238000000926 separation method Methods 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 2
- 206010028980 Neoplasm Diseases 0.000 abstract description 38
- 238000012545 processing Methods 0.000 abstract description 15
- 230000006870 function Effects 0.000 abstract description 11
- 230000004083 survival effect Effects 0.000 abstract description 10
- 238000003745 diagnosis Methods 0.000 abstract description 7
- 230000000007 visual effect Effects 0.000 abstract description 7
- 201000011510 cancer Diseases 0.000 abstract description 6
- 230000009286 beneficial effect Effects 0.000 abstract description 3
- 210000001519 tissue Anatomy 0.000 description 21
- 238000010586 diagram Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 238000000034 method Methods 0.000 description 6
- 238000004393 prognosis Methods 0.000 description 6
- 210000004027 cell Anatomy 0.000 description 5
- 238000010827 pathological analysis Methods 0.000 description 5
- 208000014018 liver neoplasm Diseases 0.000 description 4
- 230000002980 postoperative effect Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000002679 ablation Methods 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 201000007270 liver cancer Diseases 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 206010016654 Fibrosis Diseases 0.000 description 1
- 206010027476 Metastases Diseases 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000034994 death Effects 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000013399 early diagnosis Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000004761 fibrosis Effects 0.000 description 1
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 1
- 230000008595 infiltration Effects 0.000 description 1
- 238000001764 infiltration Methods 0.000 description 1
- 210000004185 liver Anatomy 0.000 description 1
- 210000005228 liver tissue Anatomy 0.000 description 1
- 210000002751 lymph Anatomy 0.000 description 1
- 230000009401 metastasis Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000017074 necrotic cell death Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 239000002994 raw material Substances 0.000 description 1
- 238000002271 resection Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000010186 staining Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 210000004881 tumor cell Anatomy 0.000 description 1
- 238000005406 washing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/28—Indexing scheme for image data processing or generation, in general involving image processing hardware
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20092—Interactive image processing based on input by user
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
Abstract
The invention provides a hepatocellular carcinoma pathological image segmentation system, and belongs to the field of medical image processing. The cell cancer pathological image segmentation system comprises the following modules: the system comprises a data preprocessing module, a segmentation model training module and a pathology image segmentation module. The segmentation system of the hepatocellular carcinoma pathological image can obviously improve the segmentation precision of the hepatocellular carcinoma pathological image, can simultaneously and accurately segment two tumor tissues of a whole tumor and a survival tumor, has wide application prospect in hepatocellular carcinoma pathological auxiliary diagnosis, and is also beneficial to pertinently treating the survival tumor area in clinical application. On the basis of the hepatocellular carcinoma pathological image segmentation system, the hepatocellular carcinoma pathological image segmentation system is further optimized and deployed to NVIDIA Jetson TX2 embedded equipment through a TensorRT reasoning acceleration engine, a visual interactive interface and functions of the hepatocellular carcinoma pathological image segmentation are designed and realized, and the clinical application prospect is wide.
Description
Technical Field
The invention belongs to the field of medical image processing, and particularly relates to a hepatocellular carcinoma pathological image segmentation system.
Background
Hepatocellular carcinoma is one of the leading causes of death due to cancer worldwide. The pathological diagnosis of hepatocellular carcinoma provides important reference values for the treatment and prognosis of patients, and better prognosis can be provided for hepatocellular carcinoma patients only by early discovery and early diagnosis. Pathologists typically use hematoxylin-eosin stained hepatocellular carcinoma histopathological sections under a microscope to obtain cell-level information for use in pathological diagnosis and scientific research. The digital pathological image is a high-definition image obtained by digitizing a stained tissue section by using a visualization technology, and the digitized pathological section enables hepatocellular carcinoma tissues to be stored and analyzed more easily.
Generally, pathological section analysis of hepatocellular carcinoma requires a professional pathologist to observe a tissue section in all directions with the naked eye under an auxiliary instrument such as a microscope, and to perform diagnostic analysis on the pathological section in combination with his own clinical practice experience. However, the phenomenon of fatigue reading of the film by the doctors of the pathology department is not exhaustive due to the shortage of the doctors of the pathology department and the complicated workload. In addition, the pathological diagnosis usually has strong individual subjectivity, and the professional knowledge and clinical experience of the pathologist directly influence the result of the pathological diagnosis, so that the risk of misdiagnosis exists in the pathological diagnosis. Therefore, the pathologist often performs repeated work to avoid the possibility of misdiagnosis and missed diagnosis, which also causes a problem that the pathologist consumes a lot of energy but works inefficiently. The deep learning image processing algorithm is applied to hepatocellular carcinoma pathology auxiliary diagnosis, and is optimally deployed to a proper hardware platform to provide support for a pathology image automatic segmentation task, so that the workload of a pathology department doctor can be reduced, the problem of insufficient medical resources is solved, the misdiagnosis rate of pathology diagnosis can be reduced, a pathology diagnosis result with reference significance is given, and the method has very important significance in clinic and scientific research.
Chinese patent application No. CN202110731127.0 discloses an analysis method and system for prognosis of postoperative recurrence of early hepatocellular carcinoma based on artificial intelligence, which can predict postoperative recurrence risk and prognosis of early hepatocellular carcinoma by using complete digital hematoxylin-eosin (HE) stained histopathological section, and the specific steps are as follows: acquiring a full-field digital slide (WSI) image of a specimen to be analyzed; extracting a foreground region of interest (ROI) from the WSI image; different regions of the digitized pathological image are identified through deep learning. The postoperative recurrence risk and prognosis of early hepatocellular carcinoma are effectively analyzed directly according to the HE stained section, and the accuracy is high while various different cell areas are identified, so that the method has great guiding significance on recurrence of patients. However, on the one hand, the prediction system is mainly used for predicting the postoperative recurrence risk and prognosis of early hepatocellular carcinoma, although six types of tissue cell regions, i.e., tumor tissue, normal liver tissue, fibrosis region, assembler region, lymph region and necrosis region, in the WSI image can be identified through the deep learning classification model, the Whole Tumor (WT) and the survival tumor (viable tumor, VT) cannot be accurately segmented, and the survival tumor region cannot be treated in a targeted manner. On the other hand, the prediction system has no visual interactive interface and function, and is not intuitive and convenient to operate.
Disclosure of Invention
The invention aims to provide a hepatocellular carcinoma pathological image segmentation system, and the invention also aims to provide a hepatocellular carcinoma pathological image segmentation visualization interactive system.
The invention provides a hepatocellular carcinoma pathological image segmentation system, which comprises the following modules:
the data preprocessing module comprises: preprocessing a hepatocellular carcinoma pathological image;
(II) a segmentation model training module: on the basis of a UNet network structure, a residual error module and an attention mechanism module are fused to construct a ResAtt-UNet network model; training a ResAtt-UNet network model by utilizing a hepatocellular carcinoma pathological image preprocessed by a training set to obtain a segmentation model;
(III) pathological image segmentation module: and (5) segmenting the hepatocellular carcinoma pathological image preprocessed by the test set by utilizing the segmentation model, and outputting a segmentation result.
Further, in the data preprocessing module, the preprocessing mode includes tissue region extraction, image block segmentation, data amplification and color standardization.
Further, the tissue area extraction can be realized by identifying and distinguishing blank background areas of the hepatocellular carcinoma pathological image in advance, and the tissue areas in the hepatocellular carcinoma pathological image are extracted independently;
the segmented image block can segment the tissue area of the hepatocellular carcinoma pathological image into image blocks with the size of 256 multiplied by 256 pixels;
the data amplification mode is data turning and rotation;
the color normalization adopts an algorithm which is a dyeing separation color normalization algorithm based on sparse nonnegative matrix decomposition.
Further, in the segmentation model training module, the position introduced by the residual module is in a convolution layer of the UNet network structure, and the position introduced by the attention mechanism module is a jump connection part of the UNet network structure.
Further, when training the ResAtt-UNet network model, the parameters are set as follows: an Adam optimizer is used, the learning rate is 1e-4, the batch size is 32, and 100epochs are iterated altogether.
The invention also provides a hepatocellular carcinoma pathological image segmentation visualization interaction system which comprises the hepatocellular carcinoma pathological image segmentation system and an embedded deployment part.
Further, the embedded deployment part comprises an NVIDIA Jetson TX2 embedded device and peripheral hardware thereof.
Furthermore, the peripheral hardware comprises a reset circuit, a power supply module, an electronic eyepiece module, a memory and input and output interaction equipment.
Further, the hepatocellular carcinoma pathology image segmentation system is deployed to the NVIDIA Jetson TX2 embedded device through a TensorRT inference acceleration engine.
The invention also provides the application of the hepatocellular carcinoma pathological image segmentation system and the hepatocellular carcinoma pathological image segmentation visualization interaction system in preparation of medical equipment for segmenting hepatocellular carcinoma pathological images.
Viable Tumors (VT) refer only to areas of malignant tumors where tumor cells remain viable and may undergo peripheral infiltration, distant metastasis, etc. Whole tumor (WholeTumor, WT) refers to all tumor regions, including surviving tumor fractions and regions previously treated with liver cancer intervention (TACE) and the like.
The experimental result shows that the hepatocellular carcinoma pathological image segmentation system can obviously improve the accuracy of hepatocellular carcinoma pathological image segmentation, can simultaneously and accurately segment two tumor tissues, namely a whole tumor (WholeTumor, WT) and a survival tumor (VibleTumor, VT), has wide application prospect in hepatocellular carcinoma pathological auxiliary diagnosis, and is also beneficial to pertinently treating the survival tumor area in clinical application.
On the basis of the hepatocellular carcinoma pathological image segmentation system, the hepatocellular carcinoma pathological image segmentation system is further optimized and deployed to NVIDIA Jetson TX2 embedded equipment through a TensorRT reasoning acceleration engine, a visual interactive interface and functions of the hepatocellular carcinoma pathological image segmentation are designed and realized, and the clinical application prospect is wide.
Obviously, many modifications, substitutions, and variations are possible in light of the above teachings of the invention, without departing from the basic technical spirit of the invention, as defined by the following claims.
The present invention will be described in further detail with reference to the following examples. This should not be understood as limiting the scope of the above-described subject matter of the present invention to the following examples. All the technologies realized based on the above contents of the present invention belong to the scope of the present invention.
Drawings
Fig. 1 is a schematic diagram of a data preprocessing flow of the hepatocellular carcinoma pathological image segmentation system in embodiment 1.
Fig. 2 is a schematic diagram of the ResAtt-UNet network model in embodiment 1.
Fig. 3 is a graph of network loss convergence.
Fig. 4 is an example of segmentation result of hepatocellular carcinoma pathology image in embodiment 1. (a) An original image; (b) WT gold standard image; (c) WT segmenting a result image; (d) VT golden standard image; (e) VT segments the resulting image.
Fig. 5 is a schematic structural design diagram of the cell cancer pathology image segmentation visualization interaction system in embodiment 2.
Fig. 6 is a schematic block diagram of an embedded end-pathology image segmentation visualization application system according to embodiment 2.
FIG. 7 is a flowchart illustrating an interactive system for segmentation and visualization of hepatocellular carcinoma pathology images in accordance with example 2.
FIG. 8 is a main interface of an application of the interactive system for segmentation visualization of hepatocellular carcinoma pathology image in accordance with embodiment 2.
Fig. 9 is an example of the operation of the hepatocellular carcinoma pathological image segmentation visualization interaction system in embodiment 2. (a) Application operation; (b) and (5) pathological image segmentation results.
Fig. 10 is a schematic diagram of the overall design scheme of the cell cancer pathology image segmentation visualization interaction system in example 2.
Detailed Description
The raw materials and equipment used in the invention are known products, and are obtained by purchasing products sold in the market.
The hepatocellular carcinoma pathology images used in the following examples were derived from PAIP2019 hepatocellular carcinoma pathology image data set and pathology image data set in washings hospital of the university of sichuan, all using H & E staining and anonymized. The PAIP2019 hepatocellular carcinoma pathological image data set is constructed by hepatocellular carcinoma pathological files acquired from the period of 2000 to 2018 in the Seoul national university Hospital. The patients are all liver cancer patients who receive liver resection operation for the first time.
Embodiment 1 construction of hepatocellular carcinoma pathological image segmentation System
Step 1: data pre-processing
The hepatocellular carcinoma pathological image is preprocessed to enable the attributes of the hepatocellular carcinoma pathological image to be uniform, influences of irrelevant factors on a segmentation algorithm are reduced, the convergence rate of model training is increased, and therefore the performance of the segmentation model is improved. The pretreatment module of the hepatocellular carcinoma pathological image is sequentially divided into a tissue region extraction module, a segmentation image block module, a data amplification module and a color standardization module (shown in figure 1).
The tissue region extraction module identifies and distinguishes blank background regions of the pathological images in advance, and individually extracts tissue regions in the pathological images, so that data of the training network only contain tissue samples.
If the whole pathological image is directly used as training data, great challenges exist on the performances of a computer memory and the like, and the whole pathological image needs to be subjected to image block segmentation operation, segmented into small-size image blocks and then subjected to subsequent operation. The image block segmentation module firstly rescales the hepatocellular carcinoma pathological image to a proper magnification ratio, and then uses a non-overlapping sliding window scheme to segment the tissue area of the pathological image into image blocks with the size of 256 multiplied by 256 pixels.
In order to fully utilize the segmented pathological image blocks, effective data amplification processing is carried out on the segmented pathological image blocks in a data amplification module, and in order to ensure the authenticity of pathological image data, only data is turned and rotated, so that a network model keeps unchanged on geometric disturbance.
In order to solve the problem that colors are different during pathological section dyeing preparation, the color standardization module reduces the influence of color difference on a model algorithm through a color standardization processing method, the color standardization module adopts a dyeing separation color standardization algorithm based on sparse nonnegative matrix decomposition, and the principle of the color standardization module is that the original image and the target image are accurately separated in color based on a sparse nonnegative matrix decomposition regularization method, so that the effect of matching the colors of the target image is achieved.
Step 2: segmentation model training
Based on the UNet network structure, a residual error module and an attention mechanism module are fused to construct a ResAtt-UNet network model (as shown in FIG. 2) of the invention. The construction method specifically comprises the following steps:
a residual error module is introduced into a convolution layer of the UNet network, the problem of gradient disappearance is solved, the features of the front layer are fully expressed to the features of the next layer, and deeper image semantic information is effectively extracted. On the basis, an attention mechanism module is introduced into a jump connection part of the UNet network, noise information in the connection processing of a low layer and a high layer of the network is fully filtered, useful low-level features are enhanced, irrelevant areas in pathological images are restrained, and a tumor tissue segmentation effect with small scale is improved, so that the prediction accuracy and the sensitivity of a hepatocellular carcinoma pathological image segmentation model are improved.
And carrying out segmentation model training on the basis of the ResAtt-UNet network model. The training optimization method adopts an Adam optimizer, the learning rate is set to be 1e-4, the batch size is set to be 32, the iteration is performed in total at 100epochs, and the training model and related parameters thereof are automatically saved every 1 epoch. Model training reaches a stop condition when the loss values converge and tend to plateau. Fig. 3 shows a network loss convergence graph.
And step 3: hepatocellular carcinoma pathology image segmentation
After the segmentation model is trained, the model is applied to a hepatocellular carcinoma pathological image to be predicted by using a sliding window inference method so as to obtain a final segmentation result.
The following is a verification of the segmentation effect of the hepatocellular carcinoma pathological image in example 1.
Randomly selecting segmentation data of the hepatocellular carcinoma pathological image, and performing visual verification on the segmentation result of the pathological image. Fig. 4 is an example of segmentation result of hepatocellular carcinoma pathology image. By observing pathological image segmentation results, the ResAtt-UNet network model shows excellent segmentation precision on two tumor tissues, namely whole tumor (WholeTumor, WT) and survival tumor (VibleTumor, VT), and the two tumor tissues can be accurately distinguished by the segmentation model.
In order to better evaluate the influence of the introduced residual module and the attention mechanism module on the segmentation performance of the UNet basic network model, an ablation experiment among modules is carried out. The Dice coefficient is used as an evaluation index of the segmentation precision of the segmentation model, the numeric area of the Dice coefficient is 0-1, and the closer the value is to 1, the better the segmentation precision is.
The experimental index pair ratios among different modules are shown in table 1, and the sequence numbers are from top to bottom: a UNet base network without introducing other modules, an UNet network with introducing a residual module, an UNet network with introducing an attention mechanism, and a ResAtt-UNet network with introducing the residual module and the attention mechanism module at the same time.
TABLE 1 ablation experiments between network model modules
| Serial number | ResBlock | AttentionBlock | WT Dice | VT Dice |
| 1 | - | - | 0.8176 | 0.7803 |
| 2 | √ | - | 0.8261 | 0.7885 |
| 3 | - | √ | 0.8248 | 0.7962 |
| 4 | √ | √ | 0.8362 | 0.8049 |
It can be seen that, compared with UNet basic network model, when the basic network only introduces residual module, the average Dice coefficient values of two types of tumor labels (WT, VT) of hepatocellular carcinoma pathological image are improved, respectively 0.8261, 0.7885; when only the attention mechanism module is introduced, the average Dice coefficient values of two types of tumor labels (WT, VT) of the hepatocellular carcinoma pathological image are improved, wherein the values are 0.8248 and 0.7962; when the residual module and the attention mechanism module are fused into the UNet basic network, the segmentation effect of the two types of tumor labels (WT, VT) of the hepatocellular carcinoma pathological image is the best, the average values of the WT Dice and the VT Dice are 0.8362 and 0.8049 respectively, and the evaluation index values are improved by 2.27 percent and 3.15 percent respectively compared with those of the UNet basic network Dice.
The results show that the hepatocellular carcinoma pathological image segmentation system can obviously improve the accuracy of hepatocellular carcinoma pathological image segmentation, and can simultaneously and accurately segment two tumor tissues, namely a whole tumor (WholeTumor, WT) and a survival tumor (VibleTumor, VT).
Embodiment 2, construction of hepatocellular carcinoma pathological image segmentation visualization interaction system
The hepatocellular carcinoma pathological image segmentation visualization interaction system of the present embodiment sequentially includes a hardware layer, a system layer, a data layer, an algorithm layer, and an application layer from bottom to top (as shown in fig. 5).
1. Hardware layer: based on the NVIDIA Jetson TX2 embedded device, the peripheral hardware comprises an electronic eyepiece module, a memory and an input-output interaction device so as to establish basic operation conditions.
2. A system layer: the system layer is used for distributing and coordinating software and hardware resources, and the NVIDIA provides a JetPack system environment which is adapted to Jetson TX2 and comprises a Linux operating system, a computer vision processing library and the like. Basic GPU related functions and image processing algorithms can be called through system API interface functions, and the portability and the expansibility of the scheme are improved.
3. And (3) a data layer: the method comprises data generated by processing pathological images, cache data to be called and user instruction data transmitted by an application layer. The data layer provides efficient data support for the algorithm layer and receives or transmits relevant data from the application layer.
4. And an algorithm layer: the algorithm layer is the hepatocellular carcinoma pathology image segmentation system of example 1. And when the algorithm layer is called by the application layer, the pathological image segmentation result processed by the segmentation system is returned to the application layer.
5. An application layer: the application layer implements a visual application using a Qt application development software design. The application program interacts with the user most closely, and the functions of data input, data processing, data viewing, data storage and the like of the whole system are simultaneously realized, so that the function of the interactive interface is directly realized. The user transmits the instruction to the application program, and the application program executes corresponding functions and displays the segmentation result of the hepatocellular carcinoma pathological image to the user.
Embodiment 2 a module schematic diagram of an embedded end pathology image segmentation visualization application system is shown in fig. 6. The application comprises five functional modules, namely a pathological image reading input module, a basic option module, a pathological image preliminary processing module, a pathological image segmentation option module and a pathological image segmentation display module, and aims to realize image segmentation processing on pathological images through a Jetson TX2 device. The application program adopts a modular design, and the maintainability and the usability of the application are effectively improved.
Embodiment 2 application main interface of hepatocellular carcinoma pathology image segmentation visualization interactive system is shown in fig. 8. The pathological image reading and inputting module is an entrance of a visual application program data source and is responsible for inputting pathological image data for processing by a subsequent functional module; the basic option module provides basic auxiliary operation options, such as cancel and reset functions; the pathological image preliminary processing module carries out preliminary processing on the input pathological image, eliminates some interference factors and ensures that a better segmentation effect can be obtained at last; the pathological image segmentation option module uses a model optimized and accelerated by TensorRT to segment the image, and can store the result image of the model segmentation; the pathological image segmentation display module is used for visually displaying the processing result.
In order to obtain a better reasoning effect on the hepatocellular carcinoma pathological image segmentation model deployed on the Jetson TX2 device, a TensorRT reasoning acceleration engine is adopted to perform optimized deployment on the model. Opening a segmentation application of the hepatocellular carcinoma pathological image in a Jetson TX2 device, loading a pathological image with tumor tissue and a segmentation model of the hepatocellular carcinoma pathological image, segmenting the pathological image, and displaying a segmentation result after the segmentation is completed, wherein fig. 9 is an operation example of the hepatocellular carcinoma pathological image segmentation visualization interaction system in embodiment 2.
In the cellular cancer pathological image segmentation visualization interaction system of embodiment 2 of the present invention, the hepatocellular cancer pathological image segmentation system of embodiment 1 is optimally deployed to NVIDIA Jetson TX2 embedded device by using a TensorRT reasoning acceleration engine, and a visualization interaction interface and functions thereof for segmenting the hepatocellular cancer pathological image are designed and implemented (as shown in fig. 10).
In summary, the present invention provides a segmentation system for pathological images of hepatocellular carcinoma. The segmentation system of the hepatocellular carcinoma pathological image can obviously improve the segmentation precision of the hepatocellular carcinoma pathological image, can simultaneously and accurately segment two tumor tissues of a whole tumor and a survival tumor, has wide application prospect in hepatocellular carcinoma pathological auxiliary diagnosis, and is also beneficial to pertinently treating the survival tumor area in clinical application. On the basis of a hepatocellular carcinoma pathological image segmentation system, the hepatocellular carcinoma pathological image segmentation system is further optimized and deployed to NVIDIA Jetson TX2 embedded equipment through a TensorRT reasoning acceleration engine, a hepatocellular carcinoma pathological image segmentation visual interactive interface and functions thereof are designed and realized, and the clinical application prospect is wide.
Claims (10)
1. A hepatocellular carcinoma pathology image segmentation system, characterized by: the system comprises the following modules:
the data preprocessing module comprises: preprocessing a hepatocellular carcinoma pathological image;
(II) a segmentation model training module: on the basis of a UNet network structure, a residual error module and an attention mechanism module are fused to construct a ResAtt-UNet network model; training a ResAtt-UNet network model by utilizing a hepatocellular carcinoma pathological image preprocessed by a training set to obtain a segmentation model;
(III) pathological image segmentation module: and (5) segmenting the hepatocellular carcinoma pathological image preprocessed by the test set by utilizing the segmentation model, and outputting a segmentation result.
2. The hepatocellular carcinoma pathology image segmentation system of claim 1, characterized in that: in the data preprocessing module, the preprocessing mode comprises tissue region extraction, image block segmentation, data amplification and color standardization.
3. The hepatocellular carcinoma pathology image segmentation system of claim 2, characterized in that: the tissue area extraction can be realized by identifying and distinguishing blank background areas of the hepatocellular carcinoma pathological image in advance, and the tissue areas in the hepatocellular carcinoma pathological image are extracted independently;
the segmentation of the image blocks can segment the tissue area of the hepatocellular carcinoma pathological image into image blocks with the size of 256 multiplied by 256 pixels;
the data amplification mode is data turning and rotation;
the color normalization adopts an algorithm which is a dyeing separation color normalization algorithm based on sparse nonnegative matrix decomposition.
4. The hepatocellular carcinoma pathology image segmentation system of claim 1, characterized in that: in the segmentation model training module, the position introduced by the residual error module is in a convolution layer of the UNet network structure, and the position introduced by the attention mechanism module is a jump connection part of the UNet network structure.
5. The hepatocellular carcinoma pathology image segmentation system of claim 4, characterized in that: when training the ResAtt-UNet network model, the parameters are set as follows: an Adam optimizer is used, the learning rate is 1e-4, the batch size is 32, and 100epochs are iterated altogether.
6. A hepatocellular carcinoma pathological image segmentation visualization interactive system is characterized in that: it comprises the hepatocellular carcinoma pathological image segmentation system and an embedded deployment part according to any one of claims 1 to 5.
7. The hepatocellular carcinoma pathology image segmentation visualization interactive system according to claim 6, characterized in that: the embedded deployment portion includes an NVIDIA Jetson TX2 embedded device and its peripheral hardware.
8. The interactive system for visualization of hepatocellular carcinoma pathology image segmentation described in claim 7, wherein: the peripheral hardware comprises a reset circuit, a power supply module, an electronic eyepiece module, a memory and input and output interaction equipment.
9. The hepatocellular carcinoma pathology image segmentation visualization interactive system according to claim 7, characterized in that: the hepatocellular carcinoma pathological image segmentation system is deployed to a NVIDIA Jetson TX2 embedded device through a TensorRT reasoning acceleration engine.
10. Use of the hepatocellular carcinoma pathological image segmentation system according to any one of claims 1 to 5 and the hepatocellular carcinoma pathological image segmentation visualization interaction system according to any one of claims 6 to 9 in preparation of a medical device for segmenting hepatocellular carcinoma pathological images.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210698119.5A CN114937045A (en) | 2022-06-20 | 2022-06-20 | Hepatocellular carcinoma pathological image segmentation system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210698119.5A CN114937045A (en) | 2022-06-20 | 2022-06-20 | Hepatocellular carcinoma pathological image segmentation system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114937045A true CN114937045A (en) | 2022-08-23 |
Family
ID=82868290
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210698119.5A Pending CN114937045A (en) | 2022-06-20 | 2022-06-20 | Hepatocellular carcinoma pathological image segmentation system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114937045A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115880262A (en) * | 2022-12-20 | 2023-03-31 | 桂林电子科技大学 | Weakly supervised pathological image tissue segmentation method based on online noise suppression strategy |
| CN116230214A (en) * | 2023-05-08 | 2023-06-06 | 浙江大学滨江研究院 | HCC and VETC auxiliary diagnosis device and equipment |
| CN116682576A (en) * | 2023-08-02 | 2023-09-01 | 浙江大学 | Liver cancer pathological prognosis system and device based on double-layer graph convolutional neural network |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108765408A (en) * | 2018-05-31 | 2018-11-06 | 杭州同绘科技有限公司 | Build the method in cancer pathology image virtual case library and the multiple dimensioned cancer detection system based on convolutional neural networks |
| CN109191472A (en) * | 2018-08-28 | 2019-01-11 | 杭州电子科技大学 | Based on the thymocyte image partition method for improving U-Net network |
| CN110570432A (en) * | 2019-08-23 | 2019-12-13 | 北京工业大学 | CT image liver tumor segmentation method based on deep learning |
| CN111627019A (en) * | 2020-06-03 | 2020-09-04 | 西安理工大学 | Liver tumor segmentation method and system based on convolutional neural network |
| CN111709952A (en) * | 2020-05-21 | 2020-09-25 | 无锡太湖学院 | MRI brain tumor automatic segmentation method based on edge feature optimization and double-flow decoding convolutional neural network |
| CN112101451A (en) * | 2020-09-14 | 2020-12-18 | 北京联合大学 | Breast cancer histopathology type classification method based on generation of confrontation network screening image blocks |
| CN112435243A (en) * | 2020-11-26 | 2021-03-02 | 山东第一医科大学附属省立医院(山东省立医院) | Automatic analysis system and method for full-slice digital pathological image |
| WO2021088747A1 (en) * | 2019-11-04 | 2021-05-14 | 中国人民解放军总医院 | Deep-learning-based method for predicting morphological change of liver tumor after ablation |
| CN113256617A (en) * | 2021-06-23 | 2021-08-13 | 重庆点检生物科技有限公司 | Pathological section virtual immunohistochemical staining method and system |
| CN113469073A (en) * | 2021-07-06 | 2021-10-01 | 西安电子科技大学 | SAR image ship detection method and system based on lightweight deep learning |
| CN113628220A (en) * | 2021-07-06 | 2021-11-09 | 山东师范大学 | Method and system for segmenting MRI brain tumor image based on improved U-Net network |
| US20210383538A1 (en) * | 2018-07-30 | 2021-12-09 | Memorial Sloan Kettering Cancer Center | Multi-modal, multi-resolution deep learning neural networks for segmentation, outcomes prediction and longitudinal response monitoring to immunotherapy and radiotherapy |
| CN113903471A (en) * | 2021-09-24 | 2022-01-07 | 上海交通大学 | Gastric cancer patient survival risk prediction method based on histopathology image and gene expression data |
| CN114565867A (en) * | 2022-01-25 | 2022-05-31 | 大连理工大学 | Unmanned aerial vehicle scene video target detection method based on convolutional neural network |
-
2022
- 2022-06-20 CN CN202210698119.5A patent/CN114937045A/en active Pending
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108765408A (en) * | 2018-05-31 | 2018-11-06 | 杭州同绘科技有限公司 | Build the method in cancer pathology image virtual case library and the multiple dimensioned cancer detection system based on convolutional neural networks |
| US20210383538A1 (en) * | 2018-07-30 | 2021-12-09 | Memorial Sloan Kettering Cancer Center | Multi-modal, multi-resolution deep learning neural networks for segmentation, outcomes prediction and longitudinal response monitoring to immunotherapy and radiotherapy |
| CN109191472A (en) * | 2018-08-28 | 2019-01-11 | 杭州电子科技大学 | Based on the thymocyte image partition method for improving U-Net network |
| CN110570432A (en) * | 2019-08-23 | 2019-12-13 | 北京工业大学 | CT image liver tumor segmentation method based on deep learning |
| WO2021088747A1 (en) * | 2019-11-04 | 2021-05-14 | 中国人民解放军总医院 | Deep-learning-based method for predicting morphological change of liver tumor after ablation |
| CN111709952A (en) * | 2020-05-21 | 2020-09-25 | 无锡太湖学院 | MRI brain tumor automatic segmentation method based on edge feature optimization and double-flow decoding convolutional neural network |
| CN111627019A (en) * | 2020-06-03 | 2020-09-04 | 西安理工大学 | Liver tumor segmentation method and system based on convolutional neural network |
| CN112101451A (en) * | 2020-09-14 | 2020-12-18 | 北京联合大学 | Breast cancer histopathology type classification method based on generation of confrontation network screening image blocks |
| CN112435243A (en) * | 2020-11-26 | 2021-03-02 | 山东第一医科大学附属省立医院(山东省立医院) | Automatic analysis system and method for full-slice digital pathological image |
| CN113256617A (en) * | 2021-06-23 | 2021-08-13 | 重庆点检生物科技有限公司 | Pathological section virtual immunohistochemical staining method and system |
| CN113469073A (en) * | 2021-07-06 | 2021-10-01 | 西安电子科技大学 | SAR image ship detection method and system based on lightweight deep learning |
| CN113628220A (en) * | 2021-07-06 | 2021-11-09 | 山东师范大学 | Method and system for segmenting MRI brain tumor image based on improved U-Net network |
| CN113903471A (en) * | 2021-09-24 | 2022-01-07 | 上海交通大学 | Gastric cancer patient survival risk prediction method based on histopathology image and gene expression data |
| CN114565867A (en) * | 2022-01-25 | 2022-05-31 | 大连理工大学 | Unmanned aerial vehicle scene video target detection method based on convolutional neural network |
Non-Patent Citations (2)
| Title |
|---|
| QIANGGUO JIN等: "RA-UNet: A hybrid deep attention-aware network to extract liver and tumor in CT scans" * |
| 蔡阿昌: "基于深度学习的核磁脑肿瘤分割研究" * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115880262A (en) * | 2022-12-20 | 2023-03-31 | 桂林电子科技大学 | Weakly supervised pathological image tissue segmentation method based on online noise suppression strategy |
| CN115880262B (en) * | 2022-12-20 | 2023-09-05 | 桂林电子科技大学 | Weak supervision pathological image tissue segmentation method based on online noise suppression strategy |
| US11935279B1 (en) | 2022-12-20 | 2024-03-19 | Guilin University Of Electronic Technology | Weakly supervised pathological image tissue segmentation method based on online noise suppression strategy |
| CN116230214A (en) * | 2023-05-08 | 2023-06-06 | 浙江大学滨江研究院 | HCC and VETC auxiliary diagnosis device and equipment |
| CN116230214B (en) * | 2023-05-08 | 2023-08-01 | 浙江大学滨江研究院 | HCC and VETC auxiliary diagnosis device and equipment |
| CN116682576A (en) * | 2023-08-02 | 2023-09-01 | 浙江大学 | Liver cancer pathological prognosis system and device based on double-layer graph convolutional neural network |
| CN116682576B (en) * | 2023-08-02 | 2023-12-19 | 浙江大学 | Liver cancer pathological prognosis system and device based on double-layer graph convolutional neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Dimitriou et al. | Deep learning for whole slide image analysis: an overview | |
| Li et al. | Classification of breast cancer histology images using multi-size and discriminative patches based on deep learning | |
| Yang et al. | Prediction of HER2-positive breast cancer recurrence and metastasis risk from histopathological images and clinical information via multimodal deep learning | |
| CN113454733B (en) | Multi-instance learner for prognostic tissue pattern recognition | |
| Aswathy et al. | Detection of breast cancer on digital histopathology images: Present status and future possibilities | |
| CN114937045A (en) | Hepatocellular carcinoma pathological image segmentation system | |
| CN112106061A (en) | Method and system for digital staining of unlabeled fluorescent images using deep learning | |
| Xu et al. | Colorectal cancer detection based on deep learning | |
| Xie et al. | Interpretable classification from skin cancer histology slides using deep learning: A retrospective multicenter study | |
| EP2774067A2 (en) | Generate percentage of positive cells for biomarkers by normalizing and autothresholding the image intensity produced by immunohistochemistry technique | |
| JP6392476B1 (en) | Biological tissue analysis apparatus and biological tissue analysis program | |
| Nateghi et al. | A deep learning approach for mitosis detection: application in tumor proliferation prediction from whole slide images | |
| Li et al. | Automated gleason grading and gleason pattern region segmentation based on deep learning for pathological images of prostate cancer | |
| Gabriel et al. | Towards high performance cell segmentation in multispectral fine needle aspiration cytology of thyroid lesions | |
| WO2022066725A1 (en) | Training end-to-end weakly supervised networks in a multi-task fashion at the specimen (supra-image) level | |
| Wang et al. | Automatic generation of pathological benchmark dataset from hyperspectral images of double stained tissues | |
| Dabass et al. | A hybrid U-Net model with attention and advanced convolutional learning modules for simultaneous gland segmentation and cancer grade prediction in colorectal histopathological images | |
| Moghadam et al. | Artificial Intelligence and Pathomics: Prostate Cancer | |
| Pradhan et al. | Data Fusion of Histological and Immunohistochemical Image Data for Breast Cancer Diagnostics using Transfer Learning. | |
| Wu et al. | Segmentation of HE-stained meningioma pathological images based on pseudo-labels | |
| EP4214626A1 (en) | Training end-to-end weakly supervised networks at the specimen (supra-image) level | |
| yahia Ibrahim et al. | An enhancement technique to diagnose colon and lung cancer by using double CLAHE and deep learning | |
| Hoque et al. | Stain normalization methods for histopathology image analysis: A comprehensive review and experimental comparison | |
| Liu et al. | Direct gene expression profile prediction for uveal melanoma from digital cytopathology images via deep learning and salient image region identification | |
| CN114445374A (en) | Image feature processing method and system based on diffusion kurtosis imaging MK image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220823 |