CN114202510A - Intelligent analysis system for pathological section images under microscope - Google Patents
Intelligent analysis system for pathological section images under microscope Download PDFInfo
- Publication number
- CN114202510A CN114202510A CN202111332278.5A CN202111332278A CN114202510A CN 114202510 A CN114202510 A CN 114202510A CN 202111332278 A CN202111332278 A CN 202111332278A CN 114202510 A CN114202510 A CN 114202510A
- Authority
- CN
- China
- Prior art keywords
- pathological section
- section image
- image
- microscope
- pathological
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000001575 pathological effect Effects 0.000 title claims abstract description 247
- 238000004458 analytical method Methods 0.000 title claims abstract description 43
- 238000012545 processing Methods 0.000 claims abstract description 62
- 238000002372 labelling Methods 0.000 claims abstract description 40
- 238000012937 correction Methods 0.000 claims abstract description 26
- 238000003745 diagnosis Methods 0.000 claims abstract description 21
- 230000003044 adaptive effect Effects 0.000 claims abstract description 10
- 230000015654 memory Effects 0.000 claims abstract description 7
- 230000002708 enhancing effect Effects 0.000 claims abstract description 4
- 238000000034 method Methods 0.000 claims description 32
- 238000001514 detection method Methods 0.000 claims description 26
- 230000003993 interaction Effects 0.000 claims description 22
- 238000005457 optimization Methods 0.000 claims description 20
- 230000002452 interceptive effect Effects 0.000 claims description 14
- 230000008569 process Effects 0.000 claims description 13
- 238000012217 deletion Methods 0.000 claims description 11
- 230000037430 deletion Effects 0.000 claims description 11
- 238000012549 training Methods 0.000 claims description 10
- 238000010191 image analysis Methods 0.000 claims description 8
- 230000007170 pathology Effects 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 4
- 230000006978 adaptation Effects 0.000 claims description 3
- 238000012790 confirmation Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 210000004027 cell Anatomy 0.000 description 20
- 230000006870 function Effects 0.000 description 14
- 201000010099 disease Diseases 0.000 description 11
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 11
- 238000003780 insertion Methods 0.000 description 9
- 230000037431 insertion Effects 0.000 description 9
- 238000003759 clinical diagnosis Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000003860 storage Methods 0.000 description 5
- 230000000007 visual effect Effects 0.000 description 5
- 238000013461 design Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000010827 pathological analysis Methods 0.000 description 3
- 206010008342 Cervix carcinoma Diseases 0.000 description 2
- 206010028980 Neoplasm Diseases 0.000 description 2
- 208000006105 Uterine Cervical Neoplasms Diseases 0.000 description 2
- 201000010881 cervical cancer Diseases 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 239000003814 drug Substances 0.000 description 2
- 102000015694 estrogen receptors Human genes 0.000 description 2
- 108010038795 estrogen receptors Proteins 0.000 description 2
- 238000003364 immunohistochemistry Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 206010006187 Breast cancer Diseases 0.000 description 1
- 208000026310 Breast neoplasm Diseases 0.000 description 1
- 101150029707 ERBB2 gene Proteins 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000004663 cell proliferation Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 239000003102 growth factor Substances 0.000 description 1
- 108091008039 hormone receptors Proteins 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 230000003211 malignant effect Effects 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000012528 membrane Substances 0.000 description 1
- 238000012758 nuclear staining Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 230000001072 progestational effect Effects 0.000 description 1
- 238000004445 quantitative analysis Methods 0.000 description 1
- 238000010186 staining Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 230000007474 system interaction Effects 0.000 description 1
- 210000004881 tumor cell Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/01—Arrangements or apparatus for facilitating the optical investigation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
- G06F18/2178—Validation; Performance evaluation; Active pattern learning techniques based on feedback of a supervisor
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H15/00—ICT specially adapted for medical reports, e.g. generation or transmission thereof
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
- G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
- G16H40/67—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for remote operation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Public Health (AREA)
- Epidemiology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Primary Health Care (AREA)
- Evolutionary Biology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Pathology (AREA)
- Immunology (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Radiology & Medical Imaging (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Business, Economics & Management (AREA)
- Business, Economics & Management (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to an intelligent analysis system for pathological section images under a microscope, which comprises a microscope, a microscope camera, a mechanical arm, a processor, a memory and a display, wherein the microscope camera is arranged on the microscope; the microscope camera is used for collecting images of pathological sections under the microscope, and the mechanical arm realizes the movement of the sections under the microscope objective lens. The processor receives a pathological section image from the microscope camera, the system is a digital pathological section image matching adaptive algorithm model, the pathological section image is processed by an algorithm through intelligent diagnosis to obtain a preliminary diagnosis result, and the labeled pathological section image is output; manually judging the marked pathological section image, and if the marking result meets the requirement, not performing correction marking; otherwise, manually correcting and labeling information of the pathological section image, and outputting a manual judgment result; and processing the artificially corrected pathological section image, retraining the model through the local data, enhancing the adaptability of the model to the local data, and outputting a final pathological report.
Description
Technical Field
The invention belongs to the technical field of pathological section diagnosis, and particularly relates to an intelligent analysis system for pathological section images under a microscope.
Background
At present, the patients with the domestic three-level hospitals, the second-level synthesis, the combination of traditional Chinese and western medicine, foot tumors, children and special hospitals for obstetrics and gynecology department can establish the pathology department.
The high efficiency, accuracy, usability and cost performance of the traditional microscope are all required to be improved, a doctor needs to repeatedly switch between a screen and the microscope, the judgment effect is different from person to person, and pathological sections are manually identified by the doctor. If the visual field under the microscope is digitalized, the accuracy and real-time performance of the visual field under the microscope can meet the actual application requirements of pathological diagnosis, and the visual field can help doctors to better obtain a diagnosis conclusion, so that the reading pressure of the doctors is obviously relieved. Just like stepping on the accelerator when driving, a pathologist can accurately present the result on a screen as soon as stepping on the pedal lightly, and accurate diagnosis and accurate treatment are realized.
The digital auxiliary interpretation processing of deep learning and the construction of the conventional microscope are integrated, so that pathological research can obtain substantial value on the windward vehicle in the current medical field, and an intelligent algorithm based on deep learning is introduced to find a sufficient place.
In China, the intelligent microscope jointly developed by Jinyu medicine, Tencent AI Lab and Shunhu optical science and technology has obtained a registration certificate issued by NMPA, and is the first intelligent microscope product approved for clinical use in China. The intelligent microscope integrates the latest technology in the aspect of pathological analysis and diagnosis at present, and carries out multiple product iterations according to the workflow and habits of pathologists, and now supports the interpretation of common nuclear staining and membrane staining quantitative analysis scenes such as breast cancer Immunohistochemistry (IHC), Ki67 (tumor cell proliferation index), ER (estrogen receptor), PR (progestational hormone receptor) and Her2 (cell surface growth factor). The following defects still exist in the practical application process:
1. the intelligent microscope developed by Tencent in combination with other companies cannot remotely move the section observation position, and the intelligent analysis time is long. Although the algorithm is rich and the automation degree is high, the developed integrated equipment needs to purchase a complete set of software and hardware to use the complete function. Because most of the intelligent microscopes integrating multiple functions are integrated equipment, the selling price is high, and artificial intelligent algorithms developed by other people cannot be added autonomously according to the actual disease condition diagnosis condition of a hospital.
2. Because the doctor can only passively receive the detection result after the pathological image is processed by the current artificial intelligence algorithm, the processing result cannot be manually corrected. Doctors often have a question about this black box mechanism that cannot be manually engaged.
3. Because the microscope and the coloring agent all have differences between hospitals, most of intelligent microscopes have poor diagnosis effect under a fixed model, have poor adaptability to local mode processing, and have low detection result precision.
Disclosure of Invention
In view of the above-mentioned drawbacks and deficiencies of the prior art, an object of the present invention is to provide an intelligent analysis system for pathological section images under a microscope.
In order to realize the task, the invention adopts the following technical solution:
the utility model provides a pathological section image intelligent analysis system under microscope which characterized in that includes:
a microscope for observing pathological sections;
the microscope camera is provided with an acquisition switch, is connected with the lens of the microscope and is used for acquiring pathological section images under the microscope;
the mechanical arm is used for connecting a moving hand wheel of the microscope, controlling the moving platform to move longitudinally and horizontally to move the section, and realizing the switching of pathological section images under the microscope;
the processor is in communication connection with the microscope camera, receives the pathological section image from the microscope camera, preprocesses the pathological section image, and performs cell detection and cell classification on the processed pathological section image;
one or more memories connected with the processor and used for storing programs and saving data;
the display is connected with the processor and used for receiving and displaying the marked pathological section image from the processor in real time;
the Bluetooth equipment is used for information interaction between the mechanical arm and the processor; and
one or more programs stored in the one or more memories and invoked for execution by the processor;
the program comprises instructions for performing the steps of:
1) a microscope camera acquires a digital pathological section image;
2) matching an adaptive algorithm model for the digital pathological section image, carrying out algorithm processing on the pathological section image through intelligent diagnosis to obtain a preliminary diagnosis result, and outputting the labeled pathological section image;
3) manually judging the marked pathological section image, and if the marking result meets the requirement, the accuracy is high, and the correction marking is not carried out; otherwise, manually correcting and labeling information of the pathological section image, and outputting a manual judgment result;
4) processing the artificially corrected pathological section image, retraining the model through the local data, and enhancing the adaptability of the model to the local data;
5) and outputting a final pathology report.
According to the invention, the digital pathological section images collected by the microscope camera are remotely operated without manual operation to move the pathological section, and the method specifically comprises the following steps:
judging whether the mechanical arm needs to be operated or not;
if the mechanical arm does not need to be moved, transmitting the currently acquired pathological section image to a processor, and processing and labeling the pathological section image; otherwise, moving the mechanical arm and judging the moving direction of the mechanical arm;
transmitting a moving direction signal of the mechanical arm through Bluetooth equipment, and controlling the mechanical arm to rotate by a motor in the mechanical arm to drive a microscope moving hand wheel to rotate;
and acquiring a new pathological section image, transmitting the new pathological section image to a processor, and carrying out algorithm processing on the pathological section image.
Further, the one or more programs include image fuzzy recognition, interactive correction pathological section labeling processing and retraining model optimization methods, so that the processing model is adaptive to local data; wherein:
the image fuzzy recognition is used for intelligently recognizing the micro deformation of the pathological section image under the microscope, intelligently recognizing the movement and the collection of the pathological section image and selecting an algorithm for processing;
the interactive correction pathological section labeling processing is used for carrying out interactive correction algorithm processing on pathological section image labeling information by a doctor, and can carry out addition, deletion and cancellation correction operation on pathological section image labeling;
the retraining model optimization method is used for retraining all corrected pathological section images, so that the corrected pathological section image models adapt to local data, and the adaptation performance of the corrected pathological section images is improved;
the program receives the pathological section image, carries out fuzzy recognition detection, judges whether the current pathological section image moves, releases the pathological section image acquired this time and does not carry out algorithm processing if the current pathological section image does not move;
if the pathological section image is acquired after moving, selecting an algorithm to process the pathological section image and outputting the information of the pathological section image with the label;
after the pathological section image is received, displaying the marked pathological section image, and judging whether the mark needs to be corrected by a doctor;
if the selected area needs to be corrected, a doctor performs adding and deleting labeling operations on the selected area, stores the corrected labeled pathological section image, performs model training on the corrected labeled pathological section image through a retraining model optimization method, and improves the adaptability of the corrected pathological section image;
carrying out manual correction confirmation on the correction result;
displaying the marked pathological section image information, and carrying out manual judgment;
if the image marking information is judged to be complete and correct manually, outputting a pathological image report as a final report;
otherwise, correcting the image labeling information, manually adding, deleting, canceling and the like to the selected area, and correcting the selected area labeling information to enable the image to meet the actual diagnosis requirement;
outputting a pathological image report, and storing the label and the image information;
and if the model is not required to be trained again for optimization, directly outputting a final pathological section image report.
Preferably, the image blur identification method specifically includes:
and calculating image edge information by using a Laplacian operator to obtain and score the image blur degree, performing sliding filtering on the blur degree score to obtain a final score, and performing intelligent image analysis if the score is higher than a threshold value, or not performing analysis.
Further preferably, the retraining model optimization is to retrain the model through local correction data, so as to improve the adaptability of the model to the local data, and specifically includes:
firstly, a doctor starts a trained model to diagnose pathological sections transmitted by a microscope camera, and different labeling information is printed on different cells to distinguish the cells;
secondly, the local data used by the trained model is less, so that the model is possibly low in local data adaptability, and a doctor can modify, replace, add, delete and the like the labels;
then adding the modified labeling information into the original pathological section image training set so as to obtain local data and finally improve the problem of data inconsistency;
and finally, when the number of the pathological section images modified by the doctor reaches a certain number, the doctor can select a retraining model optimization mode to optimize the model, and the optimized model is more adaptive to local data, so that the accuracy of intelligent analysis is improved.
The intelligent analysis system for pathological section images under a microscope has the following beneficial effects:
(1) compared with the prior intelligent microscope equipment, the system integration operation microscope is realized, so that the frequent switching operation between a computer and the microscope by a doctor is avoided. Compared with the prior art that a doctor needs to frequently and manually adjust the movable hand wheel, the intelligent analysis system for pathological section images under the microscope provided by the invention has the advantages that the doctor can directly operate the movable hand wheel to realize the switching of observation fields through the intelligent analysis system for pathological section images under the microscope, so that the operability is greatly improved.
(2) Compared with the prior system, the system has the obvious advantages of higher real-time performance and greatly reducing the time delay required by the system to display the detection result. Compared with the prior device which needs 2s-3s to run the detection program after the user clicks the detection button, the real-time display is low. The invention adopts a fuzzy recognition algorithm, automatically detects and calculates the picture information after the objective lens stops moving, and detects by utilizing the time period that a doctor stops moving the lens and clicks the detection button, thereby improving the subjective use feeling of the doctor and improving the real-time performance of the system.
(3) Compared with the traditional intelligent auxiliary diagnosis system, the system has the remarkable advantage that a doctor can manually correct the pathological detection result during diagnosis. The doctor can manually insert, delete, cancel and the like the detection conclusion, so that the problems that the doctor can only passively receive the system detection report under the traditional intelligent auxiliary diagnosis system, the analysis result cannot be corrected, and the manual operation is repeated are solved. The invention can continuously modify the training model according to the manual modification data, thereby enabling doctors to further participate in the diagnosis process and reducing the repetition rate of manual operation.
(4) The data of the retraining training hospital is utilized, the problem that the data of the model training data set and the data used by the hospital are inconsistent is solved, the generalization capability of the model is greatly improved, and the accuracy of system judgment is further improved.
Drawings
FIG. 1 is a block diagram of the structure of an intelligent analysis system for pathological section images under a microscope according to the present invention;
FIG. 2 is a flow chart of the operation of the intelligent microscopic pathological section image analysis system of the present invention;
FIG. 3 is a flow chart of intelligent pathological section image analysis;
FIG. 4 is a schematic flow diagram of a robot arm interaction process;
FIG. 5 is a schematic illustration of a blur detection flow;
FIG. 6 is a flowchart of the medical record saving of the intelligent analysis system for pathological section images under microscope according to the present invention;
FIG. 7 is an interface design of an intelligent analysis system for microscopic pathological section images according to the present invention;
FIG. 8 is a schematic view of the intelligent analysis system for microscopic pathological section images in TCT mode interaction;
FIG. 9 is an illustration of the insertion operation of the intelligent analysis system for microscopic pathological section images in the TCT mode interaction;
FIG. 10 is a diagram illustrating the deletion operation of the intelligent analysis system for pathological section images under microscope in TCT mode interaction;
FIG. 11 is an illustration of the insertion operation of the intelligent microscopic pathological section image analysis system of the present invention in ki67 mode interaction;
FIG. 12 is an illustration of the deletion operation of the intelligent microscopic pathological section image analysis system of the present invention in ki67 mode interaction;
FIG. 13 is an illustration of the intelligent analysis system for microscopic pathological section images of the present invention under ki67 mode interaction with undo operation;
fig. 14 is an interactive operation demonstration diagram of the intelligent microscopic pathological section image analysis system retraining system of the invention.
The present invention will be described in further detail with reference to the following drawings and examples.
Detailed Description
The design idea of the invention is that firstly, the problem that the slice observation position cannot be remotely moved by the existing intelligent microscope is solved, and the slice position is changed by operating and controlling the moving hand wheel; immediately processing the collected pathological section image by using a fuzzy recognition method after the microscope stops moving, so that the analysis result can be displayed in real time.
According to the problem that the processing result cannot be corrected manually in the existing problems, the problem that the adaptability of a local model is poor is solved, a retraining model-based optimization method is provided, the processing model is continuously optimized, and the system analysis accuracy is improved.
Referring to fig. 1, the present embodiment provides an intelligent analysis system for an image of a pathological section under a microscope, including:
a microscope for observing pathological sections;
the microscope camera is provided with an acquisition switch, is connected with the lens of the microscope and is used for acquiring pathological section images under the microscope;
the mechanical arm is used for connecting a moving hand wheel of the microscope, controlling the moving platform to move longitudinally and horizontally to move the section, and realizing the switching of pathological section images under the microscope;
the processor is in communication connection with the microscope camera, receives the pathological section image from the microscope camera, preprocesses the pathological section image, and performs cell detection and cell classification on the processed pathological section image;
one or more memories connected with the processor and used for storing programs and saving data;
the display is connected with the processor and used for receiving and displaying the marked pathological section image from the processor in real time;
the Bluetooth equipment is used for information interaction between the mechanical arm and the processor; and
one or more programs, stored in the one or more memories, that are invoked for execution by the processor;
the program comprises instructions for performing the steps of:
1) a microscope camera acquires a digital pathological section image;
2) matching an adaptive algorithm model for the digital pathological section image, carrying out algorithm processing on the pathological section image through intelligent diagnosis to obtain a preliminary diagnosis result, and outputting the labeled pathological section image;
3) manually judging the marked pathological section image, and if the marking result meets the requirement, the accuracy is high, and the correction marking is not carried out; otherwise, manually correcting and labeling information of the pathological section image, and outputting a manual judgment result;
4) processing the artificially corrected pathological section image, retraining the model through the local data, and enhancing the adaptability of the model to the local data;
5) and outputting a final pathology report.
When the intelligent analysis system for pathological section images under the microscope is used, the acquisition switch is turned on, and the microscope camera acquires the pathological section images of the cell smear under the microscope and sends the pathological section images to the processor; the processor carries out cell detection and cell classification on the pathological section image from the microscope camera, marks the position of the cell, the cell type and the corresponding confidence coefficient on the pathological section image, and outputs the marked pathological section image; the display receives and displays the marked pathological section image from the processor; the Bluetooth equipment is connected with the processor and the mechanical arm, and sends the moving signal to the mechanical arm to control the mechanical arm to rotate forwards and backwards so as to realize the switching of the moving angle and the moving direction; the mechanical arm controls a microscope moving hand wheel through upper, lower, left and right direction keys in the system to realize horizontal or longitudinal movement of the moving platform, so that the pathological section position observed by the microscope is changed, and the local position of the pathological section is switched in real time to observe.
The mechanical arm comprises the following functions:
firstly, the Bluetooth equipment is connected with the processor and the mechanical arm, so that normal communication between the two parties is ensured, and the operation of moving a hand wheel of a microscope is realized;
secondly, the system detects the direction of the current movement operation of the user and transmits a signal to the mechanical arm through the Bluetooth device;
and finally, the motor of the mechanical arm operates the mechanical arm to rotate the moving hand wheel of the microscope according to the transmission signal, so that the observation area switching is realized.
In this embodiment, digital pathological section image is gathered to microscope camera adopts remote operation, need not artifical manual operation and removes pathological section, specifically includes:
judging whether the mechanical arm needs to be operated or not;
if the mechanical arm does not need to be moved, transmitting the currently acquired pathological section image to a processor, and processing and labeling the pathological section image; otherwise, moving the mechanical arm and judging the moving direction of the mechanical arm;
transmitting a moving direction signal of the mechanical arm through Bluetooth equipment, and controlling the mechanical arm to rotate by a motor in the mechanical arm to drive a microscope moving hand wheel to rotate;
and acquiring a new pathological section image, transmitting the new pathological section image to a processor, and carrying out algorithm processing on the pathological section image.
The one or more programs comprise image fuzzy recognition, interactive correction pathological section labeling processing and retraining model optimization methods, so that a processing model is adaptive to local data; wherein:
the image fuzzy recognition is used for intelligently recognizing the micro deformation of the pathological section image under the microscope, intelligently recognizing the movement and the collection of the pathological section image and selecting an algorithm for processing;
the interactive correction pathological section labeling processing is used for carrying out interactive correction algorithm processing on pathological section image labeling information by a doctor, and can carry out addition, deletion and cancellation correction operation on pathological section image labeling;
the retraining model optimization method is used for retraining all corrected pathological section images, so that the corrected pathological section image models adapt to local data, and the adaptation performance of the corrected pathological section images is improved;
the program receives the pathological section image, carries out fuzzy recognition detection, judges whether the current pathological section image moves, releases the pathological section image acquired this time and does not carry out algorithm processing if the current pathological section image does not move;
if the pathological section image is acquired after moving, selecting an algorithm to process the pathological section image and outputting the information of the pathological section image with the label;
after the pathological section image is received, displaying the marked pathological section image, and judging whether the mark needs to be corrected by a doctor;
if the selected area needs to be corrected, a doctor performs adding and deleting labeling operations on the selected area, stores the corrected labeled pathological section image, performs model training on the corrected labeled pathological section image through a retraining model optimization method, and improves the adaptability of the corrected pathological section image;
carrying out manual correction confirmation on the correction result;
displaying the marked pathological section image information, and carrying out manual judgment;
if the image marking information is judged to be complete and correct manually, outputting a pathological image report as a final report;
otherwise, correcting the image labeling information, manually adding, deleting, canceling and the like to the selected area, and correcting the selected area labeling information to enable the image to meet the actual diagnosis requirement;
outputting a pathological image report, and storing the label and the image information;
and if the model is not required to be trained again for optimization, directly outputting a final pathological section image report.
In this embodiment, the retraining model optimization is to retrain the model through local correction data, so as to improve the adaptability of the model to the local data, and specifically includes:
firstly, a doctor starts a trained model to diagnose pathological sections transmitted by a microscope camera, and different labeling information is printed on different cells to distinguish the cells;
secondly, the local data used by the trained model is less, so that the model is possibly low in local data adaptability, and a doctor can modify, replace, add, delete and the like the labels;
then adding the modified labeling information into the original pathological section image training set so as to obtain local data and finally improve the problem of data inconsistency;
finally, when the number of the pathological section images modified by the doctor reaches a certain number, the doctor can select to click the upper left corner retraining button to optimize the model, and the optimized model is more adaptive to local data, so that the accuracy of intelligent analysis is improved.
Referring to fig. 2, the operation process of the intelligent analysis system for microscopic pathological section images of the embodiment is as follows:
for the local mode of operation:
uploading a pathological section, selecting a pathological section image processing mode, carrying out algorithm processing on the pathological section, then clicking a doctor, namely checking a display processing result, simultaneously setting programs of clicking insertion, clicking deletion, clicking revocation, clicking preservation and clicking retraining, and manually adding processing information in the process of clicking insertion; in clicking 'delete', deleting the processing result which is not needed; in clicking 'undo', the previous operation is resumed; in clicking 'save', saving the current operation result, the patient information and the like; in click retraining, a model is built that fits the local data through local dataset retraining.
For the real-time mode of operation:
the method comprises the steps of receiving pathological section data from a microscope camera, obtaining current pathological section data when an acquisition switch is turned on, carrying out fuzzy detection on whether the camera moves or not, displaying pathological section images if the camera moves, selecting a pathological section image processing mode, carrying out algorithm processing on pathological sections, and carrying out subsequent operation in the same local operation mode.
The intelligent analysis system for pathological section images under the microscope can process two different pathological section images, one pathological section image is used for automatically judging ki67 index, the malignant degree of a tumor can be judged, the other pathological section image is used for detecting positive cells of a TCT section of cervical cancer, and the early cervical cancer can be screened. The intelligent analysis system marks the position of the cell, the cell type and the corresponding confidence coefficient on the pathological image and outputs the marked pathological image; and the display is connected with the processor and is used for receiving and displaying the marked pathological section image. The acquisition switch is installed on the microscope camera to control the operation of the microscope camera.
As shown in FIG. 3, the pathological section image intelligent analysis firstly judges whether the current operation mode is a real-time operation mode or a local operation mode, if the current operation mode is the local operation mode, pathological section data are directly obtained, then a processing mode is selected to be ki67 or TCT, the pathological section data image is processed by an algorithm, the algorithm result is manually corrected, medical record information is written and saved by adding, deleting, canceling, saving and other operations on the pathological section data image, and the saved local data is retrained, so that the model adapts to the local data.
If the current pathological section is in the real-time operation mode, fuzzy recognition is carried out on the pathological section area, micro deformation of the pathological section image under the microscope is reduced, whether the current pathological section moves or not is judged, then the moving hand wheel of the microscope is controlled by the mechanical arm to control the movement of the pathological section, the image information of the pathological section area needing to be processed is obtained, and the subsequent operation is the same as the local operation mode.
The mechanical arm interaction process is as shown in fig. 4, the mechanical arm is connected with the processor through the bluetooth device, the user writes a moving button, the bluetooth device transmits signals to the mechanical arm, and the mechanical arm rotates and moves a microscope hand wheel according to the signals.
The fuzzy recognition detection process is shown in fig. 5, and includes the following steps:
(1) performing edge detection on the current video stream image data by using a Laplacian operator to obtain the fuzzy degree of the video stream image data, and scoring to obtain curr _ var;
(2) performing sliding filtering processing on the fuzzy degree score curr _ var to obtain a comprehensive fuzzy degree score Image _ var ═ 0.5 × last _ var +0.5 × curr _ var;
(3) and intelligently detecting the video stream image data with the comprehensive fuzzy degree score higher than the threshold value, and not detecting the video stream image data with the comprehensive fuzzy degree score lower than the threshold value.
Fig. 6 shows a flow chart of saving medical records, which includes the following steps:
(1) establishing basic information including the name, the sex, the age, the clinic number, the hospitalization number, the bed number, the pathological number, the examination hospital, the examination department, the examination physician, the examination date, the material-taking part, the clinical diagnosis and the visual observation of the patient, wherein the information of the clinical diagnosis can be written in by a background and the physician can manually modify, and other information can be manually written in by the physician;
(2) after clicking a storage button, the system stores the medical record of the patient, the original slice image, the processed slice image and the processed image labeling information into a folder;
(3) submitting medical records and detection results;
referring to fig. 7, an interface design of an intelligent analysis system for pathological section images under a microscope includes the following contents:
(1) the tool bar includes menu buttons File, Edit, View, Window, Help and Retrain, and tool buttons "open File", "open folder", "save", "print" and "Help". The toolbar is used for carrying out system basic configuration operation;
(2) a running mode selection column including a mode selection 1 selection algorithm processing mode of "TCT processing" or "ki 67 processing", and a mode selection 2 selection "local running" or "real-time running" mode;
(3) and the interactive toolbar comprises interactive buttons of 'operation', 'drawing' and 'saving'. The operational functions include diagnostic, insert, delete and undo functions. The drawing function includes drawing of annotation information of the type including a point, a rectangular frame, a free pen, and the like. The storage function stores the pathological section images corrected by the interactive operation to a local designated path;
(4) patient medical record information including name, sex, age, clinic number, hospital number, bed number, pathology number, censorship hospital, censorship department, censorship physician, censorship date, material-drawing part, clinical diagnosis, visible to the naked eye, wherein the information of clinical diagnosis can be written in by the background and the doctor can also manually modify, other information can be written in by the doctor manually. After the save button is clicked, the system can put the patient medical record, the original slice image, the processed slice image and the processed image labeling information into a folder for saving.
The intelligent analysis system for the pathological section images under the microscope has the following functions:
(1) the method comprises the steps of starting a microscope, selecting a pathological section image processing mode, designing a fuzzy recognition algorithm aiming at pathological section image analysis, processing the pathological section image immediately after the pathological section image is detected to stop moving, and displaying an analysis result in real time.
(2) And after the 'diagnosis' button is clicked, the processing result is completely displayed, or a user-defined area is selected through the 'drawing' button to display the processing result.
(3) In the TCT mode, clicking an 'insert' button, then clicking a 'drawing' button, and after selecting a disease category identifier, drawing a mark frame and the disease category identifier according to the sliding range of a mouse on a pathological section image; after clicking the 'delete' function, deleting a mark frame containing the coordinate and a disease type mark thereof on the pathological image according to the mouse click coordinate; after the cancel function is clicked, the deleted mark frame and the disease identification in the rectangular selected area are recovered according to the sliding range of the mouse; after clicking the "save" button, three types of data are saved:
processing a result by the current system;
cutting the original pathological picture according to the mark frame to obtain a local picture;
and the TXT file comprises all mark frames in the local picture and identification information of the disease species.
(4) In ki67 mode, click on the "insert" function, and then click on the "draw" button selects the insert as "rectangle" or "free pen". Selecting an insertion area according to the corresponding insertion mode, and inserting the mark point into the area; clicking the 'delete' function, and then clicking the 'drawing' button to select the delete mode to be 'point', 'rectangular box' or 'free pen'. Deleting all the mark points in the area according to the deletion area selected by the corresponding deletion mode; clicking the 'cancel' function, and then clicking the 'drawing' button to select the cancel mode to be 'rectangular frame' or 'free pen'. Restoring and displaying all the deleted mark points in the area according to the revocation area selected by the corresponding revocation mode; and clicking a 'save' function to save the current processing result to the specified address.
1) After pathological section image processing and manual correction, a medical record containing patient basic information, pathological section detection information and detection results is established and a detection report is submitted. The patient medical record comprises name, gender, age, clinic number, hospital number, bed number, pathology number, examination hospital, examination department, examination physician, examination date, material-drawing part, clinical diagnosis and visual observation, wherein the information of the clinical diagnosis can be written in by a background and the doctor can manually modify, and other information can be manually written in by the doctor. After the save button is clicked, the system can put the patient medical record, the original slice image, the processed slice image and the processed image labeling information into a folder for saving.
2) The following steps are followed when performing the retraining operation: firstly, a doctor starts a trained model to diagnose pathological section images transmitted by a microscope camera, and different labeling information is printed on different cells to distinguish the cells. Secondly, as local data used by the trained model is less, the model may not have a good effect on the local data, so that the accuracy of the model annotation needs to be judged, and a doctor can modify, replace, add, delete and the like the annotation. And then the modified marking information is added into the original training set, so that local data can be obtained, and the problem of data inconsistency is improved. Finally, when the number of the images modified by the doctor reaches a certain number, the doctor can select to click the upper left corner retraining button to optimize the model, the optimized model is more suitable for local data, and better effects can be obtained in the aspects of accuracy and the like.
The operation interaction mode comprises the following contents:
(1) the insertion mode of the present embodiment differs according to the processing mode.
1) In the TCT mode, firstly, a disease category identifier is selected, a color corresponding to the identifier is obtained, and then a rectangular area formed according to the coordinates (x0, y0) of the starting point clicked by a mouse and the coordinates (x1, y1) of the released ending point is used as an identifier frame. And a region surrounded by the starting point of the mark frame and the lower right (x1+ abs (x1-x0)/10, y0-abs (y1-y0)/10) of the mark frame is used as a disease seed identification display region.
2) The first choice in the ki67 mode is to draw in a "rectangle" or "free pen". In the "rectangular frame" mode, whether or not there is a point coordinate in a rectangular area formed by the start point coordinate (x0, y0) and the released end point coordinate (x1, y1) is judged, and the point is inserted into the enclosed rectangular area. In the free pen mode, a point coordinate set of a closed graph is formed by a mouse sliding track, whether a point coordinate exists in the closed graph or not is judged, and the existing point coordinate is inserted into a pathological image.
(2) The deletion mode of the present embodiment is different depending on the processing mode.
1) And after a 'delete' button is clicked in the TCT mode, clicking a mark frame needing to be deleted in the pathological graph, judging the mark frame containing the clicked coordinate and deleting the mark frame.
2) After clicking a 'delete' button in a Ki67 mode, selecting a 'point' delete mode, clicking a point to be deleted, and deleting the point information according to a click coordinate; selecting a rectangular frame deleting mode, forming a rectangular area by the coordinates (x0, y0) of the starting point clicked by the mouse and the coordinates (x1, y1) of the released ending point, and deleting the point coordinate information contained in the area; and selecting a free pen deleting mode, forming a point coordinate set of a closed graph by a mouse sliding track, judging points contained in a closed area surrounded by the point set, and deleting point coordinate information.
(3) The revocation mode of the present embodiment differs according to the processing mode:
1) after clicking the 'cancel' button in the TCT mode, the rectangular area formed by the coordinates of the start point (x0, y0) clicked by the mouse and the coordinates of the released end point (x1, y1) is restored to display the deleted point coordinate information of the area.
2) ki67, after clicking cancel button, selecting cancel mode of rectangle frame, restoring deleted point coordinate information of the area by rectangle area formed by starting point coordinate (x0, y0) clicked by mouse and released end point coordinate (x1, y 1); and selecting a free pen canceling mode, forming a point coordinate set of a closed graph by a mouse sliding track, and restoring deleted point coordinate information contained in a closed area surrounded by the point set.
(4) The storage method of this embodiment is different depending on the processing mode.
1) After clicking the "save" button in TCT mode:
firstly, for detection result storage, firstly acquiring pixmap displayed by a current window, storing the pixmap as temPix, drawing all mark frames and disease type identifications in the display on the temPix, secondly converting the temPix into an array form, setting the picture size to be 1920 x 1080, and then storing the detection result to a specified path \ Model \ yolov5\ TCT \ TCTImaps;
secondly, intercepting and storing the area with the mark frame, firstly acquiring the picture array before processing, secondly intercepting the area with the size of 512 by 512 according to the position information of the mark frame, and reducing the area to the size of 256 by 256. Storing the original coordinates of the intercepted area, judging whether the mark frame is completely contained in the intercepted area before intercepting, if the screenshot of the area exists, not intercepting, and finally storing the intercepted picture to \ Model \ yolov5\ TCT \ images \ train;
thirdly, for the storage of the intercepted area mark frame information and the disease identification information, after the area is intercepted, the information is written into the TXT file with the same name as the corresponding intercepted area according to the sequence of class (disease information), x-center/width, y-center/height, w-max-x-min/width, h-max-y-min/height, and finally the file is stored into \ Model \ yolov5\ TCT labels \ train.
2) After clicking a 'save' button in a Ki67 mode, firstly, selecting a save path of a processing result by a user, secondly, obtaining pixmap of a current window and converting the pixmap into an array format, then setting the size of a picture to be 1920 x 1080, and finally, writing displayed mark point information into the path specified by the user after the mark point information is written by OpenCV.
The drawing interaction mode comprises the following steps:
(1) after the point interaction mode is selected, whether a mark point exists in the range of 2-by-2 pixels with the coordinate as the center is judged by acquiring the coordinate of the mouse click position.
(2) After the 'rectangular box' interaction mode is selected, the coordinates (x0, y0) of the starting point of mouse click and the coordinates (x1, y1) of the released ending point are obtained, the coordinates (x0, y0) of the starting point are used as the top left vertex of the rectangular area, and the coordinates (x1-x0) and y1-y0 of the rectangular area are used for drawing the rectangular area according to the width and the height of the rectangular area respectively. The coordinates (x, y) are sequentially determined, and point coordinate information satisfying the conditions of x0< ═ x1 and y0< ═ y1 is output.
(3) After the free pen interaction mode is selected, the point coordinate to be selected and judged and the track information in the list are judged by acquiring the mouse sliding track and storing the track as the list. Firstly, whether the point is on the track or not is judged, and secondly, whether the point (px, py) is positioned inside the track enclosing graph or not is judged. If the ordinate is between any two points (sx, sy) and (tx, ty) on the locus, the relationship between the abscissa px and the x ═ sx + (py-sy) × (tx-sx)/(ty-sy) coordinate is determined. If x is px, the point (px, py) is within the irregular area, and if x > px, the determination of the relationship between the point (px, py) and any other two points on the trajectory is continued. Finally, point information on the track or inside the enclosed area is output.
In addition, the intelligent analysis system for pathological section images under a microscope provided by the embodiment further performs the following design on the following interactive operation interfaces:
FIG. 8 shows a representation of processing operations under a TCT interaction, illustrating the display format after inserting flag information under the TCT; FIG. 9 shows an illustration of the insertion operation in TCT mode interaction, allowing the physician to interactively insert the marker information; FIG. 10 shows a diagram illustrating deletion operation under TCT mode interaction, so that a doctor can delete and correct misjudged annotation information; FIG. 11 shows a schematic diagram of the insertion operation under ki67 mode interaction, and classification labeling is carried out on cells in the detection region; FIG. 12 presents a depiction of a delete operation under ki67 mode interaction, resulting in the deletion of selected region labeling information; FIG. 13 shows an illustration of undo operations performed under ki67 mode interaction to undo a modification operation on a selected region to facilitate a revision of a callout; FIG. 14 shows an operational illustration of the retraining system interaction to improve model accuracy by retraining the optimization model.
In summary, the intelligent analysis system for pathological section images under a microscope provided by this embodiment dynamically acquires pathological section image data by using a bit data compression technology, and identifies a moving pathological section image by using a fuzzy identification method, reduces errors caused by micro-deformation of the pathological section image, and performs intelligent analysis under a static pathological section image, thereby improving the real-time performance of the system. After the intelligent analysis result is obtained, the doctor can manually correct the detection result, participate in the generation of the pathological section image report, accord with the actual working process and improve the accuracy of the final result. Due to the fact that the data are inconsistent and the like caused by different microscopes or coloring agents used by different hospitals or scientific research institutions, the adaptability of the model and the local data is low, the system trains the model suitable for the local data by using the collected local data corrected by doctors and a method of label use retraining, the accuracy of the model is finally improved, and algorithm optimization is achieved.
Claims (5)
1. The utility model provides a pathological section image intelligent analysis system under microscope which characterized in that includes:
a microscope for observing pathological sections;
the microscope camera is provided with an acquisition switch, is connected with the lens of the microscope and is used for acquiring pathological section images under the microscope;
the mechanical arm is used for connecting a moving hand wheel of the microscope, controlling the moving platform to move longitudinally and horizontally to move the section, and realizing the switching of pathological section images under the microscope;
the processor is in communication connection with the microscope camera, receives the pathological section image from the microscope camera, preprocesses the pathological section image, and performs cell detection and cell classification on the processed pathological section image;
one or more memories connected with the processor and used for storing programs and saving data;
the display is connected with the processor and used for receiving and displaying the marked pathological section image from the processor in real time;
the Bluetooth equipment is used for information interaction between the mechanical arm and the processor; and
one or more programs, stored in the one or more memories, that are invoked for execution by the processor;
the program comprises instructions for performing the steps of:
1) a microscope camera acquires a digital pathological section image;
2) matching an adaptive algorithm model for the digital pathological section image, carrying out algorithm processing on the pathological section image through intelligent diagnosis to obtain a preliminary diagnosis result, and outputting the labeled pathological section image;
3) manually judging the marked pathological section image, and if the marking result meets the requirement, the accuracy is high, and the correction marking is not carried out; otherwise, manually correcting and labeling information of the pathological section image, and outputting a manual judgment result;
4) processing the artificially corrected pathological section image, retraining the model through the local data, and enhancing the adaptability of the model to the local data;
5) and outputting a final pathology report.
2. The intelligent analysis system for the pathological section images under a microscope according to claim 1, wherein the digital pathological section images collected by the microscope camera are remotely operated without manual operation to move the pathological section, and the system specifically comprises:
judging whether the mechanical arm needs to be operated or not;
if the mechanical arm does not need to be moved, transmitting the currently acquired pathological section image to a processor, and processing and labeling the pathological section image; otherwise, moving the mechanical arm and judging the moving direction of the mechanical arm;
transmitting a moving direction signal of the mechanical arm through Bluetooth equipment, and controlling the mechanical arm to rotate by a motor in the mechanical arm to drive a microscope moving hand wheel to rotate;
and acquiring a new pathological section image, transmitting the new pathological section image to a processor, and carrying out algorithm processing on the pathological section image.
3. The intelligent analysis system for pathological section images under microscope as claimed in claim 1, wherein the one or more programs include image blur recognition, interactive correction pathological section labeling process and retraining model optimization method, so that the processing model is adaptive to local data; wherein:
the image fuzzy recognition is used for intelligently recognizing the micro deformation of the pathological section image under the microscope, intelligently recognizing the movement and the collection of the pathological section image and selecting an algorithm for processing;
the interactive correction pathological section labeling processing is used for carrying out interactive correction algorithm processing on pathological section image labeling information by a doctor, and can carry out addition, deletion and cancellation correction operation on pathological section image labeling;
the retraining model optimization method is used for retraining all corrected pathological section images, so that the corrected pathological section image models adapt to local data, and the adaptation performance of the corrected pathological section images is improved;
the program receives the pathological section image, carries out fuzzy recognition detection, judges whether the current pathological section image moves, releases the pathological section image acquired this time and does not carry out algorithm processing if the current pathological section image does not move;
if the pathological section image is acquired after moving, selecting an algorithm to process the pathological section image and outputting the information of the pathological section image with the label;
after the pathological section image is received, displaying the marked pathological section image, and judging whether the mark needs to be corrected by a doctor;
if the selected area needs to be corrected, a doctor performs adding and deleting labeling operations on the selected area, stores the corrected labeled pathological section image, performs model training on the corrected labeled pathological section image through a retraining model optimization method, and improves the adaptability of the corrected pathological section image;
carrying out manual correction confirmation on the correction result;
displaying the marked pathological section image information, and carrying out manual judgment;
if the image marking information is judged to be complete and correct manually, outputting a pathological image report as a final report;
otherwise, correcting the image labeling information, manually adding, deleting and canceling the selected area, and correcting the selected area labeling information to enable the image to meet the actual diagnosis requirement;
outputting a pathological image report, and storing the label and the image information;
and if the model is not required to be trained again for optimization, directly outputting a final pathological section image report.
4. The intelligent analysis system for pathological section images under microscope according to claim 3, wherein the image fuzzy recognition method specifically comprises:
and calculating image edge information by using a Laplacian operator to obtain and score the image blur degree, performing sliding filtering on the blur degree score to obtain a final score, and performing intelligent image analysis if the score is higher than a threshold value, or not performing analysis.
5. The intelligent analysis system for pathological section images under microscope as claimed in claim 3, wherein the retraining model optimization is to retrain the model by local correction data, so as to improve the adaptability of the model to the local data, and specifically comprises:
firstly, a doctor starts a trained model to diagnose pathological sections transmitted by a microscope camera, and different labeling information is printed on different cells to distinguish the cells;
secondly, the local data used by the trained model is less, so that the model is possibly low in local data adaptability, and a doctor can modify, replace, add, delete and the like the labels;
then adding the modified labeling information into the original pathological section image training set so as to obtain local data and finally improve the problem of data inconsistency;
and finally, when the number of the pathological section images modified by the doctor reaches a certain number, the model can be optimized by selecting a retraining model optimization mode, and the optimized model is more adaptive to local data, so that the accuracy of intelligent analysis is improved.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111332278.5A CN114202510B (en) | 2021-11-11 | 2021-11-11 | Intelligent analysis system for pathological section image under microscope |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111332278.5A CN114202510B (en) | 2021-11-11 | 2021-11-11 | Intelligent analysis system for pathological section image under microscope |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114202510A true CN114202510A (en) | 2022-03-18 |
| CN114202510B CN114202510B (en) | 2024-01-19 |
Family
ID=80647355
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111332278.5A Active CN114202510B (en) | 2021-11-11 | 2021-11-11 | Intelligent analysis system for pathological section image under microscope |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114202510B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116364229A (en) * | 2023-04-20 | 2023-06-30 | 北京透彻未来科技有限公司 | Intelligent visual pathological report system for cervical cancer anterior lesion coning specimen |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110763678A (en) * | 2019-09-12 | 2020-02-07 | 杭州迪英加科技有限公司 | Pathological section interpretation method and system |
| CN110763677A (en) * | 2019-09-12 | 2020-02-07 | 杭州迪英加科技有限公司 | Thyroid gland frozen section diagnosis method and system |
| WO2020038245A1 (en) * | 2018-08-21 | 2020-02-27 | 腾讯科技（深圳）有限公司 | Pathological microscope, display module, control method and device, and storage medium |
| CN112132772A (en) * | 2019-06-24 | 2020-12-25 | 杭州迪英加科技有限公司 | Pathological section real-time interpretation method, device and system |
| CN113130049A (en) * | 2021-04-15 | 2021-07-16 | 黑龙江机智通智能科技有限公司 | Intelligent pathological image diagnosis system based on cloud service |
| WO2021196632A1 (en) * | 2020-03-30 | 2021-10-07 | 中国科学院深圳先进技术研究院 | Intelligent analysis system and method for panoramic digital pathological image |
-
2021
- 2021-11-11 CN CN202111332278.5A patent/CN114202510B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020038245A1 (en) * | 2018-08-21 | 2020-02-27 | 腾讯科技（深圳）有限公司 | Pathological microscope, display module, control method and device, and storage medium |
| CN112132772A (en) * | 2019-06-24 | 2020-12-25 | 杭州迪英加科技有限公司 | Pathological section real-time interpretation method, device and system |
| CN110763678A (en) * | 2019-09-12 | 2020-02-07 | 杭州迪英加科技有限公司 | Pathological section interpretation method and system |
| CN110763677A (en) * | 2019-09-12 | 2020-02-07 | 杭州迪英加科技有限公司 | Thyroid gland frozen section diagnosis method and system |
| WO2021196632A1 (en) * | 2020-03-30 | 2021-10-07 | 中国科学院深圳先进技术研究院 | Intelligent analysis system and method for panoramic digital pathological image |
| CN113130049A (en) * | 2021-04-15 | 2021-07-16 | 黑龙江机智通智能科技有限公司 | Intelligent pathological image diagnosis system based on cloud service |
Non-Patent Citations (1)
| Title |
|---|
| 王莹;段佳佳;祝闯;刘军;董慧慧;齐勇刚;金木兰;: "临床结直肠病理切片图像的自动辅助诊断", 临床与实验病理学杂志, no. 10 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116364229A (en) * | 2023-04-20 | 2023-06-30 | 北京透彻未来科技有限公司 | Intelligent visual pathological report system for cervical cancer anterior lesion coning specimen |
| CN116364229B (en) * | 2023-04-20 | 2023-11-10 | 北京透彻未来科技有限公司 | Intelligent visual pathological report system for cervical cancer anterior lesion coning specimen |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114202510B (en) | 2024-01-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11908188B2 (en) | Image analysis method, microscope video stream processing method, and related apparatus | |
| CN110473186B (en) | Detection method based on medical image, model training method and device | |
| US7945083B2 (en) | Method for supporting diagnostic workflow from a medical imaging apparatus | |
| CN112380900A (en) | Deep learning-based cervical fluid-based cell digital image classification method and system | |
| CN111430014B (en) | Glandular medical image display method, glandular medical image interaction method and storage medium | |
| CN111091127A (en) | Image detection method, network model training method and related device | |
| CN115345819A (en) | Gastric cancer image recognition system, device and application thereof | |
| CN108921854B (en) | Method and system for labeling irregular lesion region of gastrointestinal endoscope image | |
| CN111583385B (en) | Personalized deformation method and system for deformable digital human anatomy model | |
| CN112365438B (en) | Pelvis parameter automatic measurement method based on target detection neural network | |
| CN114202510B (en) | Intelligent analysis system for pathological section image under microscope | |
| US20240005494A1 (en) | Methods and systems for image quality assessment | |
| US5021770A (en) | Image display system and data input apparatus used therein | |
| CN113485555B (en) | Medical image film reading method, electronic equipment and storage medium | |
| CN112489053B (en) | Tongue image segmentation method and device and storage medium | |
| CN116779093B (en) | Method and device for generating medical image structured report and computer equipment | |
| CN110021021A (en) | Head image analytical equipment and image analysis method | |
| CN111144506B (en) | Liver bag worm identification method based on ultrasonic image, storage medium and ultrasonic equipment | |
| CN110648333B (en) | Real-time segmentation system of mammary gland ultrasonic video image based on middle-intelligence theory | |
| CN115804579B (en) | Process control system for normalized blood pressure measurement and implementation method | |
| CN116883281A (en) | Image imaging image quality enhancement system and method for image guided radiotherapy | |
| CN114783591A (en) | Three-dimensional visual auxiliary diagnosis system for brain tumor | |
| CN114757894A (en) | Bone tumor focus analysis system | |
| CN114419061A (en) | Method and system for segmenting pulmonary artery and vein blood vessels | |
| CN112163974A (en) | Operation acquisition, learning and sharing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |