CN113793307A

CN113793307A - Automatic labeling method and system suitable for multi-type pathological images

Info

Publication number: CN113793307A
Application number: CN202110965979.6A
Authority: CN
Inventors: 郑魁; 丁维龙; 朱筱婕; 赵樱莉; 李涛; 余鋆
Original assignee: Shanghai Paiying Medical Technology Co ltd
Current assignee: Shanghai Paiying Medical Technology Co ltd
Priority date: 2021-08-23
Filing date: 2021-08-23
Publication date: 2021-12-14

Abstract

The invention provides an automatic labeling method and system suitable for multi-type pathological images, which comprises the steps of carrying out classification recognition on uploaded pathological images through a trained deep learning model to obtain classification results, position coordinates of each classification and corresponding probability of each classification of the pathological images, carrying out automatic labeling on parts identified as tumor regions to generate labeling results, displaying the labeling results of the tumor regions in the pathological images, carrying out operations including labeling addition, label modification and label re-labeling after label deletion if a pathological expert thinks that the labeling results are wrong, and giving examination results and diagnosis suggestions by the pathological expert according to the labeling results. According to the pathological image automatic labeling algorithm based on the cluster analysis, the XML vector diagram label of the pathological image is given, the doctor auditing function is designed, the accuracy of automatic labeling is ensured, and the workload of a pathological expert is relieved by combining with the actual flow of a hospital.

Description

Automatic labeling method and system suitable for multi-type pathological images

Technical Field

The invention relates to the technical field of medical image diagnosis, in particular to an automatic labeling method and system suitable for multiple types of pathological images.

Background

In recent years, with the continuous development of artificial intelligence and deep learning technology, computer-aided diagnosis and treatment have become the focus of medical image research. The tissue pathology image auxiliary diagnosis method based on deep learning can accelerate the diagnosis efficiency of a pathology doctor and relieve the conditions that the pathology doctor is scarce, the culture period is long and the pathology department generally overload. The pathological image diagnosis based on deep learning needs to use a large amount of pathological data sets, but the full-slice image has huge size and the labeling work is very time-consuming, so that the current pathological image field has fewer data sets labeled at the pixel level. The existing labeling system is mostly the manual labeling of pathological experts, wastes time and labor, and is easy to have the conditions of label error and label leakage. The histopathology auxiliary diagnosis method based on deep learning often has the problems that the recognition accuracy rate is low, XML vector diagram labels are not given, the method cannot be used as a deep learning data set, one system only corresponds to one disease, and the like. Therefore, the invention designs an automatic labeling system suitable for multi-type pathological images, realizes the automatic labeling of tumor areas of the multi-type pathological images, and comprises a pathological expert auditing module, so that the workload of pathological experts is reduced, and the accuracy of labeling is ensured.

In recent years, several scholars have studied the identification of pathological images, and for example, Sun Y et al have proposed a deep learning-based system that can automatically identify tumor regions in histopathological images. Diao S et al propose a fully automated cancer region identification framework for computer-aided diagnosis of pathological WSI based on deep convolutional neural networks. Wang X et al propose a weakly supervised learning method to solve the classification problem of full-section lung cancer images, and at the same time, effectively predict the probability of cancer and detect suspected cancer regions. However, none of the above methods provides vector diagram annotation of XML, and cannot provide a data set for deep learning.

Currently, there are also some related technologies to solve the above problems. For example, chinese patent CN105608319B provides an annotation method for digital pathological images, but this method requires the user to manually select the position of the annotation point and the type of the annotation figure. The chinese patent CN110659692A realizes automatic labeling of pathological images, but it does not include a function of review by a doctor, and cannot guarantee accuracy of automatic labeling. Chinese patent CN110826560A discloses a method for labeling pathological images of esophageal cancer, but the method of the present invention only corresponds to a method for labeling diseases.

Disclosure of Invention

The invention provides an automatic labeling method and system suitable for multiple types of pathological images, and aims to overcome the defects in the prior art.

In one aspect, the present invention provides an automatic labeling method suitable for multiple types of pathological images, which includes the following steps:

s1: a pathology expert uploads a pathology image, edits related information of the uploaded pathology image, and performs checking, adding, modifying and deleting operations on the uploaded pathology image information, a disease type corresponding to the uploaded pathology image and patient information corresponding to the uploaded pathology image;

s2: according to different disease types displayed in the uploaded pathological images, the uploaded pathological images are sent to a trained deep learning model corresponding to the disease types for recognition, classification results, position coordinates of each classification and probabilities corresponding to each classification of the uploaded pathological images are obtained, parts recognized as tumor regions are automatically labeled to generate labeling results, the labeling results of the tumor regions are displayed in the pathological images, and the labeling results are returned to a pathology expert;

s3: and the pathological expert checks the labeling result, if the labeling result is wrong, the operations including adding labels, modifying labels and deleting labels and then re-labeling are carried out on the labeling result, and the pathological expert gives an audit result and a diagnosis suggestion according to the labeling result.

The method carries out two-class automatic labeling on normal and pathological multi-system type pathological images, sets the number of each class of data sets to be basically consistent, provides a pathological image automatic labeling algorithm based on cluster analysis, gives XML vector diagram labeling, designs a doctor auditing function, ensures the accuracy of automatic labeling, combines with the actual flow of a hospital, and relieves the workload of a pathological specialist.

In a specific embodiment, the sending the uploaded pathological images into a trained deep learning model corresponding to the disease type for recognition according to the difference of the disease types displayed in the uploaded pathological images to obtain classification results of the uploaded pathological images, position coordinates of each classification and probabilities corresponding to each classification, and automatically labeling the parts recognized as the tumor regions to generate labeling results specifically includes:

classifying and identifying blocks of the pathological image by using a trained deep learning model through a density-based clustering algorithm to obtain classification results and position coordinates of each classification, clustering according to the classification results and the position coordinates of each classification, and calculating a probability mean value, a standard deviation and a variance of each classification;

when the probability mean, standard deviation and variance of a class are larger than a certain threshold, the class is retained, otherwise, the class is deleted.

In a specific embodiment, the displaying the labeling result of the tumor region in the pathological image includes:

and obtaining each classified edge point through a convex hull algorithm, and displaying the labeling result of the tumor region in the pathological image in a polygonal frame mode according to the edge points.

In a specific embodiment, the method further includes S4:

the information of the pathological experts is managed, and the information of the pathological experts is inquired, added, modified and deleted, wherein the information of the pathological experts comprises names, account numbers, passwords, mobile phone numbers, hospitals and departments.

In a specific embodiment, the method further includes S5:

and checking the pathological image information and the corresponding patient information, and downloading the labeling result of the examined pathological image.

In a specific embodiment, the displaying the labeling result of the tumor region in the pathological image further includes:

the support of DZI format image display technology is realized by adopting OpenSeadragon image control, the basic display function of pathological images is realized, free zooming and moving are carried out according to the needs of users, and simultaneously the labeling result is synchronously zoomed and moved along with the needs of the users. The displayed pathological image can be freely zoomed and moved according to the needs of a user, the zoom multiple can reach sixty times, and the annotation expert can conveniently and carefully check the image information in the pathological image.

In a specific embodiment, before the uploaded pathological image is sent to a trained deep learning model corresponding to a disease type for recognition, an image mask method is used to segment a plurality of small cell tissue area maps from the uploaded pathological image, and then the segmented small cell tissue area maps are input to the trained deep learning model for classification and recognition.

In a specific embodiment, the labeling result includes a vector diagram label of XML, specifically including position information of the labeled region and classification information of the lesion type.

In a specific embodiment, the performing operations including adding a label, modifying a label, and re-labeling after deleting a label on the label in the labeling result specifically includes:

newly creating a label, deleting the label, modifying the shape of the label, modifying the type of the label and storing the label on the labeling result;

the newly-established labeling comprises adding a region labeling outline on the tumor region which is missed in the step S2;

the deleting the annotation comprises deleting the incorrect annotation in S2 on an annotation list and simultaneously deleting the incorrect annotation in S2 on an annotation display area;

the label modification comprises modifying the labeled region outline in the display region of the original label and modifying the labeled classification information in the label list;

and the step of storing the label comprises submitting all the current label information to be stored in a database.

In a specific embodiment, the labeling result for displaying the tumor region in the pathological image is drawn based on a drawing format including pencil drawing, polygon drawing, rectangle drawing and circle drawing, and is drawn according to the moving position of the mouse based on paper.

According to a second aspect of the present invention, a computer-readable storage medium is proposed, on which a computer program is stored, which computer program, when being executed by a computer processor, carries out the above-mentioned method.

According to a third aspect of the present invention, an automatic labeling system suitable for multi-type pathological images is provided, the system comprising:

pathological image management module: the method comprises the steps that a pathology expert is configured to upload pathology images, edit relevant information of the uploaded pathology images, and check, add, modify and delete the uploaded pathology image information, disease types corresponding to the uploaded pathology images and patient information corresponding to the uploaded pathology images;

pathological image automatic labeling module: the system is configured and used for sending the uploaded pathological images into a trained deep learning model corresponding to disease types for recognition according to different disease types displayed in the uploaded pathological images, obtaining classification results of the uploaded pathological images, position coordinates of each classification and probabilities corresponding to each classification, automatically labeling parts recognized as tumor regions to generate labeling results, displaying the labeling results of the tumor regions in the pathological images, and returning the labeling results to pathology experts;

pathological image labeling and auditing module: and the configuration is used for the pathological expert to check the labeling result, if the labeling result is considered to be wrong, the operations including adding labels, modifying labels and deleting labels and then re-labeling are carried out on the labeling result, and the pathological expert gives an audit result and a diagnosis suggestion aiming at the labeling result.

The method comprises the steps of identifying uploaded pathological images through a trained deep learning model to obtain classification results, position coordinates of each classification and corresponding probability of each classification of the pathological images, automatically labeling parts identified as tumor regions to generate labeling results, displaying the labeling results of the tumor regions in the pathological images, returning the labeling results to a pathologist, and if the pathologist considers that the labeling results are wrong, performing operations including labeling addition, labeling modification and labeling deletion and then labeling again on the labeling results, wherein the pathologist gives examination results and diagnosis suggestions according to the labeling results. The invention carries out two-classification automatic labeling on normal and pathological multi-system type pathological images, sets the number of each type of data set to be basically consistent, provides a pathological image automatic labeling algorithm based on cluster analysis, gives XML vector diagram labeling, designs a doctor review function, ensures the accuracy of automatic labeling, combines with the actual flow of a hospital, and relieves the workload of a pathological specialist.

Drawings

The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and together with the description serve to explain the principles of the invention. Other embodiments and many of the intended advantages of embodiments will be readily appreciated as they become better understood by reference to the following detailed description. Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:

FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;

FIG. 2 is a flow chart of an automatic labeling method suitable for multi-type pathological images according to an embodiment of the present invention;

FIG. 3 is a block diagram of the overall system for automatic labeling of multiple types of pathological images according to an embodiment of the present invention;

FIG. 4 is a flow diagram of an expert module for automatic labeling of multiple types of pathology images in accordance with a specific embodiment of the present invention;

FIG. 5 is a block diagram of an automatic labeling system for multiple types of pathological images according to an embodiment of the present invention;

FIG. 6 is a schematic block diagram of a computer system suitable for use in implementing an electronic device according to embodiments of the present application.

Detailed Description

The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.

Fig. 1 illustrates an exemplary system architecture 100 to which an automatic labeling method for multi-type pathological images according to an embodiment of the present application can be applied.

As shown in fig. 1, the system architecture 100 may include

terminal devices

101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the

terminal devices

101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The user may use the

terminal devices

101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. Various applications, such as a data processing application, a data visualization application, a web browser application, etc., may be installed on the

terminal devices

101, 102, 103.

The

terminal apparatuses

101, 102, and 103 may be hardware or software. When the

terminal devices

101, 102, 103 are hardware, they may be various electronic devices including, but not limited to, smart phones, tablet computers, laptop portable computers, desktop computers, and the like. When the

terminal apparatuses

101, 102, 103 are software, they can be installed in the electronic apparatuses listed above. It may be implemented as multiple pieces of software or software modules (e.g., software or software modules used to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.

The server 105 may be a server that provides various services, such as a background information processing server that provides support for pathological images presented on the

terminal devices

101, 102, 103. The background information processing server can process the obtained classification result and generate a processing result (such as a vector diagram label of XML).

It should be noted that the method provided in the embodiment of the present application may be executed by the server 105, or may be executed by the

terminal devices

101, 102, and 103, and the corresponding apparatus is generally disposed in the server 105, or may be disposed in the

terminal devices

101, 102, and 103.

The server may be hardware or software. When the server is hardware, it may be implemented as a distributed server cluster formed by multiple servers, or may be implemented as a single server. When the server is software, it may be implemented as multiple pieces of software or software modules (e.g., software or software modules used to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.

It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

Fig. 2 shows a flowchart of an automatic labeling method applied to multiple types of pathological images according to an embodiment of the present invention. As shown in fig. 2, the method comprises the steps of:

s201: the pathological expert uploads a pathological image, edits related information of the uploaded pathological image, and performs checking, adding, modifying and deleting operations on the uploaded pathological image information, the disease type corresponding to the uploaded pathological image and the patient information corresponding to the uploaded pathological image.

S202: and sending the uploaded pathological images into a trained deep learning model corresponding to the disease types for recognition according to different disease types displayed in the uploaded pathological images, obtaining classification results, position coordinates of each classification and probabilities corresponding to each classification of the uploaded pathological images, automatically labeling the parts recognized as tumor regions to generate labeling results, displaying the labeling results of the tumor regions in the pathological images, and returning the labeling results to a pathology expert.

through a density-based clustering algorithm, identifying blocks of the pathological image by using a trained deep learning model to obtain a classification result and position coordinates of each classification, clustering according to the classification result and the position coordinates of each classification, and calculating a probability mean value, a standard deviation and a variance of each classification;

the support of DZI format image display technology is realized by adopting OpenSeadragon image control, the basic display function of pathological images is realized, free zooming and moving are carried out according to the needs of users, and simultaneously the labeling result is synchronously zoomed and moved along with the needs of the users.

S203: and the pathological expert checks the labeling result, if the labeling result is wrong, the operations including adding labels, modifying labels and deleting labels and then re-labeling are carried out on the labeling result, and the pathological expert gives an audit result and a diagnosis suggestion according to the labeling result.

the newly-established labeling comprises adding a region labeling outline on the tumor region with the missing label in the S202;

the deleting the label comprises deleting the incorrect label in the S202 on a label list and deleting the incorrect label in the S202 on a label display area at the same time;

In a specific embodiment, the method further includes S204:

In a specific embodiment, the method further includes S205:

Fig. 3 shows an overall block diagram of the system for automatic labeling of multiple types of pathological images according to a specific embodiment of the present invention, and an embodiment of the present invention is described below according to the system shown in fig. 3, so as to further explain the present invention.

The system consists of an expert module and a manager module;

FIG. 4 illustrates a flow diagram of an expert module for automatic labeling of multiple types of pathology images in accordance with a specific embodiment of the present invention;

the expert module is used for uploading and managing pathological images and patient information by experts, checking and auditing tumor areas automatically labeled by AI of the pathological images, and adding, modifying, deleting and storing labeling results;

the administrator module is used for an administrator to manage expert information, check pathological image information and patient information and download an annotation file which is already checked by an expert;

further, the expert module includes an image list display for displaying the pathological image information and its corresponding patient information.

Further, the expert module comprises a pathological image uploading function, and the expert uploads the pathological image through uploading the image and uploads the pathological image to the background server for storage.

Further, the expert module comprises pathological image information management, and experts can add patient information corresponding to pathological images and disease types corresponding to the pathological images for subsequent automatic labeling of multi-type pathological images.

Furthermore, the expert module comprises image display, supports of DZI format image display technology are realized by adopting OpenSeadrain image controls, basic display functions of pathological images are realized, images can be freely zoomed and moved according to needs of users, the zoom times can reach sixty times, and annotation experts can conveniently and carefully check image information in the pathological images.

Further, the expert module comprises automatic labeling of pathological images, and mainly comprises the following steps:

(1) inputting a pathological image, and performing sliding cutting on the pathological image by using an image mask method to obtain a plurality of 512 x 512 cell tissue areas;

(2) inputting the cut small images of the cell tissue regions into a designed convolutional neural network recognition model for recognition, and obtaining the relevant coordinates of the small images and the canceration probability of the small images through recognition;

(3) clustering the small graph coordinates judged to be positive by using a density-based clustering algorithm;

(4) after clustering is finished, calculating the average probability, standard deviation and variance of each cluster, setting a related threshold value, filtering the clusters, and deleting the clusters which do not meet the requirements;

(5) and (3) acquiring the edge points of each cluster by using a Graham-Scan convex hull algorithm, and connecting the edge points one by one to obtain the label of the tumor area of the pathological image.

Furthermore, the expert module comprises annotation display and operation, namely, the corresponding annotation display is ensured on the basis of pathological image display. The labeling information comprises position information of the labeling area and lesion type classification information. During the zooming and moving of the image, the labeling information is zoomed and moved along with the zooming and moving of the image. Meanwhile, the marking operation comprises the operations of creating a mark, deleting the mark, modifying the shape of the mark, modifying the type of the mark, saving the mark and the like. And adding a region labeling outline on the AI missed-labeled tumor region by newly establishing a label. The system provides four drawing forms (pencil drawing, polygon drawing, rectangle drawing and circle drawing), the marking tool is mainly used for drawing according to the moving position of the mouse, and the drawing is mainly supported by paper. Deleting the label allows the expert to delete the label with incorrect AI on the label list and correspondingly delete the label in the label display area; the label modification can modify the labeled area outline in the original label display area and can also modify the labeled classification information in a label list; and the step of storing the label is to submit and store all current label information to the database.

Further, the expert module writes and submits the audit opinions and stores the audit opinions in the database.

Furthermore, the administrator module comprises an expert management module which is used for adding, deleting and modifying the expert account password and the personal information.

Further, the administrator module comprises an image management module for checking the image number, the image type and the patient information corresponding to the pathological image. The image management module also comprises a function of downloading the annotated file which is already checked by experts.

Furthermore, a B/S framework is adopted, and the method is suitable for running on middle and low-end computers. An automatic labeling module and an expert auditing module are introduced, so that the workload of experts is reduced, and a labeling result is more accurate.

Fig. 5 is a frame diagram of an automatic labeling system suitable for multi-type pathological images according to an embodiment of the present invention. The system comprises a pathological image management module 501, a pathological image automatic labeling module 502 and a pathological image labeling auditing module 503.

In a specific embodiment, the pathology image management module 501 is configured to upload pathology images by a pathology expert, edit information related to the uploaded pathology images, and perform operations of viewing, adding, modifying, and deleting on the uploaded pathology image information, disease types corresponding to the uploaded pathology images, and patient information corresponding to the uploaded pathology images;

the pathological image automatic labeling module 502 is configured to send the uploaded pathological images into a trained deep learning model corresponding to disease types for recognition according to different disease types displayed in the uploaded pathological images, obtain classification results, position coordinates of each classification and probabilities corresponding to each classification of the uploaded pathological images, automatically label parts recognized as tumor regions to generate labeling results, display the labeling results of the tumor regions in the pathological images, and return the labeling results to a pathology expert;

the pathological image annotation auditing module 503 is configured to check the annotation result by a pathologist, and if the annotation result is considered to be wrong, perform operations including adding annotations, modifying annotations, deleting annotations, and then re-annotating the annotation result, and the pathologist gives an auditing result and a diagnosis suggestion for the annotation result.

The system gives XML vector diagram annotation of the pathological image according to a pathological image automatic annotation algorithm based on cluster analysis, designs a doctor auditing function, ensures the accuracy of automatic annotation, combines with the actual flow of a hospital, and relieves the workload of a pathological specialist.

Referring now to FIG. 6, shown is a block diagram of a computer system 600 suitable for use in implementing the electronic device of an embodiment of the present application. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.

As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU)601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Liquid Crystal Display (LCD) and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program performs the above-described functions defined in the method of the present application when executed by a Central Processing Unit (CPU) 601. It should be noted that the computer readable storage medium described herein can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable storage medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules described in the embodiments of the present application may be implemented by software or hardware. The units described may also be provided in a processor, and the names of the units do not in some cases constitute a limitation of the unit itself.

Embodiments of the present invention also relate to a computer-readable storage medium having stored thereon a computer program which, when executed by a computer processor, implements the method above. The computer program comprises program code for performing the method illustrated in the flow chart. It should be noted that the computer readable medium of the present application can be a computer readable signal medium or a computer readable medium or any combination of the two.

The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the invention. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.

Claims

1. An automatic labeling method suitable for multi-type pathological images is characterized by comprising the following steps:

2. The method according to claim 1, wherein the uploading of the pathological images is sent to a trained deep learning model corresponding to disease types for recognition according to different disease types displayed in the uploaded pathological images, classification results, position coordinates of each classification and probabilities corresponding to each classification of the uploaded pathological images are obtained, and automatic labeling is performed on parts recognized as tumor regions to generate labeling results, specifically including:

3. The method of claim 2, wherein displaying the labeling result of the tumor region in a pathology image comprises:

4. The method according to claim 1, further comprising S4:

5. The method according to claim 1, further comprising S5:

6. The method of claim 1, wherein displaying the labeling result of the tumor region in a pathology image further comprises:

7. The method according to claim 1, wherein before the uploaded pathological image is sent to a trained deep learning model corresponding to a disease type for recognition, an image mask method is used to segment a plurality of small cell tissue region maps from the uploaded pathological image, and then the segmented small cell tissue region maps are input into the trained deep learning model for classification and recognition.

8. The method according to claim 1, wherein the labeling result comprises a vector diagram label of XML, specifically comprising position information of the labeled region and classification information of the lesion type.

9. The method according to claim 1, wherein the performing operations on the label in the labeling result, which include adding the label, modifying the label, and re-labeling after deleting the label, specifically includes:

10. The method according to claim 1, wherein the labeling result for displaying the tumor region in the pathology image is drawn based on a drawing form including pencil drawing, polygon drawing, rectangle drawing and circle drawing, and is drawn according to a moving position of a mouse based on paper.

11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a computer processor, carries out the method of any one of claims 1 to 10.

12. An automatic labeling system suitable for multi-type pathological images, comprising: