CN117012345A - Semi-automatic labeling method, system, equipment and medium for medical image - Google Patents
Semi-automatic labeling method, system, equipment and medium for medical image Download PDFInfo
- Publication number
- CN117012345A CN117012345A CN202310959137.9A CN202310959137A CN117012345A CN 117012345 A CN117012345 A CN 117012345A CN 202310959137 A CN202310959137 A CN 202310959137A CN 117012345 A CN117012345 A CN 117012345A
- Authority
- CN
- China
- Prior art keywords
- medical image
- labeling
- model
- ophthalmic
- medical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000002372 labelling Methods 0.000 title claims abstract description 224
- 238000003709 image segmentation Methods 0.000 claims abstract description 82
- 238000013145 classification model Methods 0.000 claims abstract description 56
- 238000012549 training Methods 0.000 claims abstract description 55
- 238000013441 quality evaluation Methods 0.000 claims abstract description 27
- 238000012545 processing Methods 0.000 claims abstract description 26
- 238000000034 method Methods 0.000 claims abstract description 24
- 238000012216 screening Methods 0.000 claims abstract description 24
- 238000012360 testing method Methods 0.000 claims description 19
- 230000006870 function Effects 0.000 claims description 15
- 238000001303 quality assessment method Methods 0.000 claims description 11
- 238000004140 cleaning Methods 0.000 claims description 10
- 238000007781 pre-processing Methods 0.000 claims description 10
- 238000010276 construction Methods 0.000 claims description 6
- 238000013178 mathematical model Methods 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000000586 desensitisation Methods 0.000 claims description 4
- 238000005516 engineering process Methods 0.000 abstract description 10
- 238000001514 detection method Methods 0.000 abstract description 6
- 238000007726 management method Methods 0.000 description 10
- 230000007547 defect Effects 0.000 description 6
- 239000008186 active pharmaceutical agent Substances 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 3
- 238000012550 audit Methods 0.000 description 3
- 230000015556 catabolic process Effects 0.000 description 3
- 238000006731 degradation reaction Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 208000030533 eye disease Diseases 0.000 description 2
- 230000003902 lesion Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 239000000243 solution Substances 0.000 description 2
- 208000010412 Glaucoma Diseases 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 206010012601 diabetes mellitus Diseases 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 210000001525 retina Anatomy 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
- 238000012384 transportation and delivery Methods 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
The application discloses a semi-automatic labeling method, a system, equipment and a medium for medical images, wherein the method comprises the steps of constructing and training a medical image segmentation model, a medical image classification model and an ophthalmic picture quality evaluation model; collecting fundus images and utilizing an ophthalmic image quality evaluation model to carry out image quality screening; creating a labeling task and a labeling platform; non-artificial labeling is carried out by utilizing a medical image segmentation model and a medical image classification model; and (5) manual labeling and auditing. By combining the artificial intelligent image processing technology and the artificial labeling mode, the method provided by the embodiment of the application covers a plurality of business scenes such as image classification, image segmentation, image detection, quality evaluation and the like, so that the clinical decision of medical science is assisted, the medical quality is improved, the medical risk is reduced, the burden of medical workers is lightened, the efficiency of the medical workers is greatly improved, and the method has higher clinical use value.
Description
Technical Field
The application relates to the technical field of medical image processing, in particular to a medical image semiautomatic labeling method, a system, equipment and a medium.
Background
For medical images, in particular fundus pictures, the image information is usually marked manually, which takes a lot of manpower and time. The classification of common eye diseases is up to tens of times at present, the fundus image information obtained by an image machine is various, and the manual classification mode is extremely time-consuming and tedious. Along with the increasing number of various eye diseases such as glaucoma, age-related macular, diabetic retina and the like, the working efficiency of medical workers at the present stage for using various labeling tools through manual means is lower.
Disclosure of Invention
The present application aims to solve at least one of the technical problems existing in the prior art. Therefore, the application provides a semi-automatic labeling method for medical images, which solves the problem that the efficiency of manually labeling focus of medical images is low at present.
The embodiment of the application also provides a medical image semiautomatic labeling system, medical image semiautomatic labeling equipment and a computer readable storage medium.
According to an embodiment of the first aspect of the application, the medical image semiautomatic labeling method comprises the following steps:
constructing and training a medical image segmentation model, a medical image classification model and an ophthalmic picture quality assessment model;
collecting a plurality of fundus images and performing preprocessing operation to obtain a fundus image dataset;
performing image quality screening on the fundus image dataset by using manual processing and/or the ophthalmic image quality evaluation model to obtain a high-quality fundus image dataset;
selecting a fundus image to be marked from the high-quality fundus image data set to create an ophthalmic marking task;
constructing an ophthalmic labeling platform, and integrating the medical image segmentation model and the medical image classification model into the ophthalmic labeling platform;
invoking the medical image segmentation model and the medical image classification model in the ophthalmic labeling platform to perform non-manual labeling on the ophthalmic labeling task so as to obtain a first labeling result;
manually labeling the first labeling result in the ophthalmic labeling platform to obtain a second labeling result;
and auditing the second labeling result to obtain a final labeling result.
The medical image semiautomatic labeling method provided by the embodiment of the application has at least the following beneficial effects:
the ophthalmic image quality evaluation model is utilized to conduct intelligent image quality screening, and manual image quality screening of professionals is combined, so that image images with complete outline and clear focus can be reserved for focus marking; the medical image segmentation model and the medical image classification model are utilized in the ophthalmic labeling platform to conduct intelligent focus labeling, and manual labeling of persons in the professional field is combined, so that a large amount of repeated labor can be reduced by utilizing the intelligent focus labeling to improve labeling efficiency, the defects of the intelligent focus labeling can be adjusted and compensated by utilizing the manual labeling, and the accuracy of focus recognition is ensured. Therefore, by combining the artificial intelligent image processing technology and the artificial labeling mode, the method provided by the embodiment of the application covers a plurality of business scenes such as image classification, image segmentation, image detection, quality evaluation and the like, thereby assisting medical clinical decision, improving medical quality, reducing medical risks, relieving the burden of medical workers, greatly improving the efficiency of the medical workers and having higher clinical use value.
According to some embodiments of the application, the medical image semiautomatic labeling method further comprises the steps of:
and training the medical image segmentation model and the medical image classification model by utilizing the final labeling result so as to improve the labeling performance of the medical image segmentation model and the medical image classification model.
According to some embodiments of the application, the preprocessing operation comprises the steps of:
and performing data cleaning operation, unstructured data conversion to structured data operation and data desensitization operation on each acquired fundus image.
According to some embodiments of the application, the medical image segmentation model is constructed and trained, comprising the steps of:
constructing the medical image segmentation model, wherein the medical image segmentation model adopts a Unet;
preparing medical image segmentation training data and medical image segmentation test data;
inputting the medical image segmentation training data into the medical image segmentation model for iterative update training until a first loss value of the medical image segmentation model processing the medical image segmentation test data is minimum, and outputting a final medical image segmentation model, wherein the first loss value is constrained by the following mathematical model:
wherein L is dice Representing a DiceLoss loss function; n represents the number of pixels of the medical image segmentation map; n is the nth pixel point in the medical image segmentation map; r is (r) n A true label representing an nth pixel in the medical image segmentation map; p is p n Representing the corresponding pixelPredicting a probability value; e represents a constant.
According to some embodiments of the application, constructing and training the medical image classification model comprises the steps of:
constructing the medical image classification model, wherein the medical image classification model adopts ResNet50;
preparing medical image classification training data and medical image classification test data;
inputting the medical image classification training data into the medical image classification model for iterative update training until a second loss value of the medical image classification model processing the medical image classification test data is minimum, and outputting a final medical image classification model, wherein the second loss value is constrained by the following mathematical model:
wherein L is ce Representing a CELoss loss function; n represents the total number of samples of the training data of the classification of the medical image in one round of training; c represents the number of classification categories; y is ic Representing a sign function (0 or 1), i.e. taking 1 if the true class of sample i is equal to c, or taking 0 otherwise; p is p ic Representing the predicted probability that the observation sample i belongs to category c.
According to some embodiments of the application, constructing the ophthalmic labeling platform employs a browser/server architecture, the browser/server architecture includes a client and a server, the server includes an application layer, a service layer and a data layer, the medical image segmentation model and the medical image classification model are integrated into the service layer, and the client is used for requesting the application layer for the manual labeling after a user logs in.
According to some embodiments of the application, the ophthalmic picture quality assessment model employs a HyperNet network and/or a transducer model.
According to a second aspect of the present application, a medical image semiautomatic labeling system includes:
the model construction and training module is used for constructing and training a medical image segmentation model, a medical image classification model and an ophthalmic picture quality evaluation model;
the fundus image acquisition module is used for acquiring a plurality of fundus images and performing preprocessing operation so as to obtain a fundus image dataset;
the image quality screening module is used for carrying out image quality screening on the fundus image dataset by utilizing manual processing and/or the ophthalmology picture quality evaluation model so as to obtain a high-quality fundus image dataset;
the task creation module is used for selecting a fundus image to be marked from the high-quality fundus image data set so as to create an ophthalmic marking task;
the marking platform construction module is used for constructing an ophthalmic marking platform and integrating the medical image segmentation model and the medical image classification model into the ophthalmic marking platform;
the non-manual labeling module is used for calling the medical image segmentation model and the medical image classification model in the ophthalmic labeling platform to perform non-manual labeling on the ophthalmic labeling task so as to obtain a first labeling result;
the manual labeling module is used for manually labeling the first labeling result in the ophthalmic labeling platform to obtain a second labeling result;
and the auditing module is used for auditing the second labeling result to obtain a final labeling result.
The medical image semiautomatic labeling system provided by the embodiment of the application has at least the following beneficial effects:
the ophthalmic image quality evaluation model is utilized to conduct intelligent image quality screening, and manual image quality screening of professionals is combined, so that image images with complete outline and clear focus can be reserved for focus marking; the medical image segmentation model and the medical image classification model are utilized in the ophthalmic labeling platform to conduct intelligent focus labeling, and manual labeling of persons in the professional field is combined, so that a large amount of repeated labor can be reduced by utilizing the intelligent focus labeling to improve labeling efficiency, the defects of the intelligent focus labeling can be adjusted and compensated by utilizing the manual labeling, and the accuracy of focus recognition is ensured. Therefore, by combining the artificial intelligent image processing technology and the artificial labeling mode, the system provided by the embodiment of the application covers a plurality of business scenes such as image classification, image segmentation, image detection, quality evaluation and the like, so that the clinical decision of medical science is assisted, the medical quality is improved, the medical risk is reduced, the burden of medical workers is lightened, the efficiency of the medical workers is greatly improved, and the system has higher clinical use value.
According to a third aspect of the application, the medical image semiautomatic labeling device comprises at least one control processor and a memory for communication connection with the at least one control processor; the memory stores instructions executable by the at least one control processor to enable the at least one control processor to perform a medical image semi-automatic labeling method according to an embodiment of the first aspect of the present application.
According to a fourth aspect of the present application, there is provided a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the medical image semiautomatic labeling method according to the first aspect of the present application.
It is to be understood that the advantages of the third aspect and the fourth aspect compared with the related art are the same as those of the first aspect compared with the related art, and reference may be made to the related description in the first aspect, which is not repeated herein.
Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application.
Drawings
The foregoing and/or additional aspects and advantages of the application will become apparent and may be better understood from the following description of embodiments taken in conjunction with the accompanying drawings in which:
FIG. 1 is a flow chart of a method for semi-automatic labeling of medical images according to one embodiment of the present application;
FIG. 2 is a schematic diagram of a medical image semiautomatic labeling method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of an ophthalmic labeling platform employing a browser/server architecture according to one embodiment of the present application;
FIG. 4 is a schematic illustration of the effect of non-manual labeling in an ophthalmic labeling platform according to one embodiment of the application;
FIG. 5 is a schematic illustration of the effect of manual labeling in an ophthalmic labeling platform according to one embodiment of the present application;
FIG. 6 is a schematic diagram of a medical image semi-automatic labeling system according to an embodiment of the present application.
Reference numerals:
a model building and training module 100;
a fundus image acquisition module 200;
an image quality screening module 300;
a task creation module 400;
the annotation platform build module 500;
a non-manual annotation module 600;
a manual annotation module 700;
an audit module 800.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the application.
In the description of the present application, the description of first, second, etc. is for the purpose of distinguishing between technical features only and should not be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.
In the description of the present application, it should be understood that the direction or positional relationship indicated with respect to the description of the orientation, such as up, down, etc., is based on the direction or positional relationship shown in the drawings, is merely for convenience of describing the present application and simplifying the description, and does not indicate or imply that the apparatus or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present application.
In the description of the present application, unless explicitly defined otherwise, terms such as arrangement, installation, connection, etc. should be construed broadly and the specific meaning of the terms in the present application can be determined reasonably by a person skilled in the art in combination with the specific content of the technical solution.
The following description of the embodiments of the present application will be made with reference to the accompanying drawings, in which it is apparent that the embodiments described below are some, but not all embodiments of the application.
Referring to fig. 1, a flowchart of a method for semi-automatically labeling medical images according to an embodiment of the present application is shown, where the method includes the following steps:
constructing and training a medical image segmentation model, a medical image classification model and an ophthalmic picture quality assessment model;
collecting a plurality of fundus images and performing preprocessing operation to obtain a fundus image dataset;
performing image quality screening on the fundus image dataset by using a manual processing and/or ophthalmic picture quality evaluation model to obtain a high-quality fundus image dataset;
selecting a fundus image to be marked from the high-quality fundus image data set to create an ophthalmic marking task;
constructing an ophthalmic labeling platform, and integrating a medical image segmentation model and a medical image classification model into the ophthalmic labeling platform;
invoking a medical image segmentation model and a medical image classification model in an ophthalmic labeling platform to perform non-manual labeling on an ophthalmic labeling task so as to obtain a first labeling result;
manually labeling the first labeling result in the ophthalmic labeling platform to obtain a second labeling result;
and auditing the second labeling result to obtain a final labeling result.
Specifically, as shown in fig. 1, it can be understood that three models, specifically, a medical image segmentation model and a medical image classification model, are first established and trained for image lesion labeling, and an ophthalmic picture quality assessment model is used for image quality screening. Then, a plurality of fundus images are selected for preliminary processing to form a fundus image dataset. And screening out the fundus images with qualified quality from the fundus image dataset by using an ophthalmic picture quality evaluation model or manual processing, thereby forming a high-quality fundus image dataset.
The purpose of the image quality screening is to: because the image picture data is taken by different doctors from different devices, there are cases where the picture quality is uneven. Therefore, before labeling focus, cleaning and screening on the definition level are needed to be carried out on the image data, in particular, the image picture with poor quality is removed, and the image picture with complete outline and clear focus is reserved, so that the image picture is used as a formal label.
The cleaning and screening modes can be divided into manual treatment and cleaning, namely manual marking or intelligent cleaning, namely artificial intelligence technology is utilized. For manual processing and cleaning, a medical background professional is provided to mark the picture quality according to three categories of clear, acceptable or unqualified definition. For intelligent cleaning, an ophthalmic image quality evaluation model is utilized to score and evaluate the image images, then, medical background professionals are combined to evaluate the image images with different scores, finally, a picture definition threshold is determined, and therefore, the image images are finally classified into three grades of definition, acceptability or disqualification. The high-quality fundus image dataset can be composed by selecting only the image picture with clear picture quality or selecting the image picture with clear and receivable picture quality according to the requirements of different embodiments.
Further, an ophthalmic labeling task is established by utilizing the high-quality fundus image data set, and focus labeling is performed on the established ophthalmic labeling platform. Specifically, the ophthalmic labeling task may include a plurality of fundus images, so in some embodiments, the plurality of fundus images are packaged into a queue, model call of intelligent labeling is performed one by one, each call succeeds in one, a picture task is distributed for subsequent manual labeling, then a manual labeling person checks the result of the intelligent labeling, modifies and adjusts the images with incorrect labeling or inaccurate focus segmentation, and obtains a final labeling result after auditing.
It can be appreciated that the time spent on manual labeling can be greatly reduced by adopting an artificial intelligence technology to process images, but for a model with a limited training data set, accurate identification and segmentation of most lesions can be generally achieved, and the defect can not be overcome by adopting manual labeling later. The purely manual labeling is inevitably faced with the problem of low manual injection efficiency of a large number of focus, and the artificial intelligence technology can reduce a large amount of repeated labor.
In the embodiment, the image picture with complete outline and clear focus can be reserved for focus marking by utilizing the ophthalmic picture quality evaluation model to carry out intelligent image quality screening and combining with artificial image quality screening of the professional field personnel; the medical image segmentation model and the medical image classification model are utilized in the ophthalmic labeling platform to conduct intelligent focus labeling, and manual labeling of persons in the professional field is combined, so that a large amount of repeated labor can be reduced by utilizing the intelligent focus labeling to improve labeling efficiency, the defects of the intelligent focus labeling can be adjusted and compensated by utilizing the manual labeling, and the accuracy of focus recognition is ensured. Therefore, by combining the artificial intelligent image processing technology and the artificial labeling mode, the method provided by the embodiment of the application covers a plurality of business scenes such as image classification, image segmentation, image detection, quality evaluation and the like, thereby assisting medical clinical decision, improving medical quality, reducing medical risks, relieving the burden of medical workers, greatly improving the efficiency of the medical workers and having higher clinical use value.
In some embodiments, the medical image semi-automatic labeling method further comprises the steps of:
and training the medical image segmentation model and the medical image classification model by utilizing the final labeling result so as to improve the labeling performance of the medical image segmentation model and the medical image classification model.
Specifically, it can be understood that by adjusting the result of the intelligent labeling by using the manual labeling, the defect that the intelligent labeling may have inaccurate labeling can be overcome, so that the final labeling result with more accurate focus labeling is input into the medical image segmentation model and the medical image classification model, and model training can be performed to strengthen the performance of the intelligent labeling.
In some embodiments, as shown in fig. 2, the preprocessing operation includes the steps of:
and performing data cleaning operation, unstructured data conversion to structured data operation and data desensitization operation on each acquired fundus image.
In particular, referring to fig. 2, it will be appreciated that fig. 2 is a schematic diagram of a design for implementing the method of the embodiments of the present application. As can be seen from the figure, in this embodiment, the data acquisition system is configured to acquire a plurality of fundus image data and store the fundus image data in a warehouse; the ophthalmology standard data management platform is used for preprocessing operations including simple data cleaning, unstructured data conversion, data type classification and other prepositive actions, and then image desensitizing tools are used for carrying out data desensitizing operations; the ophthalmic labeling platform acquires the desensitized fundus image through file service and can perform the following management: marking tool management, label management, task management, report management, and the like. For a newly built labeling task, the ophthalmic labeling platform selects intelligent labeling, the task can automatically generate an image labeling result, and modification and review of the intelligent labeling result can be manually operated on the ophthalmic labeling platform.
In some embodiments, constructing and training a medical image segmentation model includes the steps of:
constructing a medical image segmentation model, wherein the medical image segmentation model adopts a Unet;
preparing medical image segmentation training data and medical image segmentation test data;
inputting the medical image segmentation training data into a medical image segmentation model for iterative update training until a first loss value of medical image segmentation test data processed by the medical image segmentation model is minimum, and outputting a final medical image segmentation model, wherein the first loss value is constrained by the following mathematical model:
wherein L is dice Representing a DiceLoss loss function; n represents the number of pixels of the medical image segmentation map; n is the nth pixel point in the medical image segmentation map; r is (r) n A true label representing an nth pixel in the medical image segmentation map; p is p n Representing a predicted probability value for the corresponding pixel; e represents a constant.
Specifically, it can be understood that the medical image segmentation model uses the uiet, the training data is a public data set, such as DDR, IDRID, ADAM and e_optha, and the data set is divided into a training set and a test set in the training process, and the ratio of the training set to the test set is 8:2, the loss function uses DiceLoss. The medical image segmentation model is continuously trained until the loss value of the model in the test data is minimum, so as to obtain a final medical image segmentation model.
In some embodiments, constructing and training a medical image classification model includes the steps of:
constructing a medical image classification model, wherein the medical image classification model adopts ResNet50;
preparing medical image classification training data and medical image classification test data;
inputting the medical image classification training data into a medical image classification model for iterative update training until a second loss value of medical image classification test data processed by the medical image classification model is minimum, and outputting a final medical image classification model, wherein the second loss value is constrained by the following mathematical model:
wherein L is ce Representing a CELoss loss function; n represents the total number of samples of the training data of the classification of the medical image in one round of training; c represents the number of classification categories; ic representing a sign function (0 or 1), i.e. taking 1 if the true class of sample i is equal to c, or taking 0 otherwise; p is p ic Representing the predicted probability that the observation sample i belongs to category c.
Specifically, it can be appreciated that the medical image classification model uses ResNet50, the training data is a public data set, such as DDR, IDRID, ADAM and E_OPTHA, and the data set is divided into a training set and a testing set in the training process, and the ratio of the training set to the testing set is 8:2, using CELoss as the loss function. The medical image classification model is continuously trained until the loss value of the model in the test data is minimum, so as to obtain a final medical image classification model.
In some embodiments, as shown in fig. 3, the ophthalmic labeling platform is constructed by using a browser/server architecture, where the browser/server architecture includes a client and a server, the server includes an application layer, a service layer, and a data layer, the medical image segmentation model and the medical image classification model are integrated into the service layer, and the client is used for requesting the application layer for manual labeling after a user logs in.
Specifically, referring to fig. 3, it can be understood that the ophthalmologic labeling platform adopts a browser/server architecture (B/S architecture) with separated front ends, and after the front end labeling/auditor logs in the system, according to the task binding relationship, the task under each name can be seen, and the labeling of the task can be performed after clicking to enter the rendering page.
Further, the server architecture mainly comprises an application layer, a service layer and a data layer. After the client logs in the system, static pages and script data are acquired through an Nginx layer of the application layer. After the page is loaded, the user can check the information such as the task under the name, the real-time labeling quantity and the like.
Specifically, the Nginx layer primarily provides web server, reverse proxy and WAF functions. The web server is used for the browser client to acquire page static resources; the reverse proxy forwards all browser requests to each micro-service at the back end; the WAF provides access control means such as whitelists and authentication for all upstream services.
The API gateway layer mainly performs load balancing and login state, access authentication and other verification on the API interface of the service layer.
The micro service layer is an actual back-end service, a labeling service system, a file service and a user login SSO service are deployed in the layer, and each micro service quality inspection performs real-time service discovery and call load balancing through a registration center. In order to facilitate on-line configuration modification, a dynamic configuration center is used for service configuration management. Wherein:
the business system mainly provides task management, task distribution, picture marking and auditing functions. And calling an API of the artificial intelligent image processing model through the algorithm integration service to pre-label the picture at the beginning of task creation. Then, the annotator views the intelligently annotated picture on the annotating system, and the annotator carries out annotating calibration in the modes of modification, supplement, deletion, adjustment and the like on the basis of the intelligent annotating, so that focus segmentation and detection annotating can be rapidly completed. After the labeling of the labeling personnel is finished, the labeling personnel performs auditing and final adjustment of the labeling results, and the labeling results are put in storage after confirming that the problem exists. Finally, the labeling result can be used for model performance improvement training or other scientific application scenes according to the requirement.
The SSO service provides the functional services of a label person, an auditor, a system administrator, data maintenance, authority control, login and logout and the like.
The file service provides unified file management for the labeling object data set, the labeling system only needs to store the picture address of the file service, when a user labels a page, the user obtains a picture file through the URL of the picture, and when the user actually stores the picture file, the user stores labeling results through JSON data.
Further, the data layer maintains all static and dynamic data, including page data of a memory disk, redis cache data controlled by user access, result data of image annotation of a storage annotation library, image data of a storage object library (MinIO), and the like.
With continued reference to fig. 4 and fig. 5, it may be understood that in the ophthalmic labeling platform of this embodiment, first, a labeling task is created, a task type is created to select an intelligent algorithm label, and an API of an artificial intelligent image processing model is called to perform image pre-labeling, so that an obtained intelligent labeling effect is shown in fig. 4. After the task is successfully created, the task is distributed to the selected annotators, and the annotators can see the task to be handled in the annotation management. The annotate interface displays the completed annotation pictures, displays the completion state and annotates the instrument: points, lines, circles/ellipses, rectangles, polygons, brushes; image processing: zoom-in and zoom-out, brightness, contrast; auxiliary tool: moving, editing, retracting and deleting; characteristic function: the sub-label is hidden and has no red light mode. And (5) manually marking the pictures in the interface by using the tool on the ophthalmic marking platform by a marking person, clicking to finish marking, ending editing and storing marking information, ending editing, and obtaining the manual marking effect shown in figure 5.
Further, auditors are allocated when the task is created, and can see the task in audit management. And the intelligent labeling results are subjected to auditing and modifying by means of a series of labeling tools on the labeling panel, and the labeling results are modified or confirmed by auditing the picture list, the labeling drawing area, the labeling record, the labeling information and the like, so that reasonable labeling consistency judgment is achieved.
In some embodiments, the ophthalmic picture quality assessment model employs a HyperNet network and/or a transducer model.
Specifically, it can be understood that the ophthalmic image quality evaluation model adopts a correlation algorithm model of image quality evaluation (Image Quality Assessment, IQA), which is one of the basic techniques in image processing, mainly by performing a characteristic analysis study on an image, and then evaluating the image quality (degree of image distortion). The image quality evaluation plays an important role in the aspects of algorithm analysis and comparison, system performance evaluation and the like in an image processing system. In recent years, with extensive research in the field of digital images, research on image quality evaluation has also been receiving increasing attention from researchers, and many indexes and methods of image quality evaluation have been proposed and perfected.
In some embodiments, the ophthalmic picture quality assessment model may employ a HyperNet network, a Transformer model, or a combination of HyperNet network and Transformer model, among other image quality assessment algorithm models. Model training is carried out on an ophthalmic image quality evaluation model, a data set of the model training is from private data, such as Shenzhen ophthalmic hospitals, the data is subjected to desensitization treatment, 100 pairs of fundus images are taken in total, and a low-quality fuzzy fundus image and a high-quality clear fundus image in each pair are taken in actual diagnosis. The spatial resolution of each image is 2560×2560. Wherein, the degradation type of the low-quality blurred fundus image comprises virtual focus blur, motion blur, insufficient illumination, artifacts and the like. The degradation types contained in each low quality fundus image are nonlinear superposition and mixing of multiple degradation. 80 pairs in the dataset were selected as the training set, and the remaining 20 pairs of data were selected as the test set.
In addition, as shown in fig. 6, the embodiment of the application further provides a semi-automatic labeling system for medical images, which comprises:
a model construction and training module 100 for constructing and training a medical image segmentation model, a medical image classification model, and an ophthalmic picture quality assessment model;
a fundus image acquisition module 200 for acquiring a plurality of fundus images and performing a preprocessing operation to obtain a fundus image dataset;
an image quality screening module 300, configured to perform image quality screening on the fundus image dataset by using a manual processing and/or an ophthalmic image quality evaluation model, so as to obtain a high-quality fundus image dataset;
the task creation module 400 is used for selecting a fundus image to be marked from the high-quality fundus image data set so as to create an ophthalmic marking task;
the labeling platform construction module 500 is used for constructing an ophthalmic labeling platform and integrating the medical image segmentation model and the medical image classification model into the ophthalmic labeling platform;
the non-artificial labeling module 600 is configured to call the medical image segmentation model and the medical image classification model in the ophthalmic labeling platform to perform non-artificial labeling on the ophthalmic labeling task so as to obtain a first labeling result;
the manual labeling module 700 is configured to manually label the first labeling result in the ophthalmic labeling platform to obtain a second labeling result;
and the auditing module 800 is configured to audit the second labeling result to obtain a final labeling result.
Specifically, referring to fig. 6, it can be understood that the medical image semiautomatic labeling system according to the embodiment of the present application is used for implementing a medical image semiautomatic labeling method, and the medical image semiautomatic labeling system according to the embodiment of the present application corresponds to the medical image semiautomatic labeling method, and specific processing procedures refer to the medical image semiautomatic labeling method and are not described herein.
In the embodiment, the image picture with complete outline and clear focus can be reserved for focus marking by utilizing the ophthalmic picture quality evaluation model to carry out intelligent image quality screening and combining with artificial image quality screening of the professional field personnel; the medical image segmentation model and the medical image classification model are utilized in the ophthalmic labeling platform to conduct intelligent focus labeling, and manual labeling of persons in the professional field is combined, so that a large amount of repeated labor can be reduced by utilizing the intelligent focus labeling to improve labeling efficiency, the defects of the intelligent focus labeling can be adjusted and compensated by utilizing the manual labeling, and the accuracy of focus recognition is ensured. Therefore, by combining the artificial intelligent image processing technology and the artificial labeling mode, the system provided by the embodiment of the application covers a plurality of business scenes such as image classification, image segmentation, image detection, quality evaluation and the like, so that the clinical decision of medical science is assisted, the medical quality is improved, the medical risk is reduced, the burden of medical workers is lightened, the efficiency of the medical workers is greatly improved, and the system has higher clinical use value.
In addition, the embodiment of the application also provides a medical image semiautomatic labeling device, which comprises: at least one control processor and a memory for communication connection with the at least one control processor.
The memory, as a non-transitory computer readable storage medium, may be used to store non-transitory software programs as well as non-transitory computer executable programs. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory remotely located relative to the processor, the remote memory being connectable to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
A non-transitory software program and instructions required to implement a medical image semi-automatic labeling method of the above embodiments are stored in a memory, which when executed by a processor, performs a medical image semi-automatic labeling method of the above embodiments, for example, performs the method of fig. 1 described above.
The system embodiments described above are merely illustrative, in that the units illustrated as separate components may or may not be physically separate, i.e., may be located in one place, or may be distributed over a plurality of network elements. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, an embodiment of the present application further provides a computer readable storage medium, where computer executable instructions are stored, where the computer executable instructions are executed by one or more control processors, and where the one or more control processors may cause the one or more control processors to perform a method for semi-automatically labeling a medical image in the embodiment of the method, for example, to perform the method in fig. 1 described above.
Those of ordinary skill in the art will appreciate that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
The embodiments of the present application have been described in detail with reference to the accompanying drawings, but the present application is not limited to the above embodiments, and various changes can be made within the knowledge of one of ordinary skill in the art without departing from the spirit of the present application.
Claims (10)
1. The semi-automatic labeling method for the medical image is characterized by comprising the following steps of:
constructing and training a medical image segmentation model, a medical image classification model and an ophthalmic picture quality assessment model;
collecting a plurality of fundus images and performing preprocessing operation to obtain a fundus image dataset;
performing image quality screening on the fundus image dataset by using manual processing and/or the ophthalmic image quality evaluation model to obtain a high-quality fundus image dataset;
selecting a fundus image to be marked from the high-quality fundus image data set to create an ophthalmic marking task;
constructing an ophthalmic labeling platform, and integrating the medical image segmentation model and the medical image classification model into the ophthalmic labeling platform;
invoking the medical image segmentation model and the medical image classification model in the ophthalmic labeling platform to perform non-manual labeling on the ophthalmic labeling task so as to obtain a first labeling result;
manually labeling the first labeling result in the ophthalmic labeling platform to obtain a second labeling result;
and auditing the second labeling result to obtain a final labeling result.
2. The medical image semiautomatic labeling method according to claim 1, further comprising the steps of:
and training the medical image segmentation model and the medical image classification model by utilizing the final labeling result so as to improve the labeling performance of the medical image segmentation model and the medical image classification model.
3. The method for semi-automatic labeling of medical images according to claim 1 or 2, characterized in that said preprocessing operation comprises the following steps:
and performing data cleaning operation, unstructured data conversion to structured data operation and data desensitization operation on each acquired fundus image.
4. The medical image semiautomatic labeling method according to claim 1 or 2, characterized in that the medical image segmentation model is constructed and trained, comprising the steps of:
constructing the medical image segmentation model, wherein the medical image segmentation model adopts a Unet;
preparing medical image segmentation training data and medical image segmentation test data;
inputting the medical image segmentation training data into the medical image segmentation model for iterative update training until a first loss value of the medical image segmentation model processing the medical image segmentation test data is minimum, and outputting a final medical image segmentation model, wherein the first loss value is constrained by the following mathematical model:
wherein L is dice Representing a DiceLoss loss function; n represents the number of pixels of the medical image segmentation map; n is the nth pixel point in the medical image segmentation map; r is (r) n A true label representing an nth pixel in the medical image segmentation map; p is p n Representing a predicted probability value for the corresponding pixel; e represents a constant.
5. The medical image semiautomatic labeling method according to claim 1 or 2, characterized in that the medical image classification model is constructed and trained, comprising the steps of:
constructing the medical image classification model, wherein the medical image classification model adopts ResNet50;
preparing medical image classification training data and medical image classification test data;
inputting the medical image classification training data into the medical image classification model for iterative update training until a second loss value of the medical image classification model processing the medical image classification test data is minimum, and outputting a final medical image classification model, wherein the second loss value is constrained by the following mathematical model:
wherein L is ce Representing a CELoss loss function; n represents the total number of samples of the training data of the classification of the medical image in one round of training; c represents the number of classification categories; y is ic Representing a sign function (0 or 1), i.e. if sample iThe true category is equal to c, taking 1, otherwise taking 0; p is p ic Representing the predicted probability that the observation sample i belongs to category c.
6. The method according to claim 1 or 2, wherein constructing the ophthalmic labeling platform adopts a browser/server framework, the browser/server framework comprises a client and a server, the server comprises an application layer, a service layer and a data layer, the medical image segmentation model and the medical image classification model are integrated into the service layer, and the client is used for requesting the application layer for the manual labeling after a user logs in.
7. The method according to claim 1 or 2, wherein the ophthalmic image quality assessment model adopts a hypersnet network and/or a Transformer model.
8. A medical image semiautomatic labeling system, comprising:
the model construction and training module is used for constructing and training a medical image segmentation model, a medical image classification model and an ophthalmic picture quality evaluation model;
the fundus image acquisition module is used for acquiring a plurality of fundus images and performing preprocessing operation so as to obtain a fundus image dataset;
the image quality screening module is used for carrying out image quality screening on the fundus image dataset by utilizing manual processing and/or the ophthalmology picture quality evaluation model so as to obtain a high-quality fundus image dataset;
the task creation module is used for selecting a fundus image to be marked from the high-quality fundus image data set so as to create an ophthalmic marking task;
the marking platform construction module is used for constructing an ophthalmic marking platform and integrating the medical image segmentation model and the medical image classification model into the ophthalmic marking platform;
the non-manual labeling module is used for calling the medical image segmentation model and the medical image classification model in the ophthalmic labeling platform to perform non-manual labeling on the ophthalmic labeling task so as to obtain a first labeling result;
the manual labeling module is used for manually labeling the first labeling result in the ophthalmic labeling platform to obtain a second labeling result;
and the auditing module is used for auditing the second labeling result to obtain a final labeling result.
9. A medical image semiautomatic labeling device, comprising at least one control processor and a memory for communication connection with the at least one control processor; the memory stores instructions executable by the at least one control processor to enable the at least one control processor to perform the medical image semiautomatic labeling method of any of claims 1-7.
10. A computer-readable storage medium storing computer-executable instructions for causing a computer to perform the medical image semiautomatic labeling method according to any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310959137.9A CN117012345A (en) | 2023-07-31 | 2023-07-31 | Semi-automatic labeling method, system, equipment and medium for medical image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310959137.9A CN117012345A (en) | 2023-07-31 | 2023-07-31 | Semi-automatic labeling method, system, equipment and medium for medical image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117012345A true CN117012345A (en) | 2023-11-07 |
Family
ID=88561327
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310959137.9A Pending CN117012345A (en) | 2023-07-31 | 2023-07-31 | Semi-automatic labeling method, system, equipment and medium for medical image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117012345A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117786465A (en) * | 2024-02-23 | 2024-03-29 | 北京中科闻歌科技股份有限公司 | Method and system for constructing field pre-training model data |
-
2023
- 2023-07-31 CN CN202310959137.9A patent/CN117012345A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117786465A (en) * | 2024-02-23 | 2024-03-29 | 北京中科闻歌科技股份有限公司 | Method and system for constructing field pre-training model data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106709917A (en) | Neural network model training method, device and system | |
| CN107563123A (en) | Method and apparatus for marking medical image | |
| CN107194158A (en) | A kind of disease aided diagnosis method based on image recognition | |
| CN110967339B (en) | Method and device for analyzing corn ear characters and corn character analysis equipment | |
| CN106557995A (en) | Based on intelligent terminal and the Teaching Management Method of cloud platform | |
| US20200082199A1 (en) | Method and apparatus for inspecting burrs of electrode slice | |
| CN108682453A (en) | A kind of Lung neoplasm labeling system | |
| DE102006046746A1 (en) | Computer readable medium e.g. optical medium for patient's diagnostic medical reference image information database, contains instructions to produce database of image variation data and severity class grade with respect to each data | |
| JP7434537B2 (en) | Bidirectional training of machine learning models for tissue segmentation | |
| CN117012345A (en) | Semi-automatic labeling method, system, equipment and medium for medical image | |
| Lin et al. | Retinal image quality assessment for diabetic retinopathy screening: A survey | |
| CN110263748A (en) | Method and apparatus for sending information | |
| WO2022242392A1 (en) | Blood vessel image classification processing method and apparatus, and device and storage medium | |
| CN113576508A (en) | Cerebral hemorrhage auxiliary diagnosis system based on neural network | |
| CN113658175A (en) | Method and device for determining symptom data | |
| CN114121232A (en) | Classification model training method and device, electronic equipment and storage medium | |
| WO2003067256A2 (en) | Systems for analyzing microtissue arrays | |
| CN107590804A (en) | Screen picture quality evaluating method based on channel characteristics and convolutional neural networks | |
| CN108665386B (en) | Subjective scoring examination system and method based on mobile platform and image processing | |
| CN112614570A (en) | Sample set labeling method, pathological image classification method and classification model construction method and device | |
| CN114782452B (en) | Processing system and device of fluorescein fundus angiographic image | |
| CN110020597B (en) | Eye video processing method and system for auxiliary diagnosis of dizziness/vertigo | |
| CN107133474A (en) | Quick-speed generation system and method are reported in pathological diagnosis suitable for lung cancer | |
| CN111462005A (en) | Method, apparatus, computer device and storage medium for processing microscopic image | |
| CN108074240B (en) | Recognition method, recognition apparatus, computer-readable storage medium, and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |