CN114266896A - Image labeling method, model training method and device, electronic equipment and medium - Google Patents
Image labeling method, model training method and device, electronic equipment and medium Download PDFInfo
- Publication number
- CN114266896A CN114266896A CN202111653639.6A CN202111653639A CN114266896A CN 114266896 A CN114266896 A CN 114266896A CN 202111653639 A CN202111653639 A CN 202111653639A CN 114266896 A CN114266896 A CN 114266896A
- Authority
- CN
- China
- Prior art keywords
- labeling
- annotation
- image
- task
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention discloses an image labeling method, a model training device, electronic equipment and a medium. The model training method comprises the following steps: acquiring a plurality of image samples containing a target object, wherein part of the image samples contain first labeling information, other image samples do not contain the first labeling information, and each image sample also comprises second labeling information; inputting each image sample into an annotation model, generating an annotation task result of the image sample by an annotation task network of the annotation model, and predicting whether the image sample contains first annotation information by a discriminator of the annotation model; respectively calculating loss errors of the labeling task network and the loss errors of the discriminator; and adjusting the network parameters of the labeling model according to the loss error of the labeling task network and the loss error of the discriminator until the iteration stop condition is met. Therefore, the efficiency of image annotation can be improved, and the accuracy is higher.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an image annotation method, a model training method and device, electronic equipment and a medium.
Background
In recent years, along with the intensive penetration of AI (Artificial Intelligence) in the medical industry, more and more clinical tasks use AI analysis results as auxiliary diagnosis information, and reduce human subjective errors as much as possible by using AI big data analysis and learning ability, thereby improving diagnosis accuracy. At the same time, this approach also brings new problems: because the prediction ability of AI on clinical disease is not inherent, it is learned by training; in order to train a model capable of assisting clinical practice and having good performance of AI, a large amount of data is essential. Meanwhile, in the unsupervised learning mode, the semi-supervised learning mode and the supervised learning mode, the supervised learning mode is the best choice with better effect. In a supervised manner, this means that the data used for model training is labeled with the corresponding task.
At this stage and for a long time thereafter, it would be difficult to acquire a large amount of high quality medical image annotation data: on one hand, the source of data is limited, and a large amount of complete medical data is difficult to obtain due to privacy protection; on the other hand, the labeling cost is high, considerable professional knowledge and time are needed, and special labeling tools are often needed to be developed and handed to experienced doctors for different labeling contents, especially for pixel-level labeling; and the marking difference caused by the shooting equipment and the shooting environment of the medical image and the subjective idea of the marking personnel, so that the data which can be really used for network training and has high-quality marking is greatly reduced.
Disclosure of Invention
The invention provides an image labeling method, a model training method and device, electronic equipment and a medium, which are used for improving the efficiency and the accuracy of image labeling.
The invention solves the technical problems through the following technical scheme:
in a first aspect, a model training method is provided, including:
acquiring a plurality of image samples containing a target object, wherein imaging parameters of the image samples are not identical, a part of the image samples in the image samples contain first labeling information, other image samples except the part of the image samples in the image samples do not contain the first labeling information, and each image sample also comprises second labeling information; the first labeling information is related to an labeling task for the target object, and the second labeling information represents whether the image sample contains the first labeling information;
for each image sample, inputting the image sample into an annotation model, generating an annotation task result of the image sample by an annotation task network of the annotation model, and predicting whether the image sample contains first annotation information or not by a discriminator of the annotation model according to the annotation task result;
calculating the loss error of the labeling task network according to the first labeling information and the output result of the labeling task network, and calculating the loss error of the discriminator according to the second labeling information and the output result of the discriminator;
and adjusting the network parameters of the labeling model according to the loss error of the labeling task network and the loss error of the discriminator until an iteration stop condition is met.
Optionally, the annotation task network comprises an image segmentation network; the labeling task result comprises a segmentation labeling result of the target object;
or, the labeling task network comprises a classification network; the labeling task result comprises a classification labeling result of the target object;
or the labeling task network comprises an object detection network; the labeling task result comprises a detection labeling result of the target object;
or the labeling task network comprises an object identification network; and the labeling task result comprises an identification labeling result of the target object.
Optionally, the discriminator comprises a classification network; the classification network is used for classifying the labeling task result, and the classification result represents whether the image sample contains the first labeling information.
Optionally, the target object is brain parenchyma; the labeling task result is a segmentation labeling result of the brain parenchyma;
inputting the image sample into an annotation model, and generating an annotation task result of the image sample by an annotation task network of the annotation model, wherein the annotation task result comprises:
inputting an image sample containing brain parenchyma into an annotation model, and carrying out segmentation processing on the image sample by an annotation task network of the annotation model to generate a segmentation and annotation result of the brain parenchyma.
Optionally, the first labeling information is brain parenchymal region information; calculating the loss error of the labeling task network according to the first labeling information and the output result of the labeling task network, and the method comprises the following steps:
and calculating the loss error of the labeling task network according to the brain parenchyma region information and the segmentation labeling result of the brain parenchyma.
Optionally, the imaging parameters comprise display parameters or generation parameters;
when the image sample is a medical image, the display parameters include at least one of: voxel, integrity, sharpness, contrast, color, slice thickness; generating the parameters includes at least one of: scanning parameters, modes, phases;
when the image sample is a two-dimensional image, the display parameters include at least one of: pixel, integrity, sharpness, color; generating the parameters includes at least one of: shutter speed, aperture, sensitivity, exposure value, whether flash is on.
In a second aspect, an image annotation method is provided, which includes:
acquiring an image to be marked;
inputting the image to be annotated into an annotation task network of an annotation model, and annotating the image to be annotated by the annotation task network; wherein, the labeling model is obtained by training the training method of any one of the labeling models.
In a third aspect, a model training apparatus is provided, including:
the device comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring a plurality of image samples containing a target object, imaging parameters of the image samples are not identical, a partial image sample in the image samples contains first labeling information, other image samples except the partial image sample in the image samples do not contain the first labeling information, and each image sample also comprises second labeling information; the first labeling information is related to an labeling task for the target object, and the second labeling information represents whether the image sample contains the first labeling information;
the input module is used for inputting the image samples into an annotation model for each image sample, so that an annotation task result of the image sample is generated by an annotation task network of the annotation model, and a discriminator of the annotation model predicts whether the image sample contains first annotation information according to the annotation task result;
the calculation module is used for calculating the loss error of the labeling task network according to the first labeling information and the output result of the labeling task network, and calculating the loss error of the discriminator according to the second labeling information and the output result of the discriminator;
and the adjusting module is used for adjusting the network parameters of the labeling model according to the loss error of the labeling task network and the loss error of the discriminator until an iteration stop condition is met.
Optionally, the annotation task network comprises an image segmentation network; the labeling task result comprises a segmentation labeling result of the target object;
or, the labeling task network comprises a classification network; the labeling task result comprises a classification labeling result of the target object;
or the labeling task network comprises an object detection network; the labeling task result comprises a detection labeling result of the target object;
or the labeling task network comprises an object identification network; and the labeling task result comprises an identification labeling result of the target object.
Optionally, the discriminator comprises a classification network; the classification network is used for classifying the labeling task result, and the classification result represents whether the image sample contains the first labeling information.
Optionally, the target object is brain parenchyma; the labeling task result is a segmentation labeling result of the brain parenchyma;
the input module is specifically configured to:
inputting an image sample containing brain parenchyma into an annotation model, and carrying out segmentation processing on the image sample by an annotation task network of the annotation model to generate a segmentation and annotation result of the brain parenchyma.
Optionally, the first labeling information is brain parenchymal region information; when calculating the loss error of the labeling task network according to the first labeling information and the output result of the labeling task network, the calculating module is used for:
and calculating the loss error of the labeling task network according to the brain parenchyma region information and the segmentation labeling result of the brain parenchyma.
Optionally, the imaging parameters comprise display parameters or generation parameters;
when the image sample is a medical image, the display parameters include at least one of: voxel, integrity, sharpness, contrast, color, slice thickness; generating the parameters includes at least one of: scanning parameters, modes, phases;
when the image sample is a two-dimensional image, the display parameters include at least one of: pixel, integrity, sharpness, color; generating the parameters includes at least one of: shutter speed, aperture, sensitivity, exposure value, whether flash is on.
In a fourth aspect, there is provided an image annotation apparatus comprising:
the acquisition module is used for acquiring an image to be marked;
the input module is used for inputting the image to be annotated into an annotation task network of an annotation model so as to label the image to be annotated by the annotation task network; wherein, the labeling model is obtained by training the training device of any one of the labeling models.
In a fifth aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the method of any one of the above when executing the computer program.
A sixth aspect provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any of the above.
The positive progress effects of the invention are as follows: in the embodiment of the invention, a small amount of image samples containing first labeling information can be obtained, the image samples not containing the first labeling information are combined to serve as the training sample set of the labeling model, the imaging parameters of the image samples in the training sample set are not identical, and the labeling model obtained based on the training of the training sample set can label images to be labeled with different imaging parameters, such as images difficult to label manually, so that the efficiency of image labeling can be improved, the number of the training samples of the task model can be increased, the training samples of the task model can be enriched and expanded, and the training precision of the task model can be improved. And the task marking network obtained by the counterstudy is higher in accuracy.
Drawings
FIG. 1 is a flow chart of a model training method provided in an exemplary embodiment of the invention;
fig. 2 is a schematic diagram of a network architecture of a label network adopted in a model training method according to an exemplary embodiment of the present invention;
FIG. 3 is a flow chart of another model training method provided by an exemplary embodiment of the present invention;
FIG. 4 is a flowchart of an image annotation method according to an exemplary embodiment of the present invention;
FIG. 5a is a schematic diagram of an image sample including brain parenchyma according to an exemplary embodiment of the present invention;
FIG. 5b is a schematic diagram of an output result of an image segmentation model obtained by training using a model training method in the prior art;
FIG. 5c is a schematic diagram of an output result of an image segmentation model trained by using a model training method according to an exemplary embodiment of the present invention;
FIG. 6 is a block diagram of a model training apparatus according to an exemplary embodiment of the present invention;
FIG. 7 is a block diagram of an image annotation apparatus according to an exemplary embodiment of the present invention;
fig. 8 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present invention.
Detailed Description
The invention is further illustrated by the following examples, which are not intended to limit the scope of the invention.
Fig. 1 is a flowchart of a model training method according to an exemplary embodiment of the present invention, where an annotation model obtained by training may perform task annotation on images with different imaging parameters, and the images after the task annotation may be used as a training sample for training a task model, so as to expand and enrich the training sample of the task model. Referring to fig. 1, the model training method includes the steps of:
The imaging parameters of the image samples are not completely the same, a part of the image samples in the image samples contain first labeling information, other image samples except the part of the image samples in the image samples do not contain the first labeling information, and each image sample also comprises second labeling information.
The first annotation information is typically a manual annotation, which is related to the annotation task for the target object, i.e. to the intended implementation functionality of the task model. For example, suppose that the expected implementation function of the task model is image segmentation, the annotation task is to obtain a training sample of the image segmentation model, that is, the annotation task is image segmentation annotation, and the first annotation information is an interested area of a target object in the image sample; and assuming that the expected implementation function of the task model is image recognition, the labeling task is to obtain a training sample of the image recognition model, namely the labeling task is image recognition labeling, and the first labeling information is a recognition result of a target object in the image sample.
The second labeling information represents whether the image sample contains the first labeling information.
The image sample may be a medical image or a two-dimensional image captured by a camera. When the image sample is a medical image, the target object may be a part or organ of a patient, such as the brain, abdomen, lung, chest, etc. When the image sample is a two-dimensional image, the target object may be any subject, such as a building, a vehicle, a road, a person, or the like.
The imaging parameters include display parameters or generation parameters. When the image sample is a medical image, the display parameters include at least one of: parameters of the medical image such as voxel, integrity, definition, contrast, color, slice thickness and the like can be distinguished; generating the parameters includes at least one of: scanning parameters, modalities, phases, etc. may result in imaging results of the medical image or parameters with different display parameters. When the image sample is a two-dimensional image, the display parameters include at least one of: parameters of the two-dimensional image such as pixels, integrity, definition, color and the like can be distinguished; generating the parameters includes at least one of: shooting parameters such as shutter speed, aperture, ISO (sensitivity), EV (exposure) value, whether or not a flash is turned on, and the like cause imaging results of two-dimensional images or parameters different in display parameters.
Wherein the modality represents a type of medical imaging technology, or a type of imaging device. For example, the modality may be a CT (computed tomography) modality, an MRI (magnetic resonance imaging) modality, a PET (positron emission tomography) modality, or the like. The phase represents different imaging periods during the scan. For example, the CT technique may include phases such as a flat scan phase, an arterial phase, a venous phase, a delayed phase, etc.; phases of T1, T2, DWI, multi-phase enhanced images, etc. can be included in the MRI technology, and images of different phases are obtained based on different scanning sequences. The integrity characterizes the degree of absence of other parts of the image except the target object.
And 102, inputting the image sample into an annotation model for each image sample, generating an annotation task result of the image sample by an annotation task network of the annotation model, and predicting whether the image sample contains first annotation information or not by a discriminator of the annotation model according to the annotation task result.
Referring to fig. 2, the annotation model includes an annotation task network and a discriminator, the network architecture of the annotation task network can adopt but is not limited to FCN, v-net, vb-net, deep lab, etc., and the network architecture of the discriminator can adopt but is not limited to VGG16, ResNet, inclusion, etc.
During model training, the annotation task network executes an annotation task on an input image sample, outputs a generated annotation task result to the discriminator, and the discriminator predicts whether the image sample contains first annotation information according to the annotation task result.
The annotation task result generated by the annotation task network is related to the annotation task, that is, related to the first annotation information, for example:
when the annotation task is image segmentation annotation, that is, the first annotation information is segmentation annotation information (region of interest) of the target object, the annotation task network is an image segmentation network, the image segmentation network performs image segmentation processing on an input image sample, and a generated annotation task result is segmentation annotation of the target object.
When the labeling task is image classification labeling, namely the first labeling information is the classification information of the target object, the labeling task network is an image classification network, the image classification network performs image classification processing on the input image sample, and the generated labeling task result is the classification labeling of the target object.
When the annotation task is image detection annotation, namely the first annotation information is the detection result information of the target object, the annotation task network adopts an image detection network, the image detection network performs image detection processing on the input image sample, and the generated annotation task result is the detection result annotation of the target object.
When the labeling task is the object identification label, that is, the first labeling information is the identification result information of the target object, the labeling task network is the object identification network, the object identification network executes the image identification processing on the input image sample, and the generated labeling task result is the identification result label of the target object.
It should be noted that the above labeling task is only an example, and the labeling task may also be another labeling task when in actual use, which is not described again.
In one embodiment, the arbiter comprises a classification network; the classification network is used for classifying the labeling task results, and the classification results represent whether the image samples contain the first labeling information. For example, the classification result of the classification network includes two types, the a-type characteristic image sample includes first labeling information, the B-type characteristic image sample does not include the first labeling information, and if the classification network determines that the image sample includes the first labeling information according to the labeling task result output by the labeling task network, the classification network divides the image sample into the a type; and if the classification network judges that the image sample does not contain the first annotation information according to the annotation task result output by the annotation task network, the image sample is classified into B types.
And 103, calculating the loss error of the labeling task network according to the first labeling information and the output result of the labeling task network, and calculating the loss error of the discriminator according to the second labeling information and the output result of the discriminator.
The loss error of the labeling task network is calculated according to the loss function of the labeling task network, the loss error of the discriminator is calculated according to the loss function of the discriminator, and the loss function of the labeling task network and the loss function meter of the discriminator can be set according to the actual application scene or the labeling task.
And for each image sample containing the first annotation information, inputting the image sample into an annotation model for supervised learning, calculating the loss error of the annotation task network according to the loss function of the annotation task network, and enabling the annotation task network to correctly execute the annotation task on the image by minimizing the loss error of the annotation task network.
And outputting the labeling task result generated by the labeling task network to the discriminator for each image sample, obtaining the judgment result of the discriminator on the labeling task result, calculating the loss error of the discriminator according to the loss function of the discriminator, and optimizing the labeling task network by minimizing the loss error of the discriminator.
In the embodiment of the invention, the conception of a countermeasure network is adopted for model training, all image samples are judged to contain first labeling information by a labeling task network hope discriminator, namely the hope discriminator cannot distinguish whether a labeling task result output by a labeling task network is artificially labeled or generated by the labeling task network, the discriminator hopes to well judge whether the labeling task result is generated by the labeling task network, and the labeling task network and the discriminator optimize learning in the process of mutual game and can learn the difference and the commonality among different image samples. Based on the concept, the countermeasure loss of the whole annotation model can also be formed by the annotation task network and the loss function of the discriminator. Labeling the overall objective function of a model
May be represented, but is not limited to, as follows:
wherein the content of the first and second substances,
representing a loss function of the annotation task network;
a loss function representing a discriminator; (seg)*，D*) Respectively representing the optimized results.
The network parameters of the labeling task network are updated through the maximized objective function, so that the performance of the labeling task network is improved, the network parameters of the labeling task network are updated/finely adjusted through the minimized objective function, and the labeling task is better adapted.
And 104, adjusting network parameters of the labeling task network according to the loss error of the labeling task network and the loss error of the discriminator until an iteration stop condition is met.
The iteration stopping condition may be that the number of iterations is not less than a threshold of times, or both the loss error of the annotation task network determined by the iteration and the loss error of the discriminator are smaller than respective error thresholds, or the integrated loss error is smaller than the integrated error threshold, or the discriminator identifies the annotation task result generated based on the image sample that does not include the first identification information as that the image sample includes the first identification information.
After the training of the labeling model is completed, the labeling task network can be used for implementing the labeling task on the images with different imaging parameters, so that the images with higher labeling difficulty can be conveniently labeled, the number of training samples of the task model is increased, and the training samples of the task model are enriched.
In the embodiment of the invention, a small amount of image samples containing first labeling information can be obtained, the image samples not containing the first labeling information are combined to serve as the training sample set of the labeling model, the imaging parameters of the image samples in the training sample set are not identical, and the labeling model obtained based on the training of the training sample set can label images to be labeled with different imaging parameters, such as images difficult to label manually, so that the efficiency of image labeling can be improved, the number of the training samples of the task model can be increased, the training samples of the task model can be enriched and expanded, and the training precision of the task model can be improved. And the task marking network obtained by the counterstudy is higher in accuracy.
In one embodiment, in the model training process, by means of a domain adaptation thought, the annotation task network maps each image sample to the same feature space as much as possible, finds a measurement criterion, and executes an annotation task on the mapped image of the feature space, so that the annotation task network generates annotation task results for different image samples as consistent as possible, and accurate and effective migration of annotation is realized.
Fig. 3 is a flowchart of another model training method according to an exemplary embodiment of the present invention, which further illustrates a model training process by taking an example that a target object is brain parenchyma, an image sample is a medical image, and an annotation task is image segmentation and annotation.
The multiple image samples may be medical images scanned with different generation parameters, and the display parameters of the medical images are not identical.
Some image samples in the plurality of image samples are marked with brain parenchyma area information (first marking information), other image samples except for the partial image samples in the plurality of image samples are not marked with marking information of brain parenchyma, and each image sample is also marked with second marking information for representing whether the image sample contains the brain parenchyma area information.
The image segmentation network is used as a labeling task network of a labeling model.
And 303, calculating the loss error of the labeling task network according to the first labeling information and the output result of the labeling task network, and calculating the loss error of the discriminator according to the second labeling information and the output result of the discriminator.
In step 303, the loss error of the annotation task network is calculated according to the brain parenchyma region information labeled in the image sample and the segmentation labeling result of the brain parenchyma output by the annotation task network.
In one embodiment, the loss function of the annotation task network may be expressed, but is not limited to, as follows:
wherein (X, Y, Z) represents the dimensions of the image sample; c is 0, the pixel point is not in the marked brain parenchyma area, and c is 1, the pixel point belongs to the brain parenchyma; y isicRepresenting the true probability that the ith pixel belongs to class c, picRepresenting the probability of segmenting the ith pixel of the net output into class c.
It should be noted that the above-described penalty function is applied to a binary scene, and is also applied to a multi-classification scene by increasing the number of elements to which c belongs. For example, c e { a, b, c } may be set for a tri-classification scenario and c e { a, b, c, d } may be set for a tetra-classification scenario.
In one embodiment, the penalty function of the arbiter can be expressed, but is not limited to, as follows:
wherein d isiA binary label representing the ith image sample, which is used for representing whether the image sample contains the first labeling information; seg (X)s) Representing an output result of the image segmentation network when the image sample containing the second annotation information is input; seg (X)t) And an output result of the image segmentation network when the image sample not including the second annotation information is input.
In one embodiment, an overall objective function of the annotation model can also be constructed
May be represented, but is not limited to, as follows:
wherein the content of the first and second substances,
representing a loss function of the annotation task network;
a loss function representing a discriminator; (seg)*，D*) Respectively representing the optimized results.
The network parameters of the image segmentation network are updated through the maximized objective function, so that the performance of the image segmentation network is improved, the network parameters of the image segmentation network are updated/fine-tuned through the minimized objective function, and the annotation task is better adapted.
And step 304, adjusting the network parameters of the labeling model according to the loss error of the labeling task network and the loss error of the discriminator until the iteration stop condition is met.
The iteration stopping condition may be that the number of iterations is not less than a number threshold, or both the loss error of the segmented network determined by the current iteration and the loss error of the discriminator are smaller than respective error thresholds, or the integrated loss error is smaller than an integrated error threshold, or the discriminator identifies the brain parenchymal region generated by the image segmented network as an artificial identifier.
The trained image segmentation network has the capability of labeling images to be labeled with different imaging parameters, for example, the T1 image and the T2 image have different program parameters, the T1 image is bright for white matter, gray matter is dark, and cerebrospinal fluid is black, and the gray matter and cerebrospinal fluid of the T1 image are bright and white matter is dark, and the image segmentation labeling can be performed by using the image segmentation network obtained by the embodiment of the invention. Therefore, the image with difficult manual annotation can be annotated through the image segmentation network, the image annotation efficiency is improved, and the labor cost is reduced.
Fig. 4 is a flowchart of an image annotation method according to an exemplary embodiment of the present invention, where the image annotation method includes the following steps:
The image to be annotated can be an image of any type and any source, and the type and the acquisition mode of the image to be annotated are not particularly limited in the embodiment of the invention.
And 402, inputting the image to be annotated into an annotation task network of the annotation model, so that the image to be annotated is annotated by the annotation task network.
The annotation model is obtained by training the training method of the annotation model provided in any one of the above embodiments.
It will be appreciated that for different annotation tasks, different annotation task networks are required, and thus the annotation task network of step 402 is the annotation task network that matches the annotation task. For example, when the annotation task is an image segmentation task, acquiring an image segmentation network as an annotation task network; and when the annotation task is the image recognition task, acquiring the image recognition network as the annotation task network.
Because the task labeling network is obtained through counterstudy in the embodiment of the invention, the accuracy is higher. The following description will take an example in which the annotation task network is an image segmentation network and brain parenchyma is segmented. Fig. 5a is a schematic diagram of an image sample including brain parenchyma, where a brain parenchyma region of the image sample is complete, and an eye portion of the image sample is absent, that is, the integrity of the eye portion is not ideal, a sample set including the image sample is trained by using a model training method in the prior art to obtain an image segmentation model a, and a model training method in an embodiment of the present invention is used to train to obtain an image segmentation model B. And respectively inputting the images to be annotated with better integrity (different from the integrity of the image sample, namely different from the imaging parameters) into the image segmentation model A and the image segmentation model B. Fig. 5b shows the result output by the image segmentation model a, and it can be seen from the figure that the color of the brain parenchyma region and the color of the eye region are almost consistent (the gray scale is consistent), that is, the segmentation model a incorrectly identifies the eye portion as the brain parenchyma. Fig. 5c shows the result output by the image segmentation model B, where the color of the brain parenchyma region is different from the color of the eye region, and the color of the eye region is obviously darker than the color of the brain parenchyma region, so that the image segmentation model B can more accurately segment the brain parenchyma region, and the misrecognition rate is very low.
As can also be seen from fig. 5a to 5b, by using the model training method according to the embodiment of the present invention, the requirement on the image quality of the training sample is not high, and an image with an unsatisfactory integrity is used as the training sample, so that a highly accurate labeling task model can be obtained.
Corresponding to the embodiments of the model training method and the image labeling method, the invention also provides embodiments of a model training device and an image labeling device.
Fig. 6 is a schematic block diagram of a model training apparatus according to an exemplary embodiment of the present invention, where the model training apparatus includes:
an obtaining module 61, configured to obtain a plurality of image samples including a target object, where imaging parameters of the image samples are not completely the same, a partial image sample of the image samples includes first annotation information, and other image samples except the partial image sample of the image samples do not include the first annotation information, and each image sample further includes second annotation information; the first labeling information is related to an labeling task for the target object, and the second labeling information represents whether the image sample contains the first labeling information;
an input module 62, configured to, for each image sample, input the image sample into an annotation model, so as to generate an annotation task result of the image sample by an annotation task network of the annotation model, and predict, by a discriminator of the annotation model, whether the image sample includes first annotation information according to the annotation task result;
a calculating module 63, configured to calculate a loss error of the tagged task network according to the first tagged information and the output result of the tagged task network, and calculate a loss error of the discriminator according to the second tagged information and the output result of the discriminator;
and the adjusting module 64 is configured to adjust the network parameters of the labeled model according to the loss error of the labeled task network and the loss error of the discriminator until an iteration stop condition is met.
Optionally, the annotation task network comprises an image segmentation network; the labeling task result comprises a segmentation labeling result of the target object;
or, the labeling task network comprises a classification network; the labeling task result comprises a classification labeling result of the target object;
or the labeling task network comprises an object detection network; the labeling task result comprises a detection labeling result of the target object;
or the labeling task network comprises an object identification network; and the labeling task result comprises an identification labeling result of the target object.
Optionally, the discriminator comprises a classification network; the classification network is used for classifying the labeling task result, and the classification result represents whether the image sample contains the first labeling information.
Optionally, the target object is brain parenchyma; the labeling task result is a segmentation labeling result of the brain parenchyma;
the input module is specifically configured to:
inputting an image sample containing brain parenchyma into an annotation model, and carrying out segmentation processing on the image sample by an annotation task network of the annotation model to generate a segmentation and annotation result of the brain parenchyma.
Optionally, the first labeling information is brain parenchymal region information; when calculating the loss error of the labeling task network according to the first labeling information and the output result of the labeling task network, the calculating module is used for:
and calculating the loss error of the labeling task network according to the brain parenchyma region information and the segmentation labeling result of the brain parenchyma.
Optionally, the imaging parameters comprise display parameters or generation parameters;
when the image sample is a medical image, the display parameters include at least one of: voxel, integrity, sharpness, contrast, color, slice thickness; generating the parameters includes at least one of: scanning parameters, modes, phases;
when the image sample is a two-dimensional image, the display parameters include at least one of: pixel, integrity, sharpness, color; generating the parameters includes at least one of: shutter speed, aperture, sensitivity, exposure value, whether flash is on.
Fig. 7 is a schematic block diagram of an image annotation apparatus according to an exemplary embodiment of the present invention, where the image annotation apparatus includes:
an obtaining module 71, configured to obtain an image to be annotated;
the input module 72 is configured to input the image to be annotated into an annotation task network of an annotation model, so that the annotation task network annotates the image to be annotated; wherein, the labeling model is obtained by training the training device of any one of the labeling models.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the invention. One of ordinary skill in the art can understand and implement it without inventive effort.
Fig. 8 is a schematic diagram of an electronic device according to an exemplary embodiment of the present invention, and illustrates a block diagram of an exemplary electronic device 80 suitable for implementing embodiments of the present invention. The electronic device 80 shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiment of the present invention.
As shown in fig. 8, the electronic device 80 may be embodied in the form of a general purpose computing device, which may be, for example, a server device. The components of the electronic device 80 may include, but are not limited to: the at least one processor 81, the at least one memory 82, and a bus 83 connecting the various system components including the memory 82 and the processor 81.
The bus 83 includes a data bus, an address bus, and a control bus.
The memory 82 may include volatile memory, such as Random Access Memory (RAM)821 and/or cache memory 822, and may further include Read Only Memory (ROM) 823.
The processor 81 executes various functional applications and data processing, such as the methods provided by any of the above embodiments, by running a computer program stored in the memory 82.
The electronic device 80 may also communicate with one or more external devices 84 (e.g., keyboard, pointing device, etc.). Such communication may be through input/output (I/O) interfaces 85. Also, the model-generating electronic device 80 may also communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet) via the network adapter 86. As shown, the network adapter 86 communicates with the other modules of the model-generating electronic device 80 via a bus 83. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the model-generating electronic device 80, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.
It should be noted that although in the above detailed description several units/modules or sub-units/modules of the electronic device are mentioned, such a division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the units/modules described above may be embodied in one unit/module according to embodiments of the invention. Conversely, the features and functions of one unit/module described above may be further divided into embodiments by a plurality of units/modules.
Embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the method provided in any of the above embodiments.
More specific examples, among others, that the readable storage medium may employ may include, but are not limited to: a portable disk, a hard disk, random access memory, read only memory, erasable programmable read only memory, optical storage device, magnetic storage device, or any suitable combination of the foregoing.
In a possible implementation manner, the embodiment of the present invention may also be implemented in a form of a program product, which includes program code for causing a terminal device to execute a method implementing any of the above-mentioned embodiments when the program product runs on the terminal device.
Where program code for carrying out the invention is written in any combination of one or more programming languages, the program code may be executed entirely on the user device, partly on the user device, as a stand-alone software package, partly on the user device and partly on a remote device or entirely on the remote device.
While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that this is by way of example only, and that the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the spirit and scope of the invention, and these changes and modifications are within the scope of the invention.
Claims (10)
1. A method of model training, comprising:
acquiring a plurality of image samples containing a target object, wherein imaging parameters of the image samples are not identical, a part of the image samples in the image samples contain first labeling information, other image samples except the part of the image samples in the image samples do not contain the first labeling information, and each image sample also comprises second labeling information; the first labeling information is related to an labeling task for the target object, and the second labeling information represents whether the image sample contains the first labeling information;
for each image sample, inputting the image sample into an annotation model, generating an annotation task result of the image sample by an annotation task network of the annotation model, and predicting whether the image sample contains first annotation information or not by a discriminator of the annotation model according to the annotation task result;
calculating the loss error of the labeling task network according to the first labeling information and the output result of the labeling task network, and calculating the loss error of the discriminator according to the second labeling information and the output result of the discriminator;
and adjusting the network parameters of the labeling model according to the loss error of the labeling task network and the loss error of the discriminator until an iteration stop condition is met.
2. The model training method of claim 1, wherein the annotation task network comprises an image segmentation network; the labeling task result comprises a segmentation labeling result of the target object;
or, the labeling task network comprises a classification network; the labeling task result comprises a classification labeling result of the target object;
or the labeling task network comprises an object detection network; the labeling task result comprises a detection labeling result of the target object;
or the labeling task network comprises an object identification network; and the labeling task result comprises an identification labeling result of the target object.
3. The model training method of claim 1, wherein the discriminator comprises a classification network; the classification network is used for classifying the labeling task result, and the classification result represents whether the image sample contains the first labeling information.
4. The model training method according to claim 1, wherein the target object is brain parenchyma; the labeling task result is a segmentation labeling result of the brain parenchyma;
inputting the image sample into an annotation model, and generating an annotation task result of the image sample by an annotation task network of the annotation model, wherein the annotation task result comprises:
inputting an image sample containing brain parenchyma into an annotation model, and carrying out segmentation processing on the image sample by an annotation task network of the annotation model to generate a segmentation and annotation result of the brain parenchyma.
5. The model training method according to claim 4, wherein the first labeling information is brain parenchymal region information; calculating the loss error of the labeling task network according to the first labeling information and the output result of the labeling task network, and the method comprises the following steps:
and calculating the loss error of the labeling task network according to the brain parenchyma region information and the segmentation labeling result of the brain parenchyma.
6. The method for training an annotation model of claim 1, wherein the imaging parameters comprise display parameters or generation parameters;
when the image sample is a medical image, the display parameters include at least one of: voxel, integrity, sharpness, contrast, color, slice thickness; generating the parameters includes at least one of: scanning parameters, modes, phases;
when the image sample is a two-dimensional image, the display parameters include at least one of: pixel, integrity, sharpness, color; generating the parameters includes at least one of: shutter speed, aperture, sensitivity, exposure value, whether flash is on.
7. An image annotation method, comprising:
acquiring an image to be marked;
inputting the image to be annotated into an annotation task network of an annotation model, and annotating the image to be annotated by the annotation task network; wherein the labeling model is obtained by training the training method of the labeling model according to any one of claims 1 to 4.
8. A model training apparatus, comprising:
the device comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring a plurality of image samples containing a target object, imaging parameters of the image samples are not identical, a partial image sample in the image samples contains first labeling information, other image samples except the partial image sample in the image samples do not contain the first labeling information, and each image sample also comprises second labeling information; the first labeling information is related to an labeling task for the target object, and the second labeling information represents whether the image sample contains the first labeling information;
the input module is used for inputting the image samples into an annotation model for each image sample, so that an annotation task result of the image sample is generated by an annotation task network of the annotation model, and a discriminator of the annotation model predicts whether the image sample contains first annotation information according to the annotation task result;
the calculation module is used for calculating the loss error of the labeling task network according to the first labeling information and the output result of the labeling task network, and calculating the loss error of the discriminator according to the second labeling information and the output result of the discriminator;
and the adjusting module is used for adjusting the network parameters of the labeling model according to the loss error of the labeling task network and the loss error of the discriminator until an iteration stop condition is met.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111653639.6A CN114266896A (en) | 2021-12-30 | 2021-12-30 | Image labeling method, model training method and device, electronic equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111653639.6A CN114266896A (en) | 2021-12-30 | 2021-12-30 | Image labeling method, model training method and device, electronic equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114266896A true CN114266896A (en) | 2022-04-01 |
Family
ID=80831845
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111653639.6A Pending CN114266896A (en) | 2021-12-30 | 2021-12-30 | Image labeling method, model training method and device, electronic equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114266896A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115035192A (en) * | 2022-06-21 | 2022-09-09 | 北京远舢智能科技有限公司 | Method and device for determining positions of tobacco leaf distributing vehicle and conveying belt |
| CN115797266A (en) * | 2022-11-09 | 2023-03-14 | 北京百度网讯科技有限公司 | Training method of medical image processing model, image processing method and device |
-
2021
- 2021-12-30 CN CN202111653639.6A patent/CN114266896A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115035192A (en) * | 2022-06-21 | 2022-09-09 | 北京远舢智能科技有限公司 | Method and device for determining positions of tobacco leaf distributing vehicle and conveying belt |
| CN115797266A (en) * | 2022-11-09 | 2023-03-14 | 北京百度网讯科技有限公司 | Training method of medical image processing model, image processing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10482600B2 (en) | Cross-domain image analysis and cross-domain image synthesis using deep image-to-image networks and adversarial networks | |
| US11593943B2 (en) | RECIST assessment of tumour progression | |
| US10910099B2 (en) | Segmentation, landmark detection and view classification using multi-task learning | |
| US8958614B2 (en) | Image-based detection using hierarchical learning | |
| CN109102490B (en) | Automatic image registration quality assessment | |
| US9280819B2 (en) | Image segmentation techniques | |
| WO2021140426A1 (en) | Uncertainty guided semi-supervised neural network training for image classification | |
| US10679325B2 (en) | Machine learning model for automatic image registration quality assessment and correction | |
| del Toro et al. | VISCERAL–VISual Concept Extraction challenge in RAdioLogy: ISBI 2014 challenge organization | |
| CN114266896A (en) | Image labeling method, model training method and device, electronic equipment and medium | |
| US10878564B2 (en) | Systems and methods for processing 3D anatomical volumes based on localization of 2D slices thereof | |
| CN113362272A (en) | Medical image segmentation with uncertainty estimation | |
| Yang et al. | A multiorgan segmentation model for CT volumes via full convolution-deconvolution network | |
| US20220301156A1 (en) | Method and system for annotation efficient learning for medical image analysis | |
| CN114596440A (en) | Semantic segmentation model generation method and device, electronic equipment and storage medium | |
| US11682135B2 (en) | Systems and methods for detecting and correcting orientation of a medical image | |
| CN111127432B (en) | Medical image detection method, device, equipment and storage medium | |
| CN111724371A (en) | Data processing method and device and electronic equipment | |
| WO2023104464A1 (en) | Selecting training data for annotation | |
| CN113256651B (en) | Model training method and device, and image segmentation method and device | |
| CN112634255B (en) | Method and device for establishing brain focus detection model and computer equipment | |
| CN113177923A (en) | Medical image content identification method, electronic device and storage medium | |
| Lai et al. | Review on segmentation of computer-aided skeletal maturity assessment | |
| Ding et al. | Towards efficient human-machine collaboration: real-time correction effort prediction for ultrasound data acquisition | |
| CN112884060B (en) | Image labeling method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |