WO2023029348A1 - Procédé d'étiquetage d'instance d'image basé sur l'intelligence artificielle, et dispositif associé - Google Patents

Procédé d'étiquetage d'instance d'image basé sur l'intelligence artificielle, et dispositif associé Download PDF

Info

Publication number
WO2023029348A1
WO2023029348A1 PCT/CN2022/071328 CN2022071328W WO2023029348A1 WO 2023029348 A1 WO2023029348 A1 WO 2023029348A1 CN 2022071328 W CN2022071328 W CN 2022071328W WO 2023029348 A1 WO2023029348 A1 WO 2023029348A1
Authority
WO
WIPO (PCT)
Prior art keywords
instance
label
score
detection frame
image
Prior art date
Application number
PCT/CN2022/071328
Other languages
English (en)
Chinese (zh)
Inventor
王俊
高鹏
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2023029348A1 publication Critical patent/WO2023029348A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS

Definitions

  • the present application relates to the technical field of artificial intelligence, in particular to an artificial intelligence-based image instance labeling method, device, electronic equipment, and storage medium.
  • the training data set is a data set with rich label information, and collecting and labeling such a training data set usually requires a huge human cost.
  • image instance segmentation has a higher degree of difficulty, and a large amount of labeled training data is required to truly realize the instance segmentation function.
  • the number of available labeled samples is often insufficient relative to the training scale, or the cost of obtaining samples is too high.
  • annotators with relevant professional knowledge such as doctors
  • the annotation cost of annotators is too high, or the image annotation or judgment cycle is too long, these problems may cause the instance segmentation model to fail. Unable to train effectively.
  • an artificial intelligence-based image instance annotation method which can reduce the number of instances in manually annotated images and improve the accuracy of instances in images annotated by models.
  • the first aspect of the present application provides an artificial intelligence-based image instance labeling method, the method comprising:
  • An instance label of the target image is obtained based on the first label and the second label.
  • the second aspect of the present application provides an artificial intelligence-based image instance labeling device, the device comprising:
  • An instance recognition module used to call a preset instance segmentation model to identify the amount of information of each instance in the target image
  • An instance acquiring module configured to acquire a first information amount higher than a preset information amount threshold and a first instance corresponding to the first information amount from the target image, except for the first instance in the target image
  • the instance of is the second instance
  • a first labeling module configured to manually label the first label of the first instance in the target image
  • a second labeling module configured to pseudo-label the second label of the second instance in the target image based on a semi-supervised learning method
  • a label determining module configured to obtain an instance label of the target image based on the first label and the second label.
  • a third aspect of the present application provides an electronic device, the electronic device includes a processor, and the processor is configured to implement the artificial intelligence-based image instance labeling method when executing a computer program stored in a memory.
  • An instance label of the target image is obtained based on the first label and the second label.
  • the fourth aspect of the present application provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the artificial intelligence-based image instance labeling method is implemented:
  • An instance label of the target image is obtained based on the first label and the second label.
  • the artificial intelligence-based image instance labeling method, device, electronic equipment, and storage medium described in this application identify the amount of information of each instance in the target image by calling the preset instance segmentation model, so as to extract the information from the target image Obtain the first information amount higher than the preset information amount threshold, the first instance corresponding to the first information amount, and the second instance other than the first instance, and then manually mark the first label of the first instance, and the manually marked instance
  • the accuracy of the label is high. Since the second instance has low information content, it is easy to be recognized and labeled by the model.
  • the second label of the second instance is pseudo-labeled based on the semi-supervised learning method, and the labeling efficiency is high.
  • the application can be applied in the field of digital medical treatment, and the examples in medical images are marked. This application only needs to manually label a small number of instances in a target image, instead of labeling all instances in the entire target image, so as to obtain instance labels with high accuracy while reducing the workload of instance labeling.
  • FIG. 1 is a flow chart of an artificial intelligence-based image instance labeling method provided in Embodiment 1 of the present application.
  • FIG. 2 is a schematic diagram of a target image and two corresponding perturbed images provided by an embodiment of the present application.
  • Fig. 3 is a schematic diagram of a first instance of a target image provided by an embodiment of the present application.
  • Fig. 4 is a schematic diagram of a second example of the target image provided by the embodiment of the present application.
  • FIG. 5 is a structural diagram of an artificial intelligence-based image instance tagging device provided in Embodiment 2 of the present application.
  • FIG. 6 is a schematic structural diagram of an electronic device provided in Embodiment 3 of the present application.
  • the artificial intelligence-based image instance tagging method provided in the embodiment of the present application is executed by electronic equipment, and accordingly, the artificial intelligence-based image instance tagging apparatus runs in the electronic device.
  • instances in an image may be marked based on artificial intelligence technology.
  • artificial intelligence is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. .
  • Artificial intelligence basic technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation/interaction systems, and mechatronics.
  • Artificial intelligence software technology mainly includes computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning/deep learning.
  • FIG. 1 is a flow chart of an artificial intelligence-based image instance labeling method provided in Embodiment 1 of the present application.
  • the artificial intelligence-based image instance labeling method specifically includes the following steps. According to different requirements, the order of the steps in the flow chart can be changed, and some can be omitted.
  • S11 call the preset instance segmentation model to identify the amount of information of each instance in the target image.
  • the preset instance segmentation model may be a pre-trained machine learning model, which is used to identify instances in the image, so as to obtain the amount of information of the instances.
  • Instances refer to target objects in target images, e.g., pedestrians, cars, bicycles, buildings, etc.
  • the target image refers to the image that needs to be labeled with instances.
  • a target image can contain multiple instances, for example, it can contain dozens of instances. Different types of instances have different recognition difficulties. Therefore, it is necessary to consider whether it is necessary to All instances in the target image are manually annotated. Although all instances in the target image can be manually labeled, the accuracy of manual labeling is higher, but the cost of manual labeling is relatively high, and the efficiency is low. In fact, the number of images completed by manual labeling is limited. of.
  • by identifying the amount of information of the instances in the target image it is determined which instances in the target image are to be marked manually and which entities are to be marked by the instance tagging model according to the amount of information.
  • the artificial intelligence-based image instance labeling method can be applied to a medical scene.
  • the target image is a medical image
  • the instances in the target image are multiple organs.
  • Medical images refer to internal tissues obtained non-invasively for medical treatment or medical research, such as images of the stomach, abdomen, heart, knees, brain, such as computed tomography (CT), magnetic resonance Imaging (Magnetic Resonance Imaging, MRI), ultrasound (ultrasonic, US), X-ray images, EEG, and optical photography images generated by medical instruments.
  • the calling the preset instance segmentation model to identify the amount of information of each instance in the target image includes:
  • the information amount of the corresponding instance is calculated according to the class label score and the corresponding detection frame score and the contour mask score.
  • image enhancement can be achieved through image transformation.
  • the purpose of image enhancement is to increase the amount of image data and enrich the diversity of images, thereby improving the generalization ability of the model.
  • image disturbance may be introduced in a transformation manner, for example, adding noise (Noise) to the image to introduce image disturbance.
  • Image noise is the signal that is disturbed by random signals during image acquisition or transmission, which hinders people's understanding and analysis of images. The introduction of noise increases the difficulty of model identification.
  • the top image in Figure 2 is the target image
  • the middle image is Gaussian noise added to the target image
  • the bottom image is the target image Added salt and pepper noise (pepper noise) on the basis of .
  • the image obtained by adding Gaussian noise on the target image is used as the first perturbed image
  • the image obtained by adding salt and pepper noise on the target image is used as the second perturbed image.
  • Input the image (for example, the target image, the first disturbance image, the second disturbance image) into the instance segmentation model, and the instance segmentation model can output the category label of the instance belonging to a certain category in the image, and use the detection frame to frame the instance in the image
  • the position of , and the instance contour mask of the instance can be trained on the basis of the Faster R-CNN model, and the specific training process will not be described in detail. Investigate the consistency of the model's prediction of the target in the sample. If the prediction result changes little before and after the transformation, it means that the target is easier to predict and has less information. If there is a large difference in the prediction result of the target before and after the transformation, then This local target is the target that the model is more likely to confuse, and it should be actively selected for priority labeling.
  • the class label, instance detection frame and instance contour mask After invoking the preset instance segmentation model to identify the class label, instance detection frame and instance contour mask of each instance in the target image, the first perturbed image and the second perturbed image, the class label, instance detection frame and The instance contour mask is calculated to obtain the class label score, the detection frame score and the contour mask score of the instance, so that the information amount of the corresponding instance is calculated according to the class label score and the corresponding detection frame score and the contour mask score, Further, the first instance and the second instance in the target image are determined according to the amount of information of the instances.
  • the calculating the category label score of each instance based on the first category label, the second category label and the third category label includes:
  • the mean value is used as the category label score of the corresponding instance.
  • the category label score is used to evaluate whether the prediction of the instance segmentation model on the perturbed first and second perturbed images is consistent with the prediction on the target image.
  • the class probability predicted by the instance segmentation model is 0.9
  • the class probability predicted by the instance segmentation model is 0.9
  • the class predicted by the instance segmentation model is A probability of 0.89 indicates that the instance segmentation model has a high prediction consistency for this instance.
  • the class probability predicted by the instance segmentation model is 0.9, in the first perturbed image, the class probability predicted by the instance segmentation model is 0.4, and in the second perturbed image, the class predicted by the instance segmentation model is A probability of 0.7 indicates that the prediction consistency of the instance segmentation model for this instance is low.
  • the calculating the detection frame score of each instance based on the detection frame of the first instance, the detection frame of the second instance, and the detection frame of the third instance includes:
  • the corresponding detection frame score of the instance is calculated based on the first intersection and union ratio and the second intersection and union ratio according to a preset first calculation model.
  • the detection box score is used to evaluate whether the prediction of the instance segmentation model on the perturbed first and second perturbed images is consistent with the prediction on the target image.
  • intersection-over-union ratio represents the degree of overlap between two instance detection frames.
  • the smaller the intersection and union ratio IOU the smaller the overlapping area and the smaller the overlapping degree between the two instance detection frames.
  • the greater the IOU the more similar the prediction of the instance segmentation model is to the target image and the perturbed image corresponding to the IOU, that is, the higher the prediction consistency.
  • the smaller the IOU the less similar the prediction of the instance segmentation model is to the target image and the perturbed image corresponding to the IOU, that is, the lower the prediction consistency.
  • instance L2 is an instance with high information content.
  • the calculating the contour mask score of each instance based on the contour mask of the first instance, the contour mask of the second instance and the contour mask of the third instance includes:
  • a contour mask score corresponding to the instance is calculated based on the first Jaccard distance and the second Jaccard distance according to a preset second calculation model.
  • the instance contour mask is similar to the instance detection box, and is also used to evaluate whether the prediction of the instance segmentation model for the perturbed first and second perturbed images is consistent with the prediction of the target image.
  • Jaccard distance is used to describe the dissimilarity between two contour masks.
  • the larger the Jaccard distance the less overlapping area and lower similarity between two contour masks.
  • the smaller the Jaccard distance the more overlapping areas and higher similarity between two contour masks.
  • the larger the Jaccard distance the less similar the prediction of the instance segmentation model is to the target image and the disturbance image corresponding to the Jaccard distance, that is, the lower the prediction consistency.
  • the smaller the Jaccard distance the more similar the prediction of the instance segmentation model is to the target image and the perturbed image corresponding to the Jaccard distance, that is, the higher the prediction consistency.
  • the calculation of the information amount of the corresponding instance according to the category label score and the corresponding detection frame score and the contour mask score includes:
  • the final score is determined as the informativeness of the instance.
  • the average of the class label score, detection box score, and contour mask of each instance can also be calculated as the final score of the instance.
  • the final score is used to indicate whether the prediction of the instance segmentation model on the first disturbed image and the second disturbed image is consistent with the prediction on the target image.
  • the lower the final score it indicates that the prediction of the instance segmentation model on the first perturbed image and the second perturbed image is inconsistent with the prediction on the target image.
  • the performance of the first disturbed image and the second disturbed image is more unstable.
  • the higher the final score it indicates that the prediction of the instance segmentation model for the first perturbed image and the second perturbed image is consistent with the prediction of the target image, which means that even after the target image is perturbed, the target image and the perturbed image obtained by the instance segmentation model are The performance of the first perturbed image and the second perturbed image is still very stable.
  • the preset information volume threshold is a preset critical value used to indicate the level of information volume.
  • the instance is regarded as the first instance, and when the information volume of a certain instance is lower than the preset information volume threshold, the instance is regarded as the second instance.
  • the first instance refers to a set of multiple instances with an information amount higher than a preset information amount threshold, such as the area enclosed by an ellipse in the target image as shown in FIG. 3 .
  • the second instance refers to a collection of multiple instances whose information volume is lower than the preset information volume threshold, such as the area framed by an irregular figure in the target image as shown in FIG. 4 .
  • the first instance and the second instance completely constitute the set of instances in the image. That is, a certain instance in the target image is either the first instance or the second instance.
  • the prediction consistency of the instance segmentation model for the target image and the first disturbance image and the second disturbance image is low, so it should be done manually Annotation of instances, such as multiple pedestrians with obvious occlusion.
  • the first instance in the target image can be identified, and the first instance can be manually annotated by an expert with rich annotation experience, thereby improving the accuracy rate of the annotation of the first instance.
  • the instance segmentation of the medical image needs to identify multiple instance individuals in the image, and accurately outline multiple lesion areas for intelligent auxiliary diagnosis. Therefore, the instance labeling difficulty index of the medical image Higher, by manually labeling these high-information (difficult to label) instances, the accuracy will be greatly improved.
  • the second instance is an instance in the target image that is lower than the preset information threshold, the prediction consistency of the instance segmentation model for the target image and the first disturbed image and the second disturbed image is relatively high, so the difficulty of labeling is small. Pseudo-annotation of instances by supervised learning can improve the efficiency of labeling target images.
  • the semi-supervised learning method refers to the instance labeling model obtained through the joint training of the labeled sample set and the unlabeled sample set, and instance labeling of the new unlabeled image through the instance labeling model.
  • the instance label output by the instance labeling model is compared with In terms of instance labels that are artificially labeled, they are called pseudo-labels.
  • the instances in the target image are divided into the first instance and the second instance.
  • the instance label of the first instance is the first label
  • the instance label of the second instance is the second label. Therefore, after obtaining the first label and the second label, the target Labels for all instances in the image have been obtained.
  • the method also includes:
  • the annotated image marked with the instance label may refer to an image used to train the instance label model.
  • the target images marked with the instance labels can be added to the real instance labels.
  • Annotated images are used as a training set, so that the instance labeling model is updated based on the training set.
  • the test set includes test images and the real instance labels of each test image, and the test images in the test set are input into the updated instance labeling model, and the test instance labels of the test images are predicted by the updated instance labeling model.
  • the test instance label is the same as the corresponding real instance label, it indicates that the updated instance annotation model is successfully tested on the test image.
  • the test instance label is not the same as the corresponding real instance label, it indicates that the updated instance labeling model fails to test the test image. Calculate the ratio of the number of successful tests to the number of test images in the test set, and use the ratio as the test accuracy of the instance tagging model, and end the training of the instance tagging model when the test accuracy meets a preset accuracy threshold.
  • the number of images with instance labels is greatly increased by using the target images with instance labels, and the instance labeling model can be updated and trained, thereby improving the performance of the instance labeling model.
  • This application marks a small number of instances in the image instead of all instances. For example, instances with occlusion in some areas in the target image are instances with high information content.
  • instances with occlusion in some areas in the target image are instances with high information content.
  • the other instances of other instances have low information content, no manual labeling is required, and the cost of manual labeling is saved. Since the instances with low information content are easier to identify, semi-supervised learning and labeling can improve the accuracy of instance labeling.
  • the labeling efficiency of the instance is improved.
  • This application only needs to manually label a small number of instances in a target image, instead of labeling all instances in the entire target image, so as to obtain instance labels with high accuracy while reducing the workload of instance labeling.
  • This application is suitable for images with complex layouts and mutual occlusion in different areas. Applying this application to the field of intelligent auxiliary recognition of medical images can simultaneously perform region delineation and quantitative evaluation of different target locations and key organ instances, especially for image regions that may be occluded from each other, this application can perform instance segmentation more effectively.
  • FIG. 5 is a structural diagram of an artificial intelligence-based image instance tagging device provided in Embodiment 2 of the present application.
  • the artificial intelligence-based image instance labeling device 50 may include a plurality of functional modules composed of computer program segments.
  • the computer program of each program segment in the image instance tagging device 50 based on artificial intelligence can be stored in the memory of the electronic device, and executed by at least one processor to execute (see Figure 1 for details) based on artificial intelligence.
  • a feature for image instance annotation may include a plurality of functional modules composed of computer program segments.
  • the artificial intelligence-based image instance tagging device 50 can be divided into multiple functional modules according to the functions it performs.
  • the functional modules may include: an instance identification module 501 , an instance acquisition module 502 , a first labeling module 503 , a second labeling module 504 , a label determination module 505 and a model training module 506 .
  • the module referred to in this application refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, and are stored in a memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.
  • the instance identification module 501 is configured to call a preset instance segmentation model to identify the amount of information of each instance in the target image.
  • the preset instance segmentation model may be a pre-trained machine learning model, which is used to identify instances in the image, so as to obtain the amount of information of the instances.
  • Instances refer to target objects in target images, e.g., pedestrians, cars, bicycles, buildings, etc.
  • the target image refers to the image that needs to be labeled with instances.
  • a target image can contain multiple instances, for example, it can contain dozens of instances. Different types of instances have different recognition difficulties. Therefore, it is necessary to consider whether it is necessary to All instances in the target image are manually annotated. Although all instances in the target image can be manually labeled, the accuracy of manual labeling is higher, but the cost of manual labeling is relatively high, and the efficiency is low. In fact, the number of images completed by manual labeling is limited. of.
  • by identifying the amount of information of the instances in the target image it is determined which instances in the target image are to be marked manually and which entities are to be marked by the instance tagging model according to the amount of information.
  • the artificial intelligence-based image instance labeling method can be applied to a medical scene.
  • the target image is a medical image
  • the instances in the target image are multiple organs.
  • Medical images refer to internal tissues obtained non-invasively for medical treatment or medical research, such as images of the stomach, abdomen, heart, knees, brain, such as computed tomography (CT), magnetic resonance Imaging (Magnetic Resonance Imaging, MRI), ultrasound (ultrasonic, US), X-ray images, EEG, and optical photography images generated by medical instruments.
  • the instance identification module 501 calls a preset instance segmentation model to identify the amount of information of each instance in the target image, including:
  • the information amount of the corresponding instance is calculated according to the class label score and the corresponding detection frame score and the contour mask score.
  • image enhancement can be achieved through image transformation.
  • the purpose of image enhancement is to increase the amount of image data and enrich the diversity of images, thereby improving the generalization ability of the model.
  • image disturbance may be introduced in a transformation manner, for example, adding noise (Noise) to the image to introduce image disturbance.
  • Image noise is the signal that is disturbed by random signals during image acquisition or transmission, which hinders people's understanding and analysis of images. The introduction of noise increases the difficulty of model identification.
  • the top image in Figure 2 is the target image
  • the middle image is Gaussian noise added to the target image
  • the bottom image is the target image Added salt and pepper noise (pepper noise) on the basis of .
  • the image obtained by adding Gaussian noise on the target image is used as the first perturbed image
  • the image obtained by adding salt and pepper noise on the target image is used as the second perturbed image.
  • Input the image (for example, the target image, the first disturbance image, the second disturbance image) into the instance segmentation model, and the instance segmentation model can output the category label of the instance belonging to a certain category in the image, and use the detection frame to frame the instance in the image
  • the position of , and the instance contour mask of the instance can be trained on the basis of the Faster R-CNN model, and the specific training process will not be described in detail. Investigate the consistency of the model's prediction of the target in the sample. If the prediction result changes little before and after the transformation, it means that the target is easier to predict and has less information. If there is a large difference in the prediction result of the target before and after the transformation, then This local target is the target that the model is more likely to confuse, and it should be actively selected for priority labeling.
  • the class label, instance detection frame and instance contour mask After invoking the preset instance segmentation model to identify the class label, instance detection frame and instance contour mask of each instance in the target image, the first perturbed image and the second perturbed image, the class label, instance detection frame and The instance contour mask is calculated to obtain the class label score, the detection frame score and the contour mask score of the instance, so that the information amount of the corresponding instance is calculated according to the class label score and the corresponding detection frame score and the contour mask score, Further, the first instance and the second instance in the target image are determined according to the amount of information of the instances.
  • the calculating the category label score of each instance based on the first category label, the second category label and the third category label includes:
  • the mean value is used as the category label score of the corresponding instance.
  • the category label score is used to evaluate whether the prediction of the instance segmentation model on the perturbed first and second perturbed images is consistent with the prediction on the target image.
  • the class probability predicted by the instance segmentation model is 0.9
  • the class probability predicted by the instance segmentation model is 0.9
  • the class predicted by the instance segmentation model is A probability of 0.89 indicates that the instance segmentation model has a high prediction consistency for this instance.
  • the class probability predicted by the instance segmentation model is 0.9, in the first perturbed image, the class probability predicted by the instance segmentation model is 0.4, and in the second perturbed image, the class predicted by the instance segmentation model is A probability of 0.7 indicates that the prediction consistency of the instance segmentation model for this instance is low.
  • the calculating the detection frame score of each instance based on the detection frame of the first instance, the detection frame of the second instance, and the detection frame of the third instance includes:
  • the corresponding detection frame score of the instance is calculated based on the first intersection and union ratio and the second intersection and union ratio according to a preset first calculation model.
  • the detection box score is used to evaluate whether the prediction of the instance segmentation model on the perturbed first and second perturbed images is consistent with the prediction on the target image.
  • intersection-over-union ratio represents the degree of overlap between two instance detection frames.
  • the smaller the intersection and union ratio IOU the smaller the overlapping area and the smaller the overlapping degree between the two instance detection frames.
  • the greater the IOU the more similar the prediction of the instance segmentation model is to the target image and the perturbed image corresponding to the IOU, that is, the higher the prediction consistency.
  • the smaller the IOU the less similar the prediction of the instance segmentation model is to the target image and the perturbed image corresponding to the IOU, that is, the lower the prediction consistency.
  • instance L2 is an instance with high information content.
  • the calculating the contour mask score of each instance based on the contour mask of the first instance, the contour mask of the second instance and the contour mask of the third instance includes:
  • a contour mask score corresponding to the instance is calculated based on the first Jaccard distance and the second Jaccard distance according to a preset second calculation model.
  • the instance contour mask is similar to the instance detection box, and is also used to evaluate whether the prediction of the instance segmentation model for the perturbed first and second perturbed images is consistent with the prediction of the target image.
  • Jaccard distance is used to describe the dissimilarity between two contour masks.
  • the larger the Jaccard distance the less overlapping area and lower similarity between two contour masks.
  • the smaller the Jaccard distance the more overlapping areas and higher similarity between two contour masks.
  • the larger the Jaccard distance the less similar the prediction of the instance segmentation model is to the target image and the disturbance image corresponding to the Jaccard distance, that is, the lower the prediction consistency.
  • the smaller the Jaccard distance the more similar the prediction of the instance segmentation model is to the target image and the perturbed image corresponding to the Jaccard distance, that is, the higher the prediction consistency.
  • the calculation of the information amount of the corresponding instance according to the category label score and the corresponding detection frame score and the contour mask score includes:
  • the final score is determined as the informativeness of the instance.
  • the average of the class label score, detection box score, and contour mask of each instance can also be calculated as the final score of the instance.
  • the final score is used to indicate whether the prediction of the instance segmentation model on the first disturbed image and the second disturbed image is consistent with the prediction on the target image.
  • the lower the final score it indicates that the prediction of the instance segmentation model on the first perturbed image and the second perturbed image is inconsistent with the prediction on the target image.
  • the performance of the first disturbed image and the second disturbed image is more unstable.
  • the higher the final score it indicates that the prediction of the instance segmentation model for the first perturbed image and the second perturbed image is consistent with the prediction of the target image, which means that even after the target image is perturbed, the target image and the perturbed image obtained by the instance segmentation model are The performance of the first perturbed image and the second perturbed image is still very stable.
  • the instance obtaining module 502 is configured to obtain a first information amount higher than a preset information amount threshold and a first instance corresponding to the first information amount from the target image, except for the first instance in the target image An instance other than one instance is a second instance.
  • the preset information volume threshold is a preset critical value used to indicate the level of information volume.
  • the instance is regarded as the first instance, and when the information volume of a certain instance is lower than the preset information volume threshold, the instance is regarded as the second instance.
  • the first instance refers to a set of multiple instances with an information amount higher than a preset information amount threshold, such as the area enclosed by an ellipse in the target image as shown in FIG. 3 .
  • the second instance refers to a collection of multiple instances whose information volume is lower than the preset information volume threshold, such as the area framed by an irregular figure in the target image as shown in FIG. 4 .
  • the first instance and the second instance completely constitute the set of instances in the image. That is, a certain instance in the target image is either the first instance or the second instance.
  • the first labeling module 503 is configured to manually label the first label of the first instance in the target image.
  • the prediction consistency of the instance segmentation model for the target image and the first disturbance image and the second disturbance image is low, so it should be done manually Annotation of instances, such as multiple pedestrians with obvious occlusion.
  • the first instance in the target image can be identified, and the first instance can be manually annotated by an expert with rich annotation experience, thereby improving the accuracy rate of the annotation of the first instance.
  • the instance segmentation of the medical image needs to identify multiple instance individuals in the image, and accurately outline multiple lesion areas for intelligent auxiliary diagnosis. Therefore, the instance labeling difficulty index of the medical image Higher, by manually labeling these high-information (difficult to label) instances, the accuracy will be greatly improved.
  • the second labeling module 504 is configured to pseudo-label the second label of the second instance in the target image based on a semi-supervised learning method.
  • the second instance is an instance in the target image that is lower than the preset information threshold, the prediction consistency of the instance segmentation model for the target image and the first disturbed image and the second disturbed image is relatively high, so the difficulty of labeling is small. Pseudo-annotation of instances by supervised learning can improve the efficiency of labeling target images.
  • the semi-supervised learning method refers to the instance labeling model obtained through the joint training of the labeled sample set and the unlabeled sample set, and instance labeling of the new unlabeled image through the instance labeling model.
  • the instance label output by the instance labeling model is compared with In terms of instance labels that are artificially labeled, they are called pseudo-labels.
  • the label determining module 505 is configured to obtain an instance label of the target image based on the first label and the second label.
  • the instances in the target image are divided into the first instance and the second instance.
  • the instance label of the first instance is the first label
  • the instance label of the second instance is the second label. Therefore, after obtaining the first label and the second label, the target Labels for all instances in the image have been obtained.
  • model training module 506 is configured to:
  • the annotated image marked with the instance label may refer to an image used to train the instance label model.
  • the target images marked with the instance labels can be added to the real instance labels.
  • Annotated images are used as a training set, so that the instance labeling model is updated based on the training set.
  • the test set includes test images and the real instance labels of each test image, and the test images in the test set are input into the updated instance labeling model, and the test instance labels of the test images are predicted by the updated instance labeling model.
  • the test instance label is the same as the corresponding real instance label, it indicates that the updated instance annotation model is successfully tested on the test image.
  • the test instance label is not the same as the corresponding real instance label, it indicates that the updated instance labeling model fails to test the test image. Calculate the ratio of the number of successful tests to the number of test images in the test set, and use the ratio as the test accuracy of the instance tagging model, and end the training of the instance tagging model when the test accuracy meets a preset accuracy threshold.
  • the number of images with instance labels is greatly increased by using the target images with instance labels, and the instance labeling model can be updated and trained, thereby improving the performance of the instance labeling model.
  • This application marks a small number of instances in the image instead of all instances. For example, instances with occlusion in some areas in the target image are instances with high information content.
  • instances with occlusion in some areas in the target image are instances with high information content.
  • the other instances of other instances have low information content, no manual labeling is required, and the cost of manual labeling is saved. Since the instances with low information content are easier to identify, semi-supervised learning and labeling can improve the accuracy of instance labeling.
  • the labeling efficiency of the instance is improved.
  • This application only needs to manually label a small number of instances in a target image, instead of labeling all instances in the entire target image, so as to obtain instance labels with high accuracy while reducing the workload of instance labeling.
  • This application is suitable for images with complex layouts and mutual occlusion in different areas. Applying this application to the field of intelligent auxiliary recognition of medical images can simultaneously perform region delineation and quantitative evaluation of different target locations and key organ instances, especially for image regions that may be occluded from each other, this application can perform instance segmentation more effectively.
  • This embodiment provides a computer-readable storage medium, on which a computer program is stored.
  • the steps in the above-mentioned embodiment of the method for tagging an image instance based on artificial intelligence are implemented, for example, 1 shows S11-S15:
  • modules 501-505 in FIG. 5 are realized, such as modules 501-505 in FIG. 5:
  • the instance recognition module 501 is used to call a preset instance segmentation model to identify the amount of information of each instance in the target image;
  • the instance obtaining module 502 is configured to obtain a first information amount higher than a preset information amount threshold and a first instance corresponding to the first information amount from the target image, except for the first instance in the target image an instance other than an instance is a second instance;
  • the first labeling module 503 is configured to manually label the first label of the first instance in the target image
  • the second labeling module 504 is configured to pseudo-label the second label of the second instance in the target image based on a semi-supervised learning method
  • the label determining module 505 is configured to obtain an instance label of the target image based on the first label and the second label.
  • the electronic device 6 includes a memory 61 , at least one processor 62 , at least one communication bus 63 and a transceiver 64 .
  • the structure of the electronic device shown in Figure 6 does not constitute a limitation of the embodiment of the present application, it can be a bus structure or a star structure, and the electronic device 6 can also include a ratio diagram more or less other hardware or software, or a different arrangement of components.
  • the electronic device 6 is a device that can automatically perform numerical calculation and/or information processing according to preset or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits, Programmable gate arrays, digital processors and embedded devices, etc.
  • the electronic device 6 may also include a client device, which includes but is not limited to any electronic product that can interact with the client through a keyboard, mouse, remote control, touch pad or voice control device, for example, Personal computers, tablets, smartphones, digital cameras, etc.
  • the electronic device 6 is only an example, and other existing or future electronic products that can be adapted to this application should also be included in the scope of protection of this application, and are included here by reference .
  • a computer program is stored in the memory 61, and when the computer program is executed by the at least one processor 62, all or part of the steps in the above-mentioned method for tagging image instances based on artificial intelligence are implemented.
  • Described memory 61 comprises volatile and nonvolatile memory, such as Random Access Memory (Random Access Memory, RAM), Read-Only Memory (Read-Only Memory, ROM), Programmable Read-Only Memory (Programmable Read-Only Memory, PROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), one-time programmable read-only memory (One-time Programmable Read-Only Memory, OTPROM), electronic erasable rewritable only Read-only memory (Electrically-Erasable Programmable Read-Only Memory, EEPROM), read-only CD-ROM (Compact Disc Read-Only Memory, CD-ROM) or other optical disk storage, disk storage, tape storage, or can be used to
  • the computer-readable storage medium may be non-volatile or volatile. Further, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function, etc.; The data created using the node, etc.
  • Blockchain essentially a decentralized database, is a series of data blocks associated with each other using cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify its Validity of information (anti-counterfeiting) and generation of the next block.
  • the blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.
  • the at least one processor 62 is the control core (Control Unit) of the electronic device 6, and uses various interfaces and lines to connect the various components of the entire electronic device 6, by running or executing the The programs or modules in the memory 61 and call the data stored in the memory 61 to execute various functions of the electronic device 6 and process data.
  • the at least one processor 62 executes the computer program stored in the memory, it realizes all or part of the steps of the artificial intelligence-based image instance labeling method described in the embodiment of the present application; or realizes the artificial intelligence-based image instance labeling method. Label all or part of the functionality of the device.
  • the at least one processor 62 may be composed of an integrated circuit, for example, may be composed of a single packaged integrated circuit, or may be composed of multiple integrated circuits with the same function or different functions, including one or more central processing units (Central Processing unit, CPU), microprocessor, digital processing chip, graphics processor and a combination of various control chips, etc.
  • CPU central processing unit
  • microprocessor microprocessor
  • digital processing chip graphics processor
  • graphics processor a combination of various control chips, etc.
  • the at least one communication bus 63 is configured to implement communication between the memory 61 and the at least one processor 62 and the like.
  • the electronic device 6 can also include a power supply (such as a battery) for supplying power to each component.
  • the power supply can be logically connected to the at least one processor 62 through a power management device, thereby realizing Manage functions such as charging, discharging, and power management.
  • the power supply may also include one or more DC or AC power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.
  • the electronic device 6 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.
  • the above-mentioned integrated units implemented in the form of software function modules can be stored in a computer-readable storage medium.
  • the above-mentioned software function modules are stored in a storage medium, and include several instructions to make a computer device (which may be a personal computer, electronic device, or network device, etc.) or a processor (processor) execute the methods described in various embodiments of the present application part.
  • the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, and may be located in one place or distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional module in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware, or in the form of hardware plus software function modules.

Abstract

La présente demande se rapporte au domaine technique de l'intelligence artificielle. L'invention concerne un procédé d'étiquetage d'instance d'image basé sur l'intelligence artificielle, et un dispositif associé. Une quantité d'informations de chaque instance dans une image cible est identifiée au moyen de l'appel d'un modèle de segmentation d'instance prédéfini, de manière à acquérir, de l'image cible, une première quantité d'informations qui est supérieure à une valeur seuil de quantité d'informations prédéfinie, une première instance qui correspond à la première quantité d'informations, et une seconde instance autre que la première instance ; ensuite, une première étiquette de la première instance est étiquetée manuellement, de telle sorte que la précision d'une étiquette d'instance étiquetée manuellement soit élevée ; étant donné qu'une quantité d'informations de la seconde instance soit faible, et que la seconde instance soit facilement identifiée et étiquetée par un modèle, le pseudo-étiquetage est effectué sur une seconde étiquette de la seconde instance sur la base d'un mode d'apprentissage semi-supervisé, de telle sorte que l'efficacité d'étiquetage soit élevée ; et une étiquette d'instance de l'image cible est obtenue sur la base de la première étiquette et de la seconde étiquette. La présente demande peut être appliquée au domaine du traitement médical numérique pour l'étiquetage d'une instance dans une image médicale.
PCT/CN2022/071328 2021-08-30 2022-01-11 Procédé d'étiquetage d'instance d'image basé sur l'intelligence artificielle, et dispositif associé WO2023029348A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111005698.2A CN113705687B (zh) 2021-08-30 2021-08-30 基于人工智能的图像实例标注方法及相关设备
CN202111005698.2 2021-08-30

Publications (1)

Publication Number Publication Date
WO2023029348A1 true WO2023029348A1 (fr) 2023-03-09

Family

ID=78656924

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/071328 WO2023029348A1 (fr) 2021-08-30 2022-01-11 Procédé d'étiquetage d'instance d'image basé sur l'intelligence artificielle, et dispositif associé

Country Status (2)

Country Link
CN (1) CN113705687B (fr)
WO (1) WO2023029348A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113705687B (zh) * 2021-08-30 2023-03-24 平安科技（深圳）有限公司 基于人工智能的图像实例标注方法及相关设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112163634A (zh) * 2020-10-14 2021-01-01 平安科技（深圳）有限公司 实例分割模型样本筛选方法、装置、计算机设备及介质
US20210241034A1 (en) * 2020-01-31 2021-08-05 Element Al Inc. Method of and system for generating training images for instance segmentation machine learning algorithm
CN113239950A (zh) * 2021-01-13 2021-08-10 深延科技(北京)有限公司 标注方法、装置、电子设备及存储介质
CN113705687A (zh) * 2021-08-30 2021-11-26 平安科技（深圳）有限公司 基于人工智能的图像实例标注方法及相关设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210241034A1 (en) * 2020-01-31 2021-08-05 Element Al Inc. Method of and system for generating training images for instance segmentation machine learning algorithm
CN112163634A (zh) * 2020-10-14 2021-01-01 平安科技（深圳）有限公司 实例分割模型样本筛选方法、装置、计算机设备及介质
CN113239950A (zh) * 2021-01-13 2021-08-10 深延科技(北京)有限公司 标注方法、装置、电子设备及存储介质
CN113705687A (zh) * 2021-08-30 2021-11-26 平安科技（深圳）有限公司 基于人工智能的图像实例标注方法及相关设备

Also Published As

Publication number Publication date
CN113705687B (zh) 2023-03-24
CN113705687A (zh) 2021-11-26

Similar Documents

Publication Publication Date Title
JP7143008B2 (ja) 深層学習に基づく医用画像検出方法及び装置、電子機器及びコンピュータプログラム
US10929420B2 (en) Structured report data from a medical text report
CN107292086A (zh) 图像分析问答
CN108899064A (zh) 电子病历生成方法、装置、计算机设备和存储介质
CN110390674B (zh) 图像处理方法、装置、存储介质、设备以及系统
Beddiar et al. Automatic captioning for medical imaging (MIC): a rapid review of literature
CN112365980A (zh) 脑肿瘤多靶点辅助诊断与前瞻性治疗演化可视化方法及系统
Bano et al. AutoFB: Automating fetal biometry estimation from standard ultrasound planes
CN113723513A (zh) 多标签图像分类方法、装置及相关设备
CN112884060A (zh) 图像标注方法、装置、电子设备及存储介质
CN113656706A (zh) 基于多模态深度学习模型的信息推送方法及装置
WO2023029348A1 (fr) Procédé d'étiquetage d'instance d'image basé sur l'intelligence artificielle, et dispositif associé
CN115719334A (zh) 基于人工智能的医学影像评价方法、装置、设备及介质
CN117237351B (zh) 一种超声图像分析方法以及相关装置
CN113724830A (zh) 基于人工智能的用药风险检测方法及相关设备
WO2022247007A1 (fr) Procédé et appareil de classement d'images médicales, dispositif électronique et support de stockage lisible par ordinateur
CN111967539A (zh) 基于cbct数据库的颌面部骨折的识别方法、装置及终端设备
Kugener et al. Deep neural networks can accurately detect blood loss and hemorrhage control task success from video
Mason et al. Lung ultrasound segmentation and adaptation between COVID-19 and community-acquired pneumonia
CN111967540B (zh) 基于ct数据库的颌面部骨折的识别方法、装置及终端设备
CN113436155B (zh) 基于深度学习的超声臂丛神经图像识别方法
Wang et al. Accurate corresponding fiber tract segmentation via FiberGeoMap learner
Guo et al. The Gap in the Thickness: Estimating Effectiveness of Pulmonary Nodule Detection in Thick-and Thin-Section CT Images with 3D Deep Neural Networks
CN113393445B (zh) 乳腺癌影像确定方法及系统
CN111724893B (zh) 医疗影像辨识装置及医疗影像辨识方法

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE