CN114067179A

CN114067179A - Image annotation method, and training method and device of annotation model

Info

Publication number: CN114067179A
Application number: CN202111371006.6A
Authority: CN
Inventors: 郭世嘉; 廖术
Original assignee: Shanghai United Imaging Intelligent Healthcare Co Ltd
Current assignee: Shanghai United Imaging Intelligent Healthcare Co Ltd
Priority date: 2021-11-18
Filing date: 2021-11-18
Publication date: 2022-02-18

Abstract

The application relates to an image annotation method, and an annotation model training method and device. The method comprises the following steps: the computer equipment acquires medical image data to be labeled, and inputs the medical image data to be labeled into a preset labeling model to obtain a labeling result of the medical image data. The preset labeling model is a model obtained by training according to a preset data set; the preset data set comprises first marked image data and second marked image data, the first marked image data are pre-marked image data, and the second marked image data are obtained by marking adjacent unmarked image data according to the first marked image data. In the scheme, the first marked image data is accurately marked in advance, the second marked image data is obtained by marking according to the first marked image data, certain accuracy is achieved, and marking results of marking models obtained by training according to the first marked image data and the second marked image data are also accurate.

Description

Image annotation method, and training method and device of annotation model

Technical Field

The present application relates to the field of image processing technologies, and in particular, to an image annotation method, and a method and an apparatus for training an annotation model.

Background

When the machine learning training model needs a large amount of labeled data for model learning, but the labeled data is a huge workload. In particular, in the medical technology field, the task of labeling medical images is particularly difficult. Because the medical image is mainly a three-dimensional image and the labeling of the medical image is strict, professional training needs to be performed on a labeling person, the time and the labor are consumed, the cost is high, and the obtained labeling result of the medical image is inaccurate.

Disclosure of Invention

In view of the above, it is necessary to provide an image annotation method, an annotation model training method, and an annotation model training device that can improve the accuracy of medical image annotation.

In a first aspect, an image annotation method is provided, which includes:

acquiring medical image data to be marked;

inputting medical image data to be labeled into a preset labeling model to obtain a labeling result of the medical image data; the preset labeling model is a model obtained by training according to a preset data set; the preset data set comprises first marked image data and second marked image data, the first marked image data are pre-marked image data, and the second marked image data are obtained by marking adjacent unmarked image data according to the first marked image data.

In one optional embodiment, the method for training the preset labeling model includes:

determining first marked image data and second marked image data according to the original medical image data to obtain a preset data set;

and training the initial labeling model according to a preset data set to obtain a preset labeling model.

In one optional embodiment, determining the first annotated image data and the second annotated image data from the raw medical image data comprises:

labeling the first original medical image data according to a user labeling instruction to obtain first labeled image data; the first original medical image data is image data with image definition greater than or equal to a first definition threshold in the original medical image data;

labeling the second original medical image data according to the first labeled image data to obtain second labeled image data; the second original medical image data is image data which is not marked and has the image definition less than or equal to a second definition threshold value in the original medical image data; the upper and lower layers of adjacent images of the second original medical image data belong to the first labeled image data; the first sharpness threshold is greater than the second sharpness threshold.

In an optional embodiment, labeling the second original medical image data according to the first labeled image data to obtain second labeled image data includes:

determining upper-layer first labeled image data and lower-layer first labeled image data of second original medical image data from the first labeled image data;

determining a pixel union set of the labeling contours according to the first labeling contour of the upper-layer first labeling image data and the second labeling contour of the lower-layer first labeling image data;

determining foreground pixel intersection and background pixel intersection according to pixels in the upper layer first marked image data and pixels in the lower layer first marked image data;

and labeling the second original medical image data according to the labeled contour pixel union set, the foreground pixel intersection set and the background pixel intersection set to obtain second labeled image data.

In one optional embodiment, labeling the second original medical image data according to the labeled contour pixel union set, the foreground pixel intersection set, and the background pixel intersection set to obtain second labeled image data includes:

labeling pixels which do not belong to the labeled pixel union set and belong to the foreground pixel intersection in the second original medical image data as foreground pixels;

and marking pixels which do not belong to the marked pixel union set and belong to the intersection of the background pixels in the second original medical image data as background pixels.

In one optional embodiment, training the initial labeling model according to a preset data set to obtain a preset labeling model includes:

inputting a preset data set into an initial labeling model to obtain a labeling result of the preset data set;

and updating the preset data set according to the labeling result, and inputting the updated preset data set into the initial labeling model for iterative training until the initial labeling model meets the convergence condition to obtain a preset labeling model.

In one optional embodiment, inputting the preset data set into the initial labeling model to obtain a labeling result of the preset data set, includes:

inputting a preset data set into an initial labeling model to obtain the foreground prediction probability of each pixel in the preset data set;

pixels with a foreground prediction probability greater than or equal to a first probability threshold are labeled as foreground pixels, and pixels with a foreground prediction probability less than or equal to a second probability threshold are labeled as background pixels.

In an optional embodiment, the convergence condition includes:

the difference value between the number of foreground pixels in the labeling result of the current iteration and the number of foreground pixels in the labeling result of the last iteration is smaller than a difference threshold value; and/or the number of iterations satisfies a number threshold.

In a second aspect, a method for training an annotation model is provided, the method comprising:

determining first marked image data and unmarked image data according to the original medical image data; the first marked image data is pre-marked image data;

according to the first marked image data, marking adjacent unmarked image data to obtain second marked image data;

determining a preset data set according to the first marked image data and the second marked image data;

inputting a preset data set into an initial labeling model for training to obtain a preset labeling model; the preset labeling model is used for determining a labeling result of the medical image data to be labeled.

In a third aspect, an image annotation apparatus is provided, the apparatus comprising:

the acquisition module is used for acquiring medical image data to be marked;

the marking module is used for inputting the medical image data to be marked into a preset marking model to obtain a marking result of the medical image data; the preset labeling model is a model obtained by training according to a preset data set; the preset data set comprises first marked image data and second marked image data, the first marked image data are pre-marked image data, and the second marked image data are obtained by marking adjacent unmarked image data according to the first marked image data.

In a fourth aspect, there is provided a training apparatus for labeling a model, the apparatus comprising:

the first determination module is used for determining first marked image data and unmarked image data according to the original medical image data; the first marked image data is pre-marked image data;

the preprocessing module is used for labeling adjacent unlabelled image data according to the first labeled image data to obtain second labeled image data;

the second determining module is used for determining a preset data set according to the first marked image data and the second marked image data;

the training module is used for inputting a preset data set into the initial labeling model for training to obtain a preset labeling model; the preset labeling model is used for determining a labeling result of the medical image data to be labeled.

In a fifth aspect, a computer device is provided, which comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the method according to any one of the first and second aspects when executing the computer program.

A sixth aspect provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any of the first and second aspects.

In a fifth aspect, the present application further provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the method according to any of the first and second aspects when the processor executes the computer program.

In a sixth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the method of any of the first and second aspects described above.

In a seventh aspect, the present application further provides a computer program product. The computer program product comprising a computer program that, when executed by a processor, performs the method of any of the first and second aspects.

According to the image labeling method and the training method and device of the labeling model, the computer equipment obtains the medical image data to be labeled, and the medical image data to be labeled is input into the preset labeling model to obtain the labeling result of the medical image data. The preset labeling model is a model obtained by training according to a preset data set; the preset data set comprises first marked image data and second marked image data, the first marked image data are pre-marked image data, and the second marked image data are obtained by marking adjacent unmarked image data according to the first marked image data. In the scheme, the preset labeling model is obtained by training according to the first labeling image data and the second labeling image data, wherein the first labeling image data is pre-labeled data with certain accuracy, and the second labeling image data is obtained by labeling adjacent unlabeled image data according to the first labeling image data, namely the second labeling image data is used as the extended data of the labeling data and also has certain accuracy.

Drawings

FIG. 1 is a diagram of an application environment of an image annotation method according to an embodiment;

FIG. 2 is a flow chart illustrating an image annotation process according to an embodiment;

FIG. 3 is a flowchart illustrating an image annotation process according to an embodiment;

FIG. 4 is a flowchart illustrating an image annotation process according to an embodiment;

FIG. 5 is a flowchart illustrating an image annotation process according to an embodiment;

FIG. 6 is a flowchart illustrating an image annotation process according to an embodiment;

FIG. 7 is a flowchart illustrating an image annotation process according to an embodiment;

FIG. 8 is a schematic flow chart diagram illustrating a method for training a label model in accordance with another embodiment;

FIG. 9 is a schematic flow chart diagram illustrating a method for training and applying a label model in one embodiment;

FIG. 10 is a block diagram showing the construction of an image labeling apparatus according to an embodiment;

FIG. 11 is a block diagram showing the construction of an image labeling apparatus according to an embodiment;

FIG. 12 is a block diagram showing the structure of a training apparatus for labeling a model according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The image annotation method provided by the application can be applied to the application environment shown in fig. 1. In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 1. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement an image annotation method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

Those skilled in the art will appreciate that the architecture shown in fig. 1 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

The following describes in detail the technical solutions of the present application and how the technical solutions of the present application solve the above technical problems by embodiments and with reference to the drawings. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. It should be noted that the image annotation method provided in the embodiments of fig. 2 to fig. 7 of the present application is implemented by a computer device, and may also be an image annotation apparatus, which may be a part or all of the computer device through software, hardware, or a combination of software and hardware. In the following method embodiments, the execution subject is a computer device as an example.

In one embodiment, as shown in fig. 2, there is provided an image annotation method, including the steps of:

s201, medical image data to be marked are obtained.

The medical image data may be image data of any form, for example, Computed Tomography (CT) image data of each part, or Magnetic Resonance (MR) image data of each part, and the type of the medical image data is not limited in this embodiment.

In this embodiment, the computer device may obtain medical image data that needs to be labeled from the database, or may accept medical image data that needs to be labeled and is imported by other terminals or users based on the interactive interface.

S202, inputting the medical image data to be labeled into a preset labeling model to obtain a labeling result of the medical image data.

The preset labeling model is a model obtained by training according to a preset data set; the preset data set comprises first marked image data and second marked image data, the first marked image data are pre-marked image data, and the second marked image data are obtained by marking adjacent unmarked image data according to the first marked image data.

The first marked image data and the second marked image data are derived from a plurality of layers of image data of the same original image data. The pre-labeled image data (first labeled image data) may be image data obtained by manually labeling the original image data, or image data obtained by screening and labeling the original image data based on certain parameters of the image data. Here, the parameter of the image data may be a parameter such as sharpness and resolution. For example, original image data with a definition greater than or equal to a first definition threshold is labeled to obtain first labeled image data, where the first definition threshold may be 80%, that is, under the condition that the definition of the original image data is greater than or equal to 80%, the original image data is labeled to obtain first labeled image data; or, the original image data with the resolution greater than or equal to the first resolution threshold may be labeled to obtain first labeled image data, where the first resolution threshold may be 100PPI, and the original image data is labeled to obtain the first labeled image data when the resolution of the original image data is greater than or equal to 100 PPI. After the first labeled image data is obtained, the original image data, except the first labeled image data, has the remaining unlabelled image data, the computer device can screen the remaining unlabelled image data, determine that the adjacent upper and lower slice layers of image data all belong to the candidate unlabelled image data of the first labeled image data, and label the candidate unlabelled image data according to the first labeled image data, so as to obtain the second labeled image data. The first labeled image data is manually labeled, so that the result is relatively accurate, and the second labeled image data is obtained based on the first labeled image data, so that the second labeled image data is relatively accurate.

In this embodiment, the computer device trains to obtain a preset labeling model according to the first labeling image data and the second labeling image data with higher accuracy, and the labeling result of the labeling model is more accurate. And inputting the acquired medical image data to be labeled into a preset labeling model to obtain a corresponding labeling result. Optionally, the preset labeling model can be applied to the classification model, so that the input data is labeled based on the preset labeling model to obtain a labeling result, and classification is performed based on the labeling result, so that the obtained classification result is more accurate; or, the preset labeling model can also be applied to a segmentation model, and the input data is labeled based on the preset labeling model to obtain a labeling result, so that the image segmentation is performed based on the labeling result, and the efficiency and the accuracy of the image segmentation are further improved.

In the image annotation method, the computer equipment acquires medical image data to be annotated, and inputs the medical image data to be annotated into a preset annotation model to obtain an annotation result of the medical image data. The preset labeling model is a model obtained by training according to a preset data set; the preset data set comprises first marked image data and second marked image data, the first marked image data are pre-marked image data, and the second marked image data are obtained by marking adjacent unmarked image data according to the first marked image data. In the scheme, the preset labeling model is obtained by training according to the first labeling image data and the second labeling image data, wherein the first labeling image data is pre-labeled data with certain accuracy, and the second labeling image data is obtained by labeling adjacent unlabeled image data according to the first labeling image data, namely the second labeling image data is used as the extended data of the labeling data and also has certain accuracy.

The image annotation method is based on a preset annotation model, and in an alternative embodiment, as shown in fig. 3, the method for training the preset annotation model includes:

s301, determining first annotation image data and second annotation image data according to the original medical image data to obtain a preset data set.

The original medical image data is multi-slice image data, and the first marked image data and the second marked image data are derived from the multi-slice image data in the same original image data.

In this embodiment, the computer device determines first annotated image data and second annotated image data from the raw medical image data. Optionally, the first labeled image data may be image data obtained by manually labeling the original image data, for example, the first labeled image data imported by the user through the interactive interface is obtained, and for example, if the image quality of the original medical image data is high, that is, all or more continuous slice images of the original medical image data are relatively clear, the user may label the original medical image data through the interactive interface in a full-image labeling manner to obtain the first labeled image data; if only part of continuous slice images in the original medical image data are clear, a user can label the original medical image data through an interactive interface in a continuous slice labeling mode to obtain first labeled image data; if only individual slice images in the original medical image data are clear, a user can label the original medical image data through an interactive interface in a mode of labeling on an independent slice to obtain first labeled image data, and further, in the process of labeling by the user, the user can add subjective judgment of the user on the confidence coefficient of the original medical image data to further improve the labeling accuracy. In addition, the first labeled image data can also be data obtained by labeling the image data meeting the image quality requirement after the computer device performs image quality screening on the original medical image data. The second labeled image data is data obtained by performing supplementary labeling on the remaining unlabeled image data in the original medical image data according to the first labeled image data.

Optionally, in an alternative embodiment, as shown in fig. 4, the determining the first annotated image data and the second annotated image data from the raw medical image data includes:

s401, labeling the first original medical image data according to a user labeling instruction to obtain first labeled image data.

The first original medical image data is image data with image definition greater than or equal to a first definition threshold in the original medical image data, the first original medical image data comprises image data of a plurality of slices, and the first marked image data comprises image data of at least two slices.

In this embodiment, the computer device may perform data filtering according to the image definition of each of the original medical image data, and determine that the image definition is greater than or equal to a first definition threshold image data as the first original medical image data. For example, the first original medical image data is the influence data having an image sharpness of 80% or more in the original medical image data. Optionally, the computer device may output the first original medical image data through a user interaction interface, so that a user labels the first original medical image data, because the image definition of the first original medical image data is higher, the accuracy of labeling performed by the user is higher, and accordingly, the computer device receives the first labeled image data labeled by the user based on the user interaction interface. Optionally, the computer device may further label the first original medical image data through the label model after the original medical image data is subjected to definition screening, so as to obtain first labeled image data.

S402, labeling the second original medical image data according to the first labeled image data to obtain second labeled image data.

The second original medical image data is image data which is not marked and has the image definition smaller than or equal to a second definition threshold value in the original medical image data; the upper and lower layers of adjacent images of the second original medical image data belong to the first labeled image data; the first sharpness threshold is greater than the second sharpness threshold.

In this embodiment, the first original medical image data and the second original medical image data are derived from multi-slice image data of the same original medical image data, in an actual situation, there are first labeled image data labeled to obtain a partial slice and original medical image data of an unlabeled partial slice in the multi-slice image data of the same original medical image data, in an actual situation, the labeled first labeled image data and the unlabeled original medical image data may be image data of an adjacent slice, that is, the previous slice image data or the next slice image data of the original medical image data may be unlabeled image data or labeled first labeled image data, based on which in this actual situation, the unlabeled image data in the original medical image data may be screened based on the second definition threshold in this embodiment, and obtaining candidate medical image data with the definition being greater than or equal to a second definition threshold, and further determining that the candidate medical image data with the corresponding image data of the previous slice and the image data of the next slice which are both the first marked image data is second original image data. Likewise, the second raw medical image data may also include medical image data of at least one slice as described in the above embodiments.

And after the computer equipment screens out second original medical image data from the original medical image data according to the conditions, labeling the second labeled image data according to the first labeled image data to obtain second labeled image data.

In this embodiment, the computer device may determine the second original medical image data from the original medical image data according to the second definition threshold and a preset screening method of the second original medical image data, so as to perform further labeling processing on the second original medical image data, perform different labeling processing on the original medical image data with different definitions, and be more suitable for the definition of the medical image data, and obtain a more accurate labeling result.

Optionally, in an optional embodiment, as shown in fig. 5, labeling the second original medical image data according to the first labeled image data to obtain second labeled image data includes:

s501, determining upper-layer first labeled image data and lower-layer first labeled image data of second original medical image data from the first labeled image data.

In this embodiment, after the computer device determines the second original medical image data, if the second original medical image data includes 1 slice image data, the upper-layer first tagged image data and the lower-layer first tagged image data adjacent to the current second original medical image data are determined from the first tagged image data according to a slice identifier of the current second original medical image data, where the slice identifier may be a slice number. And if the second original medical image data comprises a plurality of slice layer image data, determining upper layer first labeled image data and lower layer first labeled image data which are adjacent to the second original medical image data from the first labeled image data according to the slice layer identification of each second original medical image data.

S502, determining a pixel union set of the labeling contours according to the first labeling contour of the upper-layer first labeling image data and the second labeling contour of the lower-layer first labeling image data.

In this embodiment, after determining adjacent upper-layer first labeled image data and lower-layer first labeled image data of second original medical image data, the computer device obtains a first labeled contour labeled in the upper-layer first labeled image data, obtains a second labeled contour labeled in the lower-layer first labeled image data, and determines a contour pixel union according to the first labeled contour and the second labeled contour. Optionally, the computer device may further perform dilation processing on the first labeled contour and the second labeled contour, for example, taking kernel as 5, and perform dilation processing on the first labeled contour and the second labeled contour to obtain a labeled contour pixel union C_∪This embodiment is not limited to this.

S503, determining foreground pixel intersection and background pixel intersection according to pixels in the upper layer first labeling image data and pixels in the lower layer first labeling image data.

In this embodiment, the computer device obtains each pixel in the upper-layer first labeled image data, and determines a labeled upper-layer foreground pixel and an unlabeled upper-layer background pixel from each pixel; acquiring each pixel of lower-layer first labeled image data, and determining a labeled lower-layer foreground pixel and an unlabeled lower-layer background pixel from each pixel; and determining foreground pixel intersection F according to upper layer foreground pixel and lower layer foreground pixel_∩(ii) a Determining background pixel intersection B according to upper layer background pixel and lower layer background pixel_∩。

And S504, labeling the second original medical image data according to the labeled contour pixel union set, the foreground pixel intersection set and the background pixel intersection set to obtain second labeled image data.

In this embodiment, the computer device determines the marked contour pixel union C_∪Foreground pixel intersection F_∩Background pixel intersection B_∩And then labeling each pixel in the second original medical image data according to a preset pixel division condition to obtain second labeled image data.

Optionally, in the process of determining the second labeled image data by the computer device, in one optional embodiment, labeling the second original medical image data according to the labeled contour pixel union set, the foreground pixel intersection set, and the background pixel intersection set to obtain the second labeled image data includes the following two cases:

one is as follows: labeling pixels which do not belong to the labeled pixel union set and belong to the foreground pixel intersection in the second original medical image data as foreground pixels;

the second step is as follows: and marking pixels which do not belong to the marked pixel union set and belong to the intersection of the background pixels in the second original medical image data as background pixels.

In this embodiment, the computer device gathers C according to the labeled contour pixels_∪Foreground pixel intersection F_∩Background pixel intersection B_∩The type of each pixel in the second original medical image data is determined, i.e. the second original medical image data is labeled. Wherein, if the pixel does not belong to the marked outline pixel union C_∪Belong to the foreground pixel intersection F_∩Then the pixel is determined to be the foreground label, optionally, the pixel value of the pixel may be set to a first value, where the first value may be 0; if the pixel does not belong to the marked outline pixel union C_∪Belong to the background pixel intersection B_∩Then the pixel is determined to be the background label, and optionally, the pixel value of the pixel may be set to a second value,here the second value may be 1; in addition, there is a case where a pixel belonging to the merged set of labeled pixels exists in the second original medical image data, that is, if it is determined that the pixel in the second original medical image data belongs to the merged set of labeled contour pixels C_∪Setting the pixel value of the pixel to be a third value, where the third value may be-1, and it should be noted that if the pixel value of the pixel point is the third value, it is indicated that the pixel needs to have uncertainty, that is, it is not determined whether the pixel belongs to the foreground pixel or the background pixel at all, so that the pixel is subjected to the blurring processing, and is not used as the training pixel in the training of the next iteration. In particular, the pixel

The determination formula of the type of (1) is as follows:

in this embodiment, under the condition that the definition of the second original image data is smaller than the first definition threshold, the labeling method corresponding to the first labeled image data is not adopted for processing, so that the problem of inaccurate labeling caused by the fact that the definition of the second original medical image data does not meet the first definition threshold is avoided, the second original medical image data with the definition greater than or equal to the second definition threshold is further labeled and processed, and the definition of the second original medical image data is attached, so that the labeling result obtained by adopting the labeling method is more accurate.

S302, training the initial labeling model according to a preset data set to obtain a preset labeling model.

The initial labeling model can be a machine learning model or other neural network models.

In this embodiment, the computer device determines the first labeled image data and the second labeled image data from the original medical image data according to the method provided in the above embodiment, so as to obtain a preset data set according to the first labeled image data and the second labeled image data, inputs the preset data set into the initial labeling model to train the model, and obtains a preset labeling model meeting the convergence condition or the training condition.

In this embodiment, the first annotation image and the second annotation image are obtained according to the method in the above embodiment, the first annotation image and the second annotation image are obtained by using an annotation method suitable for the definition of the first annotation image and the second annotation image, the accuracy of annotation is higher, the initial annotation model is trained by using training data with higher accuracy, and the prediction accuracy of the trained model is also higher.

Optionally, in an optional embodiment, as shown in fig. 6, the training the initial labeling model according to the preset data set to obtain a preset labeling model includes:

s601, inputting the preset data set into the initial labeling model to obtain a labeling result of the preset data set.

In this embodiment, the computer device inputs the preset data set into the initial labeling model, and performs labeling prediction on image data of an unmarked part in the preset data set to obtain a labeling result of the current preset data set.

Optionally, as shown in fig. 7, inputting the preset data set into the initial labeling model to obtain a labeling result of the preset data set, where the labeling result includes:

s701, inputting the preset data set into the initial labeling model to obtain the foreground prediction probability of each pixel in the preset data set.

In this embodiment, the computer device inputs the preset data set into the initial labeling model, and obtains the prediction probability of each pixel in the preset data set as the foreground label, where the prediction probability of each pixel in the image data of the labeled part in the preset data set and the prediction probability of each pixel in the image data of the unlabeled part in the preset data set are included. The prediction probability of each pixel in the image data of the unmarked part can be obtained from the last marking result in the iteration process.

S702, marking the pixels with the foreground prediction probability being larger than or equal to the first probability threshold as foreground pixels, and marking the pixels with the foreground prediction probability being smaller than or equal to the second probability threshold as background pixels.

In this embodiment, the computer device further processes the prediction probability of each pixel in the preset data set as the foreground label through the label model. Illustratively, the first probability threshold is T1, the second probability threshold is T2, and the computer device may label, as foreground pixels, pixels of the preset data set for which the foreground prediction probability is greater than or equal to the first probability threshold T1 by the labeling model; labeling a pixel of which the foreground prediction probability is less than or equal to a second probability threshold T2 in each pixel of the preset data set as a background pixel, and if the foreground prediction probability of the pixel is less than T1 and greater than T2, performing a blurring process on the pixel, similarly, setting the pixel value of the pixel as a third value, and not serving as a training pixel in the training of the next iteration, thereby implementing labeling prediction on image data of labeled and unlabeled parts in the preset data set, alternatively, T1 may be 0.9, and T2 may be 0.1. Wherein, according to the pixel

The foreground prediction probability of (a) determines whether a foreground pixel or background pixel formula is as follows:

in this embodiment, in the process of training the initial labeling model, labeling prediction is performed on image data of an unlabeled portion in a preset data set, only pixels determined as foreground pixels and background pixels are reserved as training pixels according to a labeling prediction result, and fuzzy pixels are removed, so that the accuracy of the training data set is improved.

And S602, updating the preset data set according to the labeling result, and inputting the updated preset data set into the initial labeling model for iterative training until the initial labeling model meets the convergence condition to obtain a preset labeling model.

Optionally, the convergence condition includes: the difference value between the number of foreground pixels in the labeling result of the current iteration and the number of foreground pixels in the labeling result of the last iteration is smaller than a difference threshold value; and/or the number of iterations satisfies a number threshold.

In the embodiment, after the computer device determines the labeling type of each pixel according to the foreground prediction probability of each pixel, adding the pixels determined as foreground pixels into the training set as foreground data, adding the pixels determined as background pixels into the training set as background data, performing fuzzy processing on the pixels with the pixel values of the third value, not serving as the training data in the training set, updating the training data set for each marking prediction, inputting the updated training data set into the marking model, continuing marking iteration until the iteration number meets the requirement of the preset number, or, determining that the labeling model reaches convergence until the difference value between the number of foreground pixels in the current labeling result and the number of foreground data in the last labeling result is smaller than a difference threshold value, and ending iteration to obtain a preset labeling model. It should be noted here that, when the difference between the number of foreground pixels in the current labeling result and the number of foreground data in the last labeling result is smaller than the difference threshold, it means that the increment of the foreground data is already small enough to be stable, that is, there is almost no unmarked data that can be marked in the training data set, and in this case, it is determined that the iteration of the labeling model reaches convergence.

In this embodiment, the computer device determines the first labeled image data according to the image definition, where the first labeled image data has higher accuracy, and then determines the second labeled image data according to the first labeled image data, so as to achieve the purpose of expanding the labeled image data, so that the data size of the preset data set for training the initial labeled model is relatively rich, and the training of the initial labeled model is relatively accurate. In addition, in the process of training an initial labeling model, labeling prediction is carried out on image data of unmarked parts in a preset data set, only pixels determined as foreground pixels and background pixels are reserved as training pixels according to a labeling prediction result, fuzzy pixels are eliminated, the accuracy of the training data set is improved, the training data set is updated according to a result output by each labeling, the data in the training data set is clearer and more accurate, and the labeling accuracy of the trained labeling model is further improved.

The following describes in detail the technical solutions of the present application and how the technical solutions of the present application solve the above technical problems by embodiments and with reference to the drawings. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. It should be noted that, in the method for training a label model provided in the embodiment of fig. 8 of the present application, an execution subject is a computer device, and may also be a training apparatus for a label model, and the training apparatus for a label model may be a part or all of a computer device by software, hardware, or a combination of software and hardware. In the following method embodiments, the execution subject is a computer device as an example.

In one embodiment, as shown in fig. 8, there is provided a method for training an annotation model, the method comprising:

s801, determining first marked image data and unmarked image data according to original medical image data; the first labeled image data is pre-labeled image data.

The original medical image data refers to multi-slice image data of a single picture, and the first labeled image data is similar to the first labeled image data in the embodiment of fig. 4 and is determined by labeling slice image data with an image definition greater than or equal to a first definition threshold, where the first definition threshold may be 80%. Then, the unmarked image data is the data in the unmarked original medical image data with the definition smaller than the first threshold. Here, the first threshold may be 80%, for example. The definition determination method is also similar to the above embodiment, and can be determined by the gray-scale value of the slice image data.

S802, according to the first marked image data, marking adjacent unmarked image data to obtain second marked image data.

In this embodiment, similar to the embodiments provided in fig. 4 and 5 above, to expand the data in the data set used by the training model, the computer device may label the unlabeled image data according to the first labeled image data. For example, the computer device may determine target unlabeled image data from the unlabeled image data, where the target unlabeled image data refers to a previous layer image data and a next layer image data that are adjacent to the target unlabeled image data and both belong to the first labeled image data. In this case, similar to the method provided in the above-mentioned embodiment of fig. 4 and 5, the computer device may label the target unlabeled image data according to the image data labeled at the previous layer and the image data labeled at the next layer, so as to achieve the purpose of expanding the labeled image data set.

And S803, determining a preset data set according to the first marked image data and the second marked image data.

In this embodiment, the computer device determines the first annotated image data obtained in the above step 801 and the second annotated image data obtained in the above step 802 as a preset data set for training the annotated model.

S804, inputting a preset data set into the initial labeling model for training to obtain a preset labeling model; the preset labeling model is used for determining a labeling result of the medical image data to be labeled.

In this embodiment, the computer device inputs a preset data set into the initial labeling model for training, and repeatedly trains the labeling model with each output result as a new input data set until the labeling model converges or the training times meet the training requirements, thereby obtaining a preset marker model. Similarly, the specific training process of the annotation model may refer to the embodiments provided in fig. 6 and fig. 7, which are not repeated in this embodiment.

According to the training method of the labeling model, the computer equipment determines first labeled image data and unlabelled image data according to original medical image data, labels adjacent unlabelled image data according to the first labeled image data to obtain second labeled image data, determines a preset data set according to the first labeled image data and the second labeled image data, and inputs the preset data set into the initial labeling model for training to obtain the preset labeling model. In the scheme, the first marked image data is pre-marked data and has certain accuracy, the second marked image data is obtained by marking adjacent unmarked image data according to the first marked image data, namely the second marked image data is used as extended data of the marked data and also has certain accuracy, and a marking model is obtained by training according to the extended data and the first marked image data and is more accurate in marking result of the medical image data.

The training method of the labeling model provided in this embodiment is similar to the training method of the preset model provided in fig. 3, and details are not repeated in this embodiment.

To better explain the above method, as shown in fig. 9, the present embodiment provides a method for training and applying a label model, which specifically includes:

s101, labeling the first original medical image data according to a user labeling instruction to obtain first labeled image data;

s102, determining upper-layer first labeled image data and lower-layer first labeled image data of second original medical image data from the first labeled image data;

s103, determining a pixel union set of the labeled contours according to the first labeled contour of the upper-layer first labeled image data and the second labeled contour of the lower-layer first labeled image data;

s104, determining a foreground pixel intersection and a background pixel intersection according to pixels in the upper layer first labeled image data and pixels in the lower layer first labeled image data;

s105, labeling the second original medical image data according to the labeled contour pixel union set, the foreground pixel intersection and the background pixel intersection to obtain second labeled image data;

s106, inputting the first annotation image data and the second annotation image data into the initial annotation model to obtain an annotation result of the preset data set;

s107, updating the preset data set according to the labeling result, inputting the updated preset data set into the initial labeling model for iterative training until the initial labeling model meets the convergence condition, and obtaining a preset labeling model;

s108, acquiring medical image data to be labeled;

s109, inputting the medical image data to be labeled into a preset labeling model to obtain a labeling result of the medical image data.

In this embodiment, the preset labeling model is obtained by training according to the first labeling image data and the second labeling image data, wherein the first labeling image data is pre-labeled data with a certain accuracy, and the second labeling image data is obtained by labeling adjacent unlabeled image data according to the first labeling image data, that is, the second labeling image data is used as the extended data of the labeling data with a certain accuracy, and the labeling model is obtained by training according to the extended data and the first labeling image data, and is more accurate in the labeling result of the medical image data, that is, the labeling of the medical image data is performed based on the preset labeling model, so that the accuracy of the labeling result is improved.

The implementation principle and technical effect of the training and application method of the annotation model provided by the above embodiment are similar to those of the above embodiment, and are not described herein again.

It should be understood that although the various steps in the flow charts of fig. 2-9 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2-9 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least some of the other steps.

In one embodiment, as shown in fig. 10, there is provided an image annotation apparatus including:

the acquisition module 01 is used for acquiring medical image data to be labeled;

the labeling module 02 is configured to input medical image data to be labeled into a preset labeling model to obtain a labeling result of the medical image data; the preset labeling model is a model obtained by training according to a preset data set; the preset data set comprises first marked image data and second marked image data, the first marked image data are pre-marked image data, and the second marked image data are obtained by marking adjacent unmarked image data according to the first marked image data.

In one alternative embodiment, as shown in fig. 11, the image annotation device further includes a training module 03;

the training module 03 is configured to determine first labeled image data and second labeled image data according to original medical image data to obtain a preset data set; and training the initial labeling model according to a preset data set to obtain a preset labeling model.

In one optional embodiment, the training module 03 is configured to label the first original medical image data according to a user labeling instruction to obtain first labeled image data; the first original medical image data is image data with image definition greater than or equal to a first definition threshold in the original medical image data; labeling the second original medical image data according to the first labeled image data to obtain second labeled image data; the second original medical image data is image data which is not marked and has the image definition less than or equal to a second definition threshold value in the original medical image data; the upper and lower layers of adjacent images of the second original medical image data belong to the first labeled image data; the first sharpness threshold is greater than the second sharpness threshold.

In an optional embodiment, the training module 03 is configured to determine, from the first labeled image data, upper-layer first labeled image data and lower-layer first labeled image data of the second original medical image data; determining a pixel union set of the labeling contours according to the first labeling contour of the upper-layer first labeling image data and the second labeling contour of the lower-layer first labeling image data; determining foreground pixel intersection and background pixel intersection according to pixels in the upper layer first marked image data and pixels in the lower layer first marked image data; and labeling the second original medical image data according to the labeled contour pixel union set, the foreground pixel intersection set and the background pixel intersection set to obtain second labeled image data.

In one optional embodiment, the training module 03 is configured to label, as a foreground pixel, a pixel in the second original medical image data that does not belong to the labeled pixel union set and belongs to the foreground pixel intersection; and marking pixels which do not belong to the marked pixel union set and belong to the intersection of the background pixels in the second original medical image data as background pixels.

In one optional embodiment, the training module 03 is configured to input a preset data set into the initial labeling model to obtain a labeling result of the preset data set; and updating the preset data set according to the labeling result, and inputting the updated preset data set into the initial labeling model for iterative training until the initial labeling model meets the convergence condition to obtain a preset labeling model.

In one optional embodiment, the training module 03 is configured to input the preset data set into the initial labeling model, so as to obtain a foreground prediction probability of each pixel in the preset data set; pixels with a foreground prediction probability greater than or equal to a first probability threshold are labeled as foreground pixels, and pixels with a foreground prediction probability less than or equal to a second probability threshold are labeled as background pixels.

In an optional embodiment, the convergence condition includes: the difference value between the number of foreground pixels in the labeling result of the current iteration and the number of foreground pixels in the labeling result of the last iteration is smaller than a difference threshold value; and/or the number of iterations satisfies a number threshold.

For specific limitations of the image annotation device, reference may be made to the above limitations of the image annotation method, which is not described herein again. The modules in the image labeling device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, as shown in fig. 12, there is provided a training apparatus for labeling a model, including:

the first determining module 11 is configured to determine first labeled image data and unlabeled image data according to the original medical image data; the first marked image data is pre-marked image data;

the preprocessing module 12 is configured to label adjacent unlabeled image data according to the first labeled image data to obtain second labeled image data;

a second determining module 13, configured to determine a preset data set according to the first labeled image data and the second labeled image data;

the training module 14 is configured to input a preset data set into the initial labeling model for training, so as to obtain a preset labeling model; the preset labeling model is used for determining a labeling result of the medical image data to be labeled.

For the specific definition of the training device for the annotation model, reference may be made to the above definition of the training method for the annotation model, and details are not described here. The modules in the training device for labeling models can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:

acquiring medical image data to be marked;

The implementation principle and technical effect of the computer device provided by the above embodiment are similar to those of the above method embodiment, and are not described herein again.

In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:

acquiring medical image data to be marked;

The implementation principle and technical effect of the computer-readable storage medium provided by the above embodiments are similar to those of the above method embodiments, and are not described herein again.

In one embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, performs the steps of:

acquiring medical image data to be marked;

The computer program product provided by the above embodiment has similar implementation principles and technical effects to those of the above method embodiment, and is not described herein again.

determining first marked image data and unmarked image data according to the original medical image data;

and inputting the preset data set into the initial labeling model for training to obtain a preset labeling model.

and inputting the preset data set pair into the initial labeling model for training to obtain a preset labeling model.

It should be noted that, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. An image annotation method, characterized in that the method comprises:

acquiring medical image data to be marked;

inputting the medical image data to be labeled into a preset labeling model to obtain a labeling result of the medical image data; the preset labeling model is a model obtained by training according to a preset data set; the preset data set comprises first marked image data and second marked image data, the first marked image data are pre-marked image data, and the second marked image data are obtained by marking adjacent unmarked image data according to the first marked image data.

2. The method of claim 1, wherein the method for training the preset labeling model comprises:

determining the first marked image data and the second marked image data according to original medical image data to obtain the preset data set;

and training an initial labeling model according to the preset data set to obtain the preset labeling model.

3. The method of claim 2, wherein said determining said first annotated image data and said second annotated image data from raw medical image data comprises:

labeling first original medical image data according to a user labeling instruction to obtain first labeled image data; the first original medical image data is image data with image definition larger than or equal to a first definition threshold in the original medical image data;

labeling second original medical image data according to the first labeled image data to obtain second labeled image data; the second original medical image data is image data which is not marked and has image definition smaller than or equal to a second definition threshold value in the original medical image data; the adjacent upper and lower layers of images of the second original medical image data belong to the first marked image data; the first sharpness threshold is greater than the second sharpness threshold.

4. The method of claim 3, wherein said labeling second original medical image data according to said first labeled image data to obtain said second labeled image data comprises:

determining upper-layer first labeled image data and lower-layer first labeled image data of the second original medical image data from the first labeled image data;

determining foreground pixel intersection and background pixel intersection according to pixels in the upper layer first labeled image data and pixels in the lower layer first labeled image data;

5. The method according to claim 4, wherein the labeling the second original medical image data according to the labeled contour pixel union set, the foreground pixel intersection set and the background pixel intersection set to obtain the second labeled image data comprises:

labeling pixels, which do not belong to the labeled pixel union set and belong to the foreground pixel intersection, in the second original medical image data as foreground pixels;

6. The method of claim 2, wherein the training an initial labeling model according to the preset data set to obtain the preset labeling model comprises:

inputting the preset data set into the initial labeling model to obtain a labeling result of the preset data set;

and updating the preset data set according to the labeling result, inputting the updated preset data set into the initial labeling model for iterative training until the initial labeling model meets a convergence condition, and obtaining the preset labeling model.

7. The method of claim 6, wherein the inputting the preset data set into the initial labeling model to obtain a labeling result of the preset data set comprises:

inputting the preset data set into the initial labeling model to obtain the foreground prediction probability of each pixel in the preset data set;

and marking the pixels with the foreground prediction probability being larger than or equal to a first probability threshold as foreground pixels, and marking the pixels with the foreground prediction probability being smaller than or equal to a second probability threshold as background pixels.

8. A method of training a label model, the method comprising:

inputting the preset data set into an initial labeling model for training to obtain a preset labeling model; the preset labeling model is used for determining a labeling result of the medical image data to be labeled.

9. An image annotation apparatus, characterized in that the apparatus comprises:

the acquisition module is used for acquiring medical image data to be marked;

10. A training apparatus for labeling a model, the apparatus comprising:

the training module is used for inputting the preset data set into an initial labeling model for training to obtain a preset labeling model; the preset labeling model is used for determining a labeling result of the medical image data to be labeled.