CN115512116A

CN115512116A - Image segmentation model optimization method and device, electronic equipment and readable storage medium

Info

Publication number: CN115512116A
Application number: CN202211357197.5A
Authority: CN
Inventors: 程健; 易林志; 刘涛; 吴振洲; 张瑞; 付鹤; 徐红
Original assignee: Beijing Ande Yizhi Technology Co ltd
Current assignee: Beijing Ande Yizhi Technology Co ltd
Priority date: 2022-11-01
Filing date: 2022-11-01
Publication date: 2022-12-23
Anticipated expiration: 2042-11-01
Also published as: CN115512116B

Abstract

The disclosure relates to an image segmentation model optimization method, an image segmentation model optimization device, an electronic device and a readable storage medium. And determining a first loss parameter according to a preset loss function, the labeling segmentation result and the prediction segmentation result, and determining a plurality of corresponding first characteristic parameters and a plurality of corresponding second characteristic parameters according to the prediction loss function and the plurality of labeling segmentation results and prediction segmentation results respectively. And adjusting the image segmentation model by a second loss parameter determined according to the first loss parameter and the first characteristic parameter and the second characteristic parameter. According to the method, the loss parameter determined by the annotation segmentation result and the prediction segmentation result together, the first characteristic parameter and the second characteristic parameter determined respectively jointly determine the model loss, and the model is adjusted, so that the model optimization effect is improved, and the segmentation effect of the image segmentation model is further improved.

Description

Image segmentation model optimization method and device, electronic equipment and readable storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to an image segmentation model optimization method and apparatus, an electronic device, and a readable storage medium.

Background

Image segmentation is one of the basic problems in the field of computer vision and image processing, and refers to dividing an image into a plurality of non-interlaced regions according to the characteristics of color, gray scale, texture, shape and the like, and enabling the characteristics to present certain similarity in the same region and present obvious difference among different regions. The segmentation effect under the current image segmentation scene is limited by the training effect of the image segmentation model, and the better the training effect is, the better the segmentation effect of the obtained image segmentation model is. Therefore, the segmentation effect of the image segmentation model can be improved in a mode of optimizing the training effect of the model.

Disclosure of Invention

In view of the above, the present disclosure provides an image segmentation model optimization method, an image segmentation model optimization device, an electronic device, and a readable storage medium, which aim to optimize an image segmentation model by adjusting a loss function.

According to a first aspect of the present disclosure, there is provided an image segmentation model optimization method, the method comprising:

determining a plurality of sample images and corresponding annotation segmentation results;

respectively inputting the plurality of sample images into an image segmentation model for image segmentation to obtain corresponding prediction segmentation results;

determining a first loss parameter according to a preset loss function, the labeling segmentation result and the prediction segmentation result;

determining a plurality of first characteristic parameters according to the prediction loss function and the plurality of labeled segmentation results;

determining a plurality of second characteristic parameters according to the prediction loss function and a plurality of prediction segmentation results;

determining a second loss parameter according to the plurality of first characteristic parameters and the plurality of second characteristic parameters;

adjusting the image segmentation model according to the first loss parameter and the second loss parameter.

In a possible implementation manner, the determining a first loss parameter according to the preset loss function, the labeling segmentation result, and the prediction segmentation result includes:

determining the corresponding relation between the labeling segmentation result and the prediction segmentation result corresponding to the same sample image;

taking each labeled segmentation result as a first independent variable, taking a corresponding predicted segmentation result as a second independent variable, and inputting the second independent variable into the preset loss function to obtain a corresponding segmentation result difference;

and determining a first loss parameter according to the average value of the segmentation result difference.

In a possible implementation manner, the determining a plurality of first feature parameters according to the predictive loss function and a plurality of the label segmentation results includes:

acquiring two labeling segmentation results from the plurality of labeling segmentation results as a first labeling result and a second labeling result respectively to obtain corresponding labeling result pairs;

and taking the first labeling result in each labeling result pair as a first independent variable, and taking the second labeling result as a second independent variable to be input into the preset loss function to obtain a corresponding first characteristic parameter.

In a possible implementation manner, the determining a plurality of second feature parameters according to the prediction loss function and a plurality of the prediction partition results includes:

acquiring two prediction division results from the plurality of prediction division results as a first prediction result and a second prediction result respectively to obtain corresponding prediction result pairs;

and taking the first prediction result in each prediction result pair as a first independent variable, taking the second prediction result as a second independent variable, and inputting the second independent variable into the preset loss function to obtain a corresponding second characteristic parameter.

In one possible implementation manner, the determining a second loss parameter according to the plurality of first feature parameters and the plurality of second feature parameters includes:

respectively determining a first independent variable and a second independent variable corresponding to each first characteristic parameter, and a first independent variable and a second independent variable corresponding to each second characteristic parameter;

determining a first characteristic parameter and a second characteristic parameter of which the corresponding first independent variable and the corresponding second independent variable have corresponding relations as corresponding characteristic parameter groups;

and calculating the average value of the difference values of the first characteristic parameter and the second characteristic parameter in each characteristic parameter group to obtain a second loss parameter.

In one possible implementation, the adjusting the image segmentation model according to the first loss parameter and the second loss parameter includes:

calculating a weighted sum of the first loss parameter and the second loss parameter to obtain a model loss parameter;

and adjusting the image segmentation model according to the model loss parameter.

In a possible implementation manner, the weight of the first loss parameter is 1, and the weight of the second loss parameter is a matrix determined according to the number of the sample images, and each element in the matrix characterizes a relationship between two sample images. According to a second aspect of the present disclosure, there is provided an image segmentation model optimization apparatus, the apparatus comprising:

the sample determining module is used for determining a plurality of sample images and corresponding annotation segmentation results;

the result prediction module is used for respectively inputting the plurality of sample images into an image segmentation model for image segmentation to obtain corresponding prediction segmentation results;

the first loss determining module is used for determining a first loss parameter according to a preset loss function, the labeling segmentation result and the prediction segmentation result;

the first characteristic determining module is used for determining a plurality of first characteristic parameters according to the predictive loss function and the plurality of the annotation segmentation results;

a second feature determination module, configured to determine a plurality of second feature parameters according to the prediction loss function and the plurality of prediction segmentation results;

a second loss determining module, configured to determine a second loss parameter according to the plurality of first characteristic parameters and the plurality of second characteristic parameters;

and the model adjusting module is used for adjusting the image segmentation model according to the first loss parameter and the second loss parameter.

In one possible implementation manner, the preset loss function includes a first argument and a second argument, and the first loss determining module includes:

the corresponding relation determining submodule is used for determining that the labeling segmentation result and the prediction segmentation result which correspond to the same sample image are in a corresponding relation;

a result difference determining submodule, configured to take each labeled segmentation result as a first independent variable, take a corresponding predicted segmentation result as a second independent variable, and input the second independent variable into the preset loss function, so as to obtain a corresponding segmentation result difference;

and the first loss determining submodule is used for determining a first loss parameter according to the average value of the segmentation result difference.

In one possible implementation manner, the first feature determination module includes:

the first result pair determining submodule is used for acquiring two labeling segmentation results from the plurality of labeling segmentation results to respectively serve as a first labeling result and a second labeling result so as to obtain corresponding labeling result pairs;

and the first parameter determining submodule is used for taking the first labeling result in each labeling result pair as a first independent variable, taking the second labeling result as a second independent variable and inputting the second labeling result into the preset loss function to obtain a corresponding first characteristic parameter.

In one possible implementation manner, the second feature determining module includes:

a second result pair determining submodule, configured to obtain two prediction division results from the multiple prediction division results, where the two prediction division results are used as a first prediction result and a second prediction result, respectively, and obtain a corresponding prediction result pair;

and the second parameter determination submodule is used for taking the first prediction result in each prediction result pair as a first independent variable, taking the second prediction result as a second independent variable and inputting the second prediction result into the preset loss function to obtain a corresponding second characteristic parameter.

In one possible implementation manner, the second loss determining module includes:

the independent variable determining submodule is used for respectively determining a first independent variable and a second independent variable corresponding to each first characteristic parameter, and a first independent variable and a second independent variable corresponding to each second characteristic parameter;

the parameter set determining submodule is used for determining a first characteristic parameter and a second characteristic parameter of which the corresponding first independent variable and the corresponding second independent variable have a corresponding relationship as corresponding characteristic parameter sets;

and the second loss determining submodule is used for calculating the average value of the difference values of the first characteristic parameter and the second characteristic parameter in each characteristic parameter group to obtain a second loss parameter.

In one possible implementation, the model adjustment module includes:

the loss parameter determination submodule is used for calculating the weighted sum of the first loss parameter and the second loss parameter to obtain a model loss parameter;

and the model adjusting submodule is used for adjusting the image segmentation model according to the model loss parameter.

In a possible implementation manner, the weight of the first loss parameter is 1, and the weight of the second loss parameter is a matrix determined according to the number of the sample images, and each element in the matrix characterizes a relationship between two sample images.

According to a third aspect of the present disclosure, there is provided an electronic device comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described method when executing the memory-stored instructions.

According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer program instructions, wherein the computer program instructions, when executed by a processor, implement the above-described method.

According to a fifth aspect of the disclosure, there is provided a computer program product comprising computer readable code, or a non-transitory computer readable storage medium carrying computer readable code, which when run in a processor of an electronic device, the processor in the electronic device performs the above method.

In the embodiment of the present disclosure, a plurality of sample images and corresponding labeled segmentation results are determined, and the plurality of sample images are respectively input into an image segmentation model to obtain corresponding predicted segmentation results. And determining a first loss parameter according to a preset loss function, the labeling segmentation result and the prediction segmentation result, and determining a plurality of corresponding first characteristic parameters and a plurality of corresponding second characteristic parameters according to the prediction loss function and the plurality of labeling segmentation results and prediction segmentation results respectively. And adjusting the image segmentation model by a second loss parameter determined according to the first loss parameter and the first characteristic parameter and the second characteristic parameter. According to the image segmentation method and device, the loss parameter determined by the annotation segmentation result and the prediction segmentation result together, the model loss determined by the first characteristic parameter and the second characteristic parameter respectively, and the model adjustment are performed, so that the model optimization effect is improved, and the segmentation effect of the image segmentation model is further improved.

Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

FIG. 1 shows a flow diagram of a method of image segmentation model optimization according to an embodiment of the present disclosure;

FIG. 2 illustrates a schematic diagram of a model training process in accordance with an embodiment of the present disclosure;

FIG. 3 illustrates a schematic diagram of determining a second loss parameter in accordance with an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of an image segmentation model optimization apparatus according to an embodiment of the present disclosure;

FIG. 5 shows a schematic diagram of an electronic device according to an embodiment of the present disclosure;

fig. 6 shows a schematic diagram of another electronic device according to an embodiment of the present disclosure.

Detailed Description

Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the subject matter of the present disclosure.

In a possible implementation manner, the image segmentation model optimization method according to the embodiment of the present disclosure may be executed by an electronic device, such as a terminal device or a server. The terminal device may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, or other fixed or mobile terminal devices, and the server may be a single server or a server cluster formed by multiple servers. The electronic device may implement the image segmentation model optimization method by way of the processor invoking computer readable instructions stored in the memory. Optionally, the embodiments of the present disclosure may be applied to an application scenario in which any image segmentation model is optimized.

Fig. 1 shows a flow chart of an image segmentation model optimization method according to an embodiment of the present disclosure. As shown in fig. 1, the image segmentation model optimization method of the embodiment of the present disclosure may include the following steps S10 to S70.

And S10, determining a plurality of sample images and corresponding annotation segmentation results.

In one possible implementation, the electronic device determines a plurality of sample images and annotation segmentation results corresponding to each sample image for optimizing an image segmentation model. The type and format of the sample image can be determined according to the application scene of the image segmentation model. For example, when the image segmentation model is used for lesion segmentation in a medical image, the sample image may include any medical image such as a computed tomography image, a positron emission tomography image, an ultrasound image, and magnetic resonance imaging. When the image segmentation model is used for the identification segmentation of an animal or a human object in an image, the sample image may comprise an object image with the animal or the human object.

Optionally, the annotation segmentation result corresponding to each sample image may be predetermined by automatic annotation or manual annotation. For example, in the case where the sample image is a medical image, the labeling segmentation result may be information obtained by labeling the position of a lesion in the medical image for a doctor. Under different application scenarios, the content included in the annotation segmentation result is different. For example, when the sample image is a medical image, the annotation segmentation result may include lesion position information. When the sample image is the object image, the annotation segmentation result may include position information of a position where the object is located in the image.

And S20, respectively inputting the sample images into an image segmentation model for image segmentation to obtain corresponding prediction segmentation results.

In a possible implementation manner, after determining a plurality of sample images, the electronic device inputs each sample image into the image segmentation model to perform image segmentation, so as to obtain a corresponding prediction segmentation result. The prediction segmentation result is determined according to an application scene of the image segmentation model, wherein the prediction segmentation result comprises at least one segmentation region. For example, when the sample image is a medical image and the image segmentation model is the same as the lesion identification segmentation, the prediction segmentation result includes a position where a lesion may exist in the sample image as a segmentation region. When the sample image is a target image and the image segmentation model is used for object recognition segmentation, the prediction segmentation result includes a position where an object may exist in the sample image as a segmentation region. Optionally, the predicted segmentation result may further include probability values of possible image segmentation positions, such as a lesion position and an object position, which indicate the possibility of image segmentation at the position.

Optionally, after obtaining the annotation segmentation result and the prediction segmentation result corresponding to the plurality of sample images, the electronic device may perform model optimization through a difference between the annotation segmentation result and the prediction segmentation result.

FIG. 2 shows a schematic diagram of a model training process according to an embodiment of the present disclosure. As shown in fig. 2, after determining a plurality of sample images 20 and annotation segmentation results 23 corresponding to each sample image 20, the electronic device inputs the sample images 20 into an image segmentation model 21, and performs image segmentation on the input sample images 20 by the image segmentation model 21 to obtain corresponding prediction segmentation results 22. Further, a model loss 24 is determined according to the corresponding prediction segmentation result 22 and the labeling segmentation result 23 of each sample image 20, and then the image segmentation model 21 is adjusted according to the model loss to perform model optimization.

And S30, determining a first loss parameter according to a preset loss function, the labeling segmentation result and the prediction segmentation result.

In one possible implementation manner, after determining the annotation segmentation result and the prediction segmentation result corresponding to the plurality of sample images, a model loss for adjusting the image segmentation model may be determined according to the annotation segmentation result and the prediction segmentation result. The model loss in embodiments of the present disclosure may include at least one loss parameter. Optionally, the electronic device may first determine a first loss parameter as one of the model losses according to a preset loss function, the labeled segmentation result, and the predicted segmentation result, so as to further determine the model loss together with other loss parameters. Wherein the first loss parameter is used for characterizing the difference between the real segmentation result and the predicted segmentation result of the sample image. The preset loss function is at least one loss function preset by the electronic equipment, and can be any function capable of optimizing the model. For example, at least one of a Dice loss function (Dice loss function) and a cross entropy loss function may be included.

Optionally, the preset loss function may include a first argument and a second argument, and the first argument and the second argument are used to respectively characterize the annotation segmentation result and the predictive segmentation result corresponding to the sample image when the first loss parameter is determined, that is, the annotation segmentation result and the predictive segmentation result corresponding to the same sample image are respectively taken as the first argument and the second argument to be brought into the preset loss function to determine the first loss parameter. The electronic equipment firstly determines that the labeling segmentation result corresponding to the same sample image and the prediction segmentation result are in a corresponding relation, each labeling segmentation result is used as a first independent variable, the corresponding prediction segmentation result is used as a second independent variable and is input into a preset loss function, and corresponding segmentation result differences are obtained. And determining a first loss parameter according to the average value of the difference of the segmentation results.

In one possible implementation, in determining the first loss parameter, the first argument in the predetermined loss function is y _i The second independent variable is p _i And y is y in the case of N pixel points in the sample image _i And p _i The probability that the pixel i belongs to the segmented region in the annotated segmentation result and the predicted segmentation result can be characterized respectively. Wherein, y _i Characterizing a corresponding pixel i in the annotated segmentation result as 1 in the positive sample region，y _i A value of 0 characterizes the corresponding pixel i as being in the negative sample region. p is a radical of _i And is a value between 0 and 1, and represents the probability value of the corresponding pixel i in the predicted segmentation result in the segmentation region. The electronic equipment can substitute the probability value representing that each pixel of the sample image belongs to the segmentation region in the annotation segmentation result and the prediction segmentation result into y as a first independent variable _i And the second argument is p _i And obtaining the difference of the segmentation results, and further calculating the average value of the difference of the segmentation results corresponding to each sample image to obtain a first loss parameter.

Optionally, the Dice loss function determines a loss of the image segmentation model according to a degree of overlap between the annotated segmentation result and the predicted segmentation result. Wherein the corresponding segmentation region in the segmentation result is labeled as a positive sample region, and the rest of the background region is labeled as a negative sample region. Further, an overlapping region of the corresponding divided region in the prediction division result and the positive sample region is set as a true positive rate TP, an overlapping region of the background region other than the divided region in the prediction division result and the positive sample region is set as a false negative rate FN, an overlapping region of the divided region in the prediction division result and the negative sample region is set as a false positive rate FP, and an overlapping region of the background region other than the divided region in the prediction division result and the negative sample region is set as a true negative rate TN. The similarity DiceScore of the labeling segmentation result and the prediction segmentation result can be determined based on the formula (1) through the true positive rate TP, the false positive rate FP, the false negative rate FN, and the true negative rate TN.

In the case where the sample image includes N pixel points, the formula (1) may be further converted into the formula (2) based on the above contents to perform similarity calculation. Further, calculating to obtain model dice loss L according to the similarity _Dice 。

Optionally, the cross entropy loss function determines the loss of the model according to the distribution of each pixel in the sample image in the corresponding pi in the prediction segmentation result. The function is determined according to equation (4).

Further, when the preset loss function simultaneously includes the cross entropy loss function and the Dice loss function, a weighted sum of the cross entropy loss function and the Dice loss function may be calculated to determine the preset loss function, and the labeling segmentation result and the prediction segmentation result corresponding to each sample image are substituted into the prediction loss function to determine the first loss parameter.

And S40, determining a plurality of first characteristic parameters according to the prediction loss function and the plurality of annotation segmentation results. In one possible implementation, after determining the first loss parameter, the electronic device may further determine a second loss parameter to determine a model loss of the image segmentation model together according to the first loss parameter and the second loss parameter for model optimization. Wherein the second loss parameter may characterize a difference between different annotated segmentation results and a difference between different matched predicted segmentation results. Therefore, the electronic device may determine a difference between different labeled segmentation results as a first feature parameter, determine a difference between different predicted segmentation results as a second feature parameter, and then determine a second loss parameter according to the first feature parameter and the second feature parameter.

Optionally, the process of determining the first characteristic parameter by the electronic device may include obtaining two annotation segmentation results from the multiple annotation segmentation results as a first annotation result and a second annotation result, respectively, to obtain a corresponding annotation result pair. And taking the first labeling result in each labeling result pair as a first independent variable, and taking the second labeling result as a second independent variable to input a preset loss function to obtain a corresponding first characteristic parameter. Optionally, the first annotation result and the second annotation result in the annotation result pair determined by the electronic device have a corresponding order, that is, the two annotation segmentation results are sequentially used as an annotation result pair obtained by the first annotation result and the second annotation result, and the annotation result pairs sequentially used as the second annotation result and the first annotation result are different.

In the process of determining the first characteristic parameter, a first argument in the preset loss function is determined as y _i The second independent variable is y _j And y is y in the case of N pixel points in the sample image _i And y _j The probabilities of pixels i and j belonging to the segmented regions in the segmentation result of the label can be characterized respectively. Wherein, y _i Or y _j When 1, the corresponding pixel in the segmentation result is labeled as being in the positive sample region, y _i Or y _j A value of 0 characterizes the corresponding pixel as being in the negative sample region. The electronic equipment can substitute the probability value representing that each pixel of the sample image belongs to the segmentation region in the first labeling result and the second labeling result in each labeling result pair into the first argument y _i And the second argument is y _j And obtaining a corresponding first characteristic parameter. Namely, when the preset loss function is a Dice loss function, the first characteristic parameter can be obtained through calculation of the formula (5), and when the preset loss function is a cross entropy loss function, the first characteristic parameter can be obtained through calculation of the formula (6). When the preset loss function simultaneously comprises the cross entropy loss function and the Dice loss function, the labeling result pairs can be respectively substituted into a calculation formula (5) and a formula (6) to calculate, weigh and determine.

And S50, determining a plurality of second characteristic parameters according to the prediction loss function and the plurality of prediction division results.

In a possible implementation manner, the electronic device may determine the second feature parameter according to a prediction loss function and the prediction segmentation result while determining the first feature parameter. Alternatively, the process of determining the second characteristic parameter by the electronic device may be similar to the process of determining the first characteristic parameter. That is, the electronic device may obtain two prediction division results from the plurality of prediction division results as the first prediction result and the second prediction result, respectively, to obtain corresponding pairs of prediction results. And taking the first prediction result in each prediction result pair as a first independent variable, taking the second prediction result as a second independent variable, and inputting the second independent variable into a preset loss function to obtain a corresponding second characteristic parameter. Optionally, the first prediction result and the second prediction result in the prediction result pair determined by the electronic device have a corresponding order, that is, the prediction result pair obtained as the first prediction result and the second prediction result in turn in the two prediction division results is different from the prediction result pair obtained as the second prediction result and the first prediction result in turn.

In the process of determining the second characteristic parameter, determining the first argument in the preset loss function as p _i The second independent variable is p _j And p is p under the condition that the sample image comprises N pixel points _i And p _j The probabilities of the predicted segmentation result in which pixel i and pixel j belong to the segmented region can be characterized separately. Wherein p is _i Or p _j A value of 1 characterizes the corresponding pixel in the predictive segmentation result as a positive sample region, p _i Or p _j A value of 0 characterizes the corresponding pixel as being in the negative sample region. The electronic device may substitute a probability value representing that each pixel of the sample image belongs to the segmentation region in the first prediction result and the second prediction result in each prediction result pair into the first argument as p _i And the second argument is p _j And obtaining a corresponding second characteristic parameter. Namely, when the preset loss function is the Dice loss function, the second characteristic parameter can be obtained through calculation of the formula (7), and when the preset loss function is the cross entropy loss function, the second characteristic parameter can be obtained through calculation of the formula (8). When the preset loss function comprises the cross entropy loss function and the Dice loss function at the same time, the prediction result pair can be respectively substituted into a calculation formula (7) and a formula (8) to calculate the weighted sumAnd (4) determining.

And S60, determining a second loss parameter according to the plurality of first characteristic parameters and the plurality of second characteristic parameters.

In a possible implementation manner, after determining first feature parameters corresponding to a plurality of standard result pairs and second feature parameters corresponding to a plurality of predicted result pairs, the electronic device determines a second loss parameter according to the plurality of first feature parameters and the plurality of second feature parameters. The process for the electronic device to determine the second loss parameter may include: first, a first independent variable and a second independent variable corresponding to each first characteristic parameter, and a first independent variable and a second independent variable corresponding to each second characteristic parameter are respectively determined. And determining a first characteristic parameter and a second characteristic parameter of which the corresponding first independent variable and the corresponding second independent variable have corresponding relations as corresponding characteristic parameter groups, and calculating the average value of the difference values of the first characteristic parameter and the second characteristic parameter in each characteristic parameter group to obtain a second loss parameter. The first independent variable and the second independent variable which have the corresponding relation are respectively an annotation segmentation result and a prediction segmentation result which correspond to the same sample image.

Alternatively, the electronic device may obtain the second loss parameter L by calculating according to the following formula (9) _Pair . Wherein N is _pair And the quantity of the characteristic parameter sets is Y, the labeled segmentation result corresponding to the sample image is Y, the predicted segmentation result corresponding to the sample image is P, and the L is a preset loss function. i and j are each 0-N _pair A value in between. Namely, when the preset loss function is a Dice loss function, the second loss parameter can be obtained through calculation in the formula (10), and when the preset loss function is a cross entropy loss function, the second loss parameter can be obtained through calculation in the formula (11).

Fig. 3 illustrates a schematic diagram of determining a second loss parameter in accordance with an embodiment of the disclosure. As shown in fig. 2, the electronic device can determine that the annotation segmentation result belongs to the Y domain and the prediction segmentation result belongs to the P domain. After determining the labeled segmentation result and the predicted segmentation result corresponding to a plurality of sample images and adding the labeled segmentation result and the predicted segmentation result into the Y domain and the P domain respectively, any two labeled segmentation results Y can be obtained from the Y domain _i And Y _j Determining a pair of labeling results, and acquiring two corresponding prediction segmentation results P from the P domain _i And P _j And determining a prediction result pair matched with the labeling result pair. The electronic device may input the annotation result pair and the matched prediction result pair into a preset loss model to calculate corresponding first loss parameters and second loss parameters, and then calculate an average value of differences between the first loss parameters of all the annotation result pairs and the second loss parameters of the matched prediction results to obtain second loss parameters.

And S70, adjusting the image segmentation model according to the first loss parameter and the second loss parameter.

In a possible implementation manner, after determining the first loss parameter and the second loss parameter, the electronic device determines a model loss parameter according to both the first loss parameter and the second loss parameter, and adjusts the image segmentation model according to the model loss parameter. The model loss parameter represents the overall loss of the image segmentation model and can be determined by calculating the weighted sum of the first loss parameter and the second loss parameter. That is, the electronic device may calculate a weighted sum of the first loss parameter and the second loss parameter to obtain the model loss parameter. And adjusting the image segmentation model according to the model loss parameter.

Alternatively, the weights of the first loss parameter and the second loss parameter may be set in advance. For example, the first loss parameter may be determined to have a weight of 1, and the second loss parameter may be determined to have a weight of a matrix determined according to the number of sample images. Wherein each element in the matrix characterizes a relationship between two sample images. Namely, when the preset loss function is a Dice loss function, the model loss parameter can be obtained through calculation of the formula (12), and when the preset loss function is a cross entropy loss function, the model loss parameter can be obtained through calculation of the formula (13).

L＝L _Dice +λL _Pair-Dice (12)

L＝L _CE +λL _Pair-CE (13)

Where λ is a weight matrix of the second loss parameter. λ is a matrix of NxN when the number of sample images is N, each element l in the matrix _ij The representation considers the relationship between sample i and sample j. Diagonal element l ₁₁ To l _nn Showing the relationship between the interior of the sample, i.e., L _Dice Or L _CE The loss function calculates the resulting result. Diagonal elements play a main role in the model training process, and elements in the upper triangular matrix play an auxiliary role. In the case of determining the value of each element in the matrix, the diagonal elements may be summed to obtain an average value, while the upper triangular elements excluding the diagonal elements may be summed to obtain an average value. Further, a suitable proportional relationship is selected between the two calculated averages. Further, when the number of sample images is too large, the complexity of the process of calculating the correlation between the samples is high, which may cause the practical training period to be long, and M (M) may be randomly selected among the samples<N) sample pairs are determined as a matrix of weights, thereby reducing the amount of calculation and shortening the training cycle time.

Based on the technical features, the embodiments of the present disclosure can combine the existing Dice loss function or cross entropy loss function to obtain a new model loss determination mode by using the mutual relationship between samples. The model loss determining mode increases the precision of the obtained model loss parameters, the optimization effect of the model is improved through the process of training the model by the model loss parameters, and then the segmentation effect of the image segmentation model is improved.

In some embodiments, functions of or modules included in the apparatus provided in the embodiments of the present disclosure may be used to execute the method described in the above method embodiments, and specific implementation thereof may refer to the description of the above method embodiments, and for brevity, will not be described again here.

Embodiments of the present disclosure also provide a computer-readable storage medium, on which computer program instructions are stored, and when executed by a processor, the computer program instructions implement the above method. The computer readable storage medium may be a volatile or non-volatile computer readable storage medium.

An embodiment of the present disclosure further provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described method when executing the memory-stored instructions.

The disclosed embodiments also provide a computer program product comprising computer readable code or a non-transitory computer readable storage medium carrying computer readable code, which when run in a processor of an electronic device, the processor in the electronic device performs the above method.

Fig. 4 shows a schematic diagram of an image segmentation model optimization apparatus according to an embodiment of the present disclosure. As shown in fig. 4, the image segmentation model optimization apparatus according to the embodiment of the present disclosure may include:

a sample determination module 40, configured to determine a plurality of sample images and corresponding annotation segmentation results;

a result prediction module 41, configured to input the sample images into an image segmentation model respectively for image segmentation, so as to obtain corresponding prediction segmentation results;

a first loss determining module 42, configured to determine a first loss parameter according to a preset loss function, the labeled segmentation result, and the predicted segmentation result;

a first feature determining module 43, configured to determine a plurality of first feature parameters according to the predictive loss function and the plurality of labeled segmentation results;

a second feature determining module 44, configured to determine a plurality of second feature parameters according to the prediction loss function and the plurality of prediction partition results;

a second loss determining module 45, configured to determine a second loss parameter according to the plurality of first characteristic parameters and the plurality of second characteristic parameters;

a model adjustment module 46 configured to adjust the image segmentation model according to the first loss parameter and the second loss parameter.

In a possible implementation manner, the preset loss function includes a first argument and a second argument, and the first loss determining module 42 includes:

In a possible implementation manner, the first feature determining module 43 includes:

the first result pair determining submodule is used for acquiring two labeling segmentation results from the plurality of labeling segmentation results and respectively using the two labeling segmentation results as a first labeling result and a second labeling result to obtain a corresponding labeling result pair;

In a possible implementation manner, the second feature determining module 44 includes:

and the second parameter determining submodule is used for taking the first prediction result in each prediction result pair as a first independent variable, taking the second prediction result as a second independent variable and inputting the second prediction result into the preset loss function to obtain a corresponding second characteristic parameter.

In a possible implementation manner, the second loss determining module 45 includes:

In one possible implementation, the model adjustment module 46 includes:

Fig. 5 shows a schematic diagram of an electronic device according to an embodiment of the disclosure. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.

Referring to fig. 5, electronic device 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communications component 816.

The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations at the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the electronic device 800 is in an operation mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the electronic device 800. For example, the sensor assembly 814 may detect an open/closed state of the electronic device 800, the relative positioning of components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, orientation or acceleration/deceleration of the electronic device 800, and a change in temperature of the electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 may access a wireless network based on a communication standard, such as WiFi,2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic components for performing the above-described methods.

In an exemplary embodiment, a non-transitory computer-readable storage medium, such as the memory 804, is also provided that includes computer program instructions executable by the processor 820 of the electronic device 800 to perform the above-described methods.

Fig. 6 shows a schematic diagram of another electronic device according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server or terminal device. Referring to fig. 6, electronic device 1900 includes a processing component 1922 further including one or more processors and memory resources, represented by memory 1932, for storing instructions, e.g., applications, executable by processing component 1922. The application programs stored in memory 1932 may include one or more modules that each correspond to a set of instructions. Further, the processing component 1922 is configured to execute instructions to perform the above-described method.

The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system stored in memory 1932, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, or the like.

In an exemplary embodiment, a non-transitory computer readable storage medium, such as the memory 1932, is also provided that includes computer program instructions executable by the processing component 1922 of the electronic device 1900 to perform the above-described methods.

The present disclosure may be systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for causing a processor to implement various aspects of the present disclosure.

The computer-readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for optimizing an image segmentation model, the method comprising:

2. The method of claim 1, wherein the predetermined loss function comprises a first argument and a second argument, and wherein determining a first loss parameter based on the predetermined loss function, the annotated segmentation result, and the predicted segmentation result comprises:

3. The method of claim 2, wherein determining a plurality of first feature parameters according to the predictive loss function and a plurality of the annotation segmentation results comprises:

4. The method of claim 3, wherein determining a plurality of second feature parameters according to the predictive loss function and a plurality of the predictive partitions comprises:

5. The method of claim 4, wherein determining a second loss parameter based on the first plurality of characteristic parameters and the second plurality of characteristic parameters comprises:

6. The method according to any one of claims 1-5, wherein said adapting the image segmentation model according to the first loss parameter and the second loss parameter comprises:

7. The method according to claim 6, wherein the first loss parameter is weighted by 1, and the second loss parameter is weighted by a matrix determined according to the number of sample images, each element in the matrix characterizing a relationship between two of the sample images.

8. An apparatus for optimizing an image segmentation model, the apparatus comprising:

the first characteristic determining module is used for determining a plurality of first characteristic parameters according to the predictive loss function and the plurality of the label segmentation results;

a second feature determining module, configured to determine a plurality of second feature parameters according to the predictive loss function and the plurality of predictive segmentation results;

a second loss determination module, configured to determine a second loss parameter according to the plurality of first characteristic parameters and the plurality of second characteristic parameters;

9. An electronic device, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to implement the method of any one of claims 1 to 7 when executing the memory-stored instructions.

10. A non-transitory computer readable storage medium having stored thereon computer program instructions, wherein the computer program instructions, when executed by a processor, implement the method of any one of claims 1 to 7.