WO2021135552A1

WO2021135552A1 - Segmentation effect assessment method and apparatus based on deep learning, and device and medium

Info

Publication number: WO2021135552A1
Application number: PCT/CN2020/123255
Authority: WO
Inventors: 史鹏; 刘莉红; 刘玉宇; 肖京
Original assignee: 平安科技（深圳）有限公司
Priority date: 2020-06-28
Filing date: 2020-10-23
Publication date: 2021-07-08
Also published as: CN111753843A

Abstract

The present application relates to artificial intelligence and deep learning. Disclosed are a segmentation effect assessment method and apparatus based on deep learning, and a device and a storage medium. The method comprises: acquiring a vehicle image, and acquiring segmented images according to the vehicle image and a pre-trained component segmentation model; creating training labels according to features of the segmented images, wherein the training labels comprise good segmentation effect labels indicating that the segmented images have the features of conforming to a preset shape and/or being uniform in color, and bad segmentation effect labels indicating that the segmented images have the features of having dots and/or color crossover; labeling the segmented images according to the training labels, so as to obtain labeled images; acquiring a target segmentation effect assessment model according to a constructed deep learning model and the labeled images; and acquiring an image to be tested, and acquiring an assessment result according to the image to be tested and the target segmentation effect assessment model. By means of the segmentation effect assessment method proposed in the present application, the segmentation effect of a component segmentation model can be assessed, and an image that fails to be segmented by the segmentation model can be filtered out.

Description

Method, device, equipment and medium for evaluating segmentation effect based on deep learning

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on June 28, 2020, the application number is 202010599352.9, and the application title is "Deep learning-based segmentation effect evaluation method, device, equipment and medium", and its entire content Incorporated in this application by reference.

Technical field

This application relates to the technical field of artificial intelligence and deep learning, and in particular to a method, device, computer equipment, and readable storage medium for evaluating segmentation effects based on deep learning.

To

Background technique

With the rapid increase in the number of cars, traffic collisions also occur frequently, so the problem of vehicle damage is also increasing. For this reason, insurance companies have successively launched auto insurance services to protect the safety of the people’s vehicles and property. When the insured vehicle is involved in a traffic accident, the insurance company is required to arrange for the damage of the damaged vehicle to be assessed and based on the maximum damage of the vehicle. The amount of auto insurance claims is determined based on circumstances, and auto insurance loss determination is a key link in claims.

At present, the judgment of vehicle damage mainly relies on manual estimation. The damage assessor conducts on-site investigation and judgment at the scene of the vehicle accident. Manual classification by the damage assessor will take a lot of time, not only requires a lot of labor costs, but also is inefficient. , It is not conducive to the rapid realization of auto insurance claims, and the damage assessor needs to classify the different images collected and distinguish the damage of the components according to the images. It is easily affected by various subjective factors and cannot guarantee the damage assessment of the vehicle damage. Accuracy.

Therefore, in order to solve the above-mentioned problems, the application of computer vision image recognition and semantic segmentation technology to the scene of vehicle damage assessment came into being. In the prior art, the usual process is: acquiring a picture of a vehicle through an image acquisition tool, and then using a computer to automatically recognize the picture and semantic segmentation technology to reflect the damage condition of the vehicle, and to judge the damage condition of the vehicle. However, in the current technology, the applicant realizes that due to the complexity of the actual scene and the robustness of the semantic segmentation model, the process of image segmentation by the part segmentation model may cause segmentation failure, and the part segmentation is the subsequent part recognition. Based on the part segmentation model, if the segmentation effect of the segmentation model cannot be accurately obtained, it may affect the subsequent accurate assessment of the damaged features of the various parts of the vehicle. In this way, there may be a problem of inconsistent damage results, which will cause vehicle users Or insurance companies bring economic losses. Therefore, the evaluation of the effect of vehicle component segmentation needs to be further improved.

To

Summary of the invention

The technical problem to be solved by this application is to provide a segmentation effect evaluation method, device, computer equipment and readable storage medium based on deep learning for the problem that segmentation failures may occur in the segmented image of the vehicle components in the prior art. Achieve accurate assessment of the segmentation effect of the part segmentation model and filter the invalid images.

The first aspect of the present application provides a segmentation effect evaluation method, the method includes:

Acquiring a vehicle picture, and acquiring a segmented picture according to the vehicle picture and the pre-trained component segmentation model;

Create training tags based on the features of the segmented pictures, the training tags include tags with good segmentation effect and tags with poor segmentation effect; wherein, the segmentation effect is good means that the segmented picture has a predetermined shape and/or uniform color. Feature, the poor segmentation effect means that the segmented picture has the feature of flower dots and/or color crossing;

Labeling the segmented picture according to the label with good segmentation effect and the label with poor segmentation effect, to obtain an annotated picture;

Constructing a deep learning model, and obtaining a target segmentation effect evaluation model according to the deep learning model and the labeled picture;

Obtain a picture to be tested that has undergone component segmentation, and obtain an evaluation result of the picture to be tested according to the picture to be tested and the target segmentation effect evaluation model.

A second aspect of the present application provides a segmentation effect evaluation device, which includes:

The first acquisition module is configured to acquire a vehicle picture, and acquire a segmented picture according to the vehicle picture and the pre-trained component segmentation model;

The label creation module is used to create training labels based on the features of the segmented picture. The training labels include labels with good segmentation effect and labels with poor segmentation effect. The good segmentation effect means that the segmented picture has a shape conforming to a preset shape. And/or the feature of uniform color, the poor segmentation effect means that the segmented picture has the feature of flower dots and/or color crossing;

An annotation module, configured to annotate the segmented picture according to the label with good segmentation effect and the label with poor segmentation effect, to obtain the annotated picture;

The second acquisition module is configured to construct a deep learning model, and acquire a target segmentation effect evaluation model according to the deep learning model and the labeled picture;

The evaluation result module is used to obtain the picture to be tested for which the component segmentation has been performed, and obtain the evaluation result of the picture to be tested according to the picture to be tested and the target segmentation effect evaluation model.

A third aspect of the present application provides a computer device, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor executes the computer-readable instruction The following steps are implemented when ordering:

The fourth aspect of the present application provides one or more readable storage media storing computer readable instructions. When the computer readable instructions are executed by one or more processors, the one or more processors perform the following steps :

To

Beneficial effect

The above-mentioned deep learning-based segmentation effect evaluation method, device, computer equipment and readable storage medium are implemented in one of the solutions: obtaining segmentation pictures generated by the pre-training component segmentation model; creating training labels based on the features of the segmentation pictures, Training tags include labels with good segmentation effect and labels with poor segmentation effect. A good segmentation effect means that the segmented image has features that conform to the preset shape and/or color uniformity, and a poor segmentation effect means that the segmentation image has flowers and/or color intersections. According to the label of good segmentation effect and the label of poor segmentation effect, label the segmented image to obtain the labeled image; construct the deep learning model to obtain the target segmentation effect evaluation model according to the deep learning model and the labeled image; obtain the component segmentation For the picture to be tested, the evaluation result of the picture to be tested is obtained according to the picture to be tested and the target segmentation effect evaluation model. This application is based on the deep learning segmentation effect evaluation method, creates training labels based on the features of the pre-trained component segmentation model segmentation picture, and combines the deep learning model for testing and training to obtain the target segmentation effect evaluation model, which can achieve accurate evaluation of the component segmentation model The segmentation effect and filter segmentation failure pictures.

The details of one or more embodiments of the present application are presented in the following drawings and description, and other features and advantages of the present application will become apparent from the description, drawings and claims.

To

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

FIG. 1 is a schematic flowchart of a segmentation effect evaluation method in an embodiment of the present application;

Fig. 2 is a schematic diagram of a feature with good segmentation effect in an embodiment of the present application;

3 is a schematic diagram of a feature with poor segmentation effect in an embodiment of the present application;

4 is a schematic diagram of a process for obtaining a target segmentation effect evaluation model in an embodiment of the present application;

FIG. 5 is a schematic diagram of a process for obtaining an evaluation result of obtaining a picture to be tested in an embodiment of the present application;

FIG. 6 is a schematic diagram of a structure of a segmentation effect evaluation device in an embodiment of the present application;

FIG. 7 is a schematic diagram of a structure of a computer device in an embodiment of the present application.

To

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The deep learning-based segmentation effect evaluation method provided by the embodiment of the present application can be applied to a scene for segmentation of vehicle parts. Specifically, as shown in FIG. 1, it may include the following steps S10-S50:

S10: Obtain a vehicle picture, and obtain a segmented picture according to the vehicle picture and the pre-trained component segmentation model.

In one embodiment, a vehicle picture is acquired, and segmentation pictures are acquired according to the vehicle picture and the pre-trained component segmentation model. Specifically, a certain number of vehicle pictures may be acquired first, for example, 200,000 vehicle pictures are acquired, and the acquired 20 Ten thousand vehicle pictures are input to the pre-trained component segmentation model to obtain segmentation pictures based on vehicle pictures. Of course, more vehicle pictures such as 500,000 vehicle pictures can be obtained, which is not limited here. In this embodiment, it can be understood that the pre-trained component segmentation model is a vehicle component segmentation model that has been pre-trained, and the corresponding segmentation picture can be obtained according to the vehicle picture and the pre-trained component segmentation model.

S20: Create training labels based on the features of the segmented image. The training labels include labels with good segmentation effect and labels with poor segmentation effect. A good segmentation effect means that the segmented image has features that conform to the preset shape and/or color, and the segmentation effect is poor. It means that the segmented picture has the characteristics of flower dots and/or color crossing.

In an application scenario, based on the complexity of the actual traffic scene and the robustness of the part segmentation model, the process of segmenting pictures through the pre-trained part segmentation model may have the problem of image segmentation failure. It is understandable that if the part segmentation model is When the problem of image segmentation failure occurs in the process of segmenting pictures, the segmented images generated by the component segmentation model correspond to different features. The features can include, for example, shape features and/or color features. In this case, the shape feature of the segmented image can be used. And/or different features such as color features to create training labels. Among them, the color feature can be that the part segmentation model is set in advance to output different colors according to different parts of the vehicle. In one embodiment, training labels can be created according to the characteristics of the segmented picture. Specifically, training labels can be divided into labels including good segmentation effect labels and poor segmentation effect labels.

Based on the features of the segmented pictures, in one embodiment, a good segmentation effect may be that the segmented pictures generated by the component segmentation model have features that conform to a preset shape and/or features that are uniform in color. Among them, the feature that conforms to the preset shape can be conformity to the contour of each preset component or the feature of the preset area of each component. It can be understood that the segmented image of the vehicle involves different components. When the area of a certain component in the segmented image conforms to the preset component When the contour, or the area enclosed by the contour of the component or the features of other preset areas, etc., it can be considered that the segmented picture has features that conform to the preset shape. The feature of uniform color may be that the color of a part in the segmented picture has the feature of uniform color and/or single color. Specifically, as shown in Figure 2, where A and B can respectively represent different parts of the vehicle and display different colors, for example, A represents the door of the vehicle and displays dark green, and B represents the leaf of the vehicle and displays light green. At this point, it can be based on whether the actual segmented image meets the features of the preset shape and/or the uniform color. If the segmented image meets the features of the preset shape and/or the feature of uniform color, it can be determined that the segmented image has a good segmentation effect. It can also be determined that the segmented picture shown in FIG. 2 has a feature of good segmentation effect.

In one embodiment, the poor segmentation effect means that the segmented picture generated by the component segmentation model has the feature of flower dots and/or the feature of color crossing. Among them, the feature with dots can be that the area involved in a certain part in the segmented picture has the features of dots in the picture. Specifically, based on the various parts of the vehicle, the contour of the vehicle can also be used as the shape feature. If the contour of a certain part appears Features such as dots or jaggedness, or spots or dots in the area enclosed by the contour of a part, can be considered to have poor segmentation effect of the segmented image; features with color intersections can be displayed for the area involved in a part of the segmented image A variety of different colors, or cross fusion of different colors, it is understandable that based on the above-mentioned color features, the pre-trained part segmentation model is set in advance, that is, different colors can be output according to different parts of the vehicle. Normally, segmentation A certain part in the picture corresponds to the same color. If multiple colors appear at this time, or when the colors intersect, it can be considered that the segmentation effect of the current segmentation picture is poor. Exemplarily, as shown in Fig. 3, for example, C can represent the window of a vehicle, C1 represents the window display in gray; D can represent the leaf panel of the vehicle, C2 represents the leaf panel displays dark red, and C3 represents the leaf panel displays blue. C4 represents the leaf panel showing light red; for example, E represents a certain part of the vehicle, C5 represents a certain part showing yellow, C6 represents a certain part showing pink, C7 represents a certain part showing black, and C8 represents a certain part showing purple. Among them, the contour of the car window C appears similar to jagged features, the contour of the fender D appears similar to flower dots and jagged features, and the area of the car window C and the fender D involves multiple colors, for example, the car window C involves To C3 blue, fender D involves C3 blue and C4 pink; there are multiple colors on a certain part E, such as C6 pink, C7 black and C8 purple cross fusion. It can be seen that based on Figure 3 The features of the C component, the D component, and the E component can confirm that the segmented image shown in FIG. 3 has the feature of poor segmentation effect.

In the foregoing embodiment, it can be understood that the segmented pictures generated based on the component segmentation model can be defined and classified according to different features corresponding to the segmented pictures, and corresponding training labels can be created accordingly. It should be noted that the schematic diagrams shown in FIG. 2 and FIG. 3 are only used for illustration, and do not represent actual divided pictures.

S30: Annotate the segmented picture according to the label with good segmentation effect and the label with poor segmentation effect to obtain the annotated picture.

In an application scenario, based on the component segmentation model to generate different features of the segmented image, you can create labels with good segmentation effects and labels with poor segmentation effects, and segmentation of the pre-trained component segmentation model based on the tags with good segmentation effects and poor segmentation effects. The pictures are labeled. Specifically, manual labeling can be used to label the segmented pictures with features that conform to the preset shape and/or color uniformity with a good segmentation effect label, and for those that have the features of flower dots and/or color intersections. The segmented pictures are labeled with poor segmentation effect labels, so that an annotated image based on the labels with good segmentation effect and training labels with poor segmentation effect can be obtained.

S40: Construct a deep learning model, and obtain a target segmentation effect evaluation model based on the deep learning model and the marked pictures.

In one embodiment, based on artificial intelligence and deep learning technology, a deep learning model can be constructed, and a target segmentation effect evaluation model can be obtained according to the deep learning model and the marked pictures. In this embodiment, the deep learning model can be the MobilenetV1 model. It can be understood that the MobilenetV1 model, as a lightweight deep learning model developed by Google in the early days, has the characteristics of light weight, small calculation amount and high speed, and can be applied to portable terminals equipment. In addition, as a preferred embodiment, the deep learning model can also be the MobilenetV2 model. The MobilenetV2 model is a new generation of lightweight deep learning model released by Google. Compared with the previous version of the MobilenetV1 model, the MobilenetV2 model can also be used on the one hand. Further improve the accuracy and computing speed of deep learning model training, on the other hand, it can also reduce the amount of model training parameters, that is, it can reduce the amount of calculation and memory usage, and the deep learning model can be applied to mobile terminal devices. It should be noted that the following embodiments mainly take the MobilenetV2 model as an example, but it is not limited to only the MobilenetV2 model.

In one embodiment, in order to further ensure the privacy and security of the target effect evaluation model, in this embodiment, the target effect evaluation model may also be stored in the blockchain network.

In one embodiment, the mobilenetV2 model is based on the deep learning model. In the above step S40, the target segmentation effect evaluation model is obtained according to the deep learning model and the labeled pictures. Specifically, as shown in FIG. 4, the following steps may be included S401-S404:

S401: Divide the marked pictures into a training set and a test set.

Based on the labeled pictures with training labels after being labeled, the labeled pictures can be divided into training set and test set. Specifically, the labeled pictures can be divided into training set and test set according to a certain preset ratio, exemplarily , The preset ratio can be, for example, set to 3:1, and the labeled pictures can be divided into training set and test set according to the ratio of 3:1. It should be noted that the preset ratio can also be, for example, 3: 2, etc., which are not limited here, and can be set according to the actual situation.

S402: Perform data amplification on the training set and the test set to obtain an amplified training set and an amplified test set.

In one embodiment, based on the classified training set and test set, data amplification can also be performed on the training set and test set in the marked pictures, so that a new training set and test set can be formed. In an application scenario, for example, during the model training phase, the training set and the test set can be augmented with data. The data augmentation can include, but is not limited to, random cropping, random horizontal flipping, and random rotation, etc. The labeled images in the training set and test set can be randomly cropped, randomly horizontally flipped, or randomly rotated, etc., or you can randomly set different contrast, brightness, and saturation to the labeled images, and you can get More new training sets and test sets, that is, an augmented training set and augmented test set can be obtained. In this embodiment, by performing data amplification on the training set and the test set, the data volume of the training set and the test set can be increased to more than the original multiple orders of magnitude, thereby increasing the diversity of the training set, and not only can avoid overfitting , It also helps to improve the recognition performance of subsequent models. In addition, it should be noted that data amplification can also be achieved in other ways, such as acquiring more data sets, which is not limited here.

S403: Input the augmented training set into the MobilenetV2 model for training, and obtain the training model.

Based on the acquired augmented training set, the augmented training set can be input to the MobilenetV2 model for training to obtain the training model. Among them, the model can be saved every epoch, that is, the new training set is trained and saved once. To obtain the final training model. Specifically, the augmented training set can also be preprocessed with pictures and then input into the MobilenetV2 model for training. The preprocessing of the pictures can include, but is not limited to, picture cropping or picture scaling. Illustratively, for example, the augmented training set is unified After scaling to a size of 365*365 and/or uniformly cropping to a size of 320*320, input to the MobilenetV2 model for training. This can reduce the amount of sample data under the premise of the same sample size, so as to reduce the amount of calculation of the MobilenetV2 model , And adjusting to a fixed and unified format is also conducive to the processing of other steps later. In this embodiment, the deep learning model preferentially selects the MobilenetV2 model. The MobileNetV2 model is a lightweight deep learning model. Compared with the MobilenetV1 model, the MobileNetV2 model can greatly reduce the amount of calculation while also improving the efficiency of model training. And the accuracy of model training.

S404: Input the amplified test set into the training model for testing, and obtain the target segmentation effect evaluation model according to the accuracy of the test.

Based on the training model obtained through the training of the augmented training set, the augmented test set can be input to the training model for testing, and the target segmentation effect evaluation model can be obtained according to the accuracy of the test.

In an embodiment, in step S404, that is, obtaining the target segmentation effect evaluation model according to the accuracy of the test, specifically, the following steps S4041-S4042 may be included:

S4041: Adjust the model parameters according to the accuracy of the test, and obtain the test model with the top K scores of accuracy.

The amplification test set involves related model parameters during the testing process of the training model. The model parameters can include, but are not limited to, for example, learning rate, optimization method, regular term, batch size, etc., and can be continuously based on the accuracy of the test. Adjust the model parameters. Illustratively, you can continuously adjust the learning rate, optimizer method, regularization term, batch size and other model parameters according to the accuracy of the test set, and train and test enough epochs until the model converges, which can be implemented in different Under the model parameters, select the test model with the top K scores of test accuracy, and use the test model with the top K scores of accuracy as the target segmentation effect model. In this embodiment, it can be understood that the target segmentation effect model generated through the training of the annotated image has good segmentation effect and poor segmentation effect training label based on the training label with good segmentation effect and poor segmentation effect. Attributes.

S4042: Use the test model with the top K rankings in the accuracy score as the target segmentation effect evaluation model.

Based on the selection of test models with the top K positions in the test accuracy score, the test models with the top K positions in the accuracy score can be used as the target segmentation effect evaluation model. Specifically, the top K test models can be selected first, for example, for example, the top 2 test models or the top 3 test models are selected. This is not limited here, and can be performed according to the actual situation. select. Based on obtaining the top K test models, the top K test models can be used as the target segmentation effect evaluation model.

S50: Obtain a picture to be tested that has undergone component segmentation, and obtain an evaluation result of the picture to be tested according to the picture to be tested and the target segmentation effect evaluation model.

Based on the segmentation pictures generated by different component segmentation models in the actual scene, when it is necessary to verify the segmentation effect of the segmentation model or filter the segmented images that have failed segmentation, you can obtain the test images that have been segmented and based on the components that have been segmented. The segmented picture to be tested and the target segmentation effect evaluation model obtain the evaluation result based on the picture to be tested. In this embodiment, it can be understood that when the evaluation result based on the picture to be tested is obtained, that is, the segmentation effect of the component segmentation model can be correspondingly evaluated according to the evaluation result of the picture to be tested, so that continuous training can be carried out according to the evaluation result to improve and Improve the accuracy of the part segmentation model. Further, the segmentation quality is good and the segmentation quality is poor based on the above process definition, and partially invalid segmented pictures can also be filtered in advance according to the evaluation result, thereby improving the accuracy of obtaining the segmented pictures.

In an embodiment, the evaluation result of the picture to be tested is obtained according to the picture to be tested and the target segmentation effect evaluation model. Specifically, as shown in FIG. 5, steps S501-S503 may be included:

S501: Scale the picture to be tested to a preset size to obtain a scaled picture.

Based on the segmented pictures that need to be verified, picture preprocessing can be performed on each segmented picture. The picture preprocessing can include, but is not limited to, for example, picture scaling and/or picture cropping, which is not limited here.

In an application scenario, the images to be tested based on component segmentation may involve a variety of different sizes, and the target effect evaluation model has certain restrictions on the size of the input images, so these images of different sizes need to be adjusted to a fixed Large and small pictures, specifically, each picture to be tested can be scaled to a preset size to obtain a scaled picture based on the preset size, where the preset size can be set to, for example, a size of 365*365 or 480*480 The size is not limited here. The preset size in this embodiment is 365*365 as an example, that is, each picture to be tested can be scaled to 365*365, and the acquisition is based on the preset size of 365*365. Zoomed picture. In this embodiment, through preprocessing of image scaling on the image to be tested, the data volume of the sample can be reduced on the premise that the sample volume remains unchanged, and the adjustment to a fixed and uniform format is also beneficial to processing in other steps.

S502: Perform edge cropping on the zoomed picture to obtain an image block used to input the target segmentation effect evaluation model to obtain the evaluation result.

In an embodiment, picture preprocessing may be performed again on the zoomed picture. Illustratively, the zoomed picture may also be edge cropped to obtain an image block used to input the target segmentation effect evaluation model to obtain the evaluation result. Based on the size of the zoomed picture obtained in step S501 of 365*365, the zoomed picture can be edge cropped. It can be understood that the edges can represent unimportant features in the zoomed picture. Specifically, the zoomed picture is cropped based on the center point of the zoomed picture. For the edge, the image block extending from the center point of the zoomed picture to the four sides to a size of 320*320 can be reserved, that is, the zoomed picture can be cropped into a 320*320 image block. In this embodiment, it can be understood that based on the zooming of the picture in step S501, and then cropping the zoomed picture, the data volume of the sample can be further reduced. It should be noted that the process of image scaling and image cropping in the above steps is only for illustration, and the sequence is not limited.

S503: Obtain an evaluation result according to the image block and the target segmentation effect evaluation model.

Based on the acquired target segmentation effect model, the evaluation result may be acquired according to the image block and the target segmentation effect evaluation model. Specifically, by creating a corresponding forward propagation program, and inputting the image pre-processed image block into the target segmentation effect model, the evaluation result based on the specific image to be tested can be obtained.

In this embodiment, it can be understood that the target segmentation effect model generated based on the training tags, that is, the target segmentation effect model has the attributes of good segmentation effect and poor segmentation effect, when segmentation pictures generated by different component segmentation models are input to the target segmentation In the effect model, one evaluation result of two training labels with good output segmentation effect or poor segmentation effect can be obtained. In this way, the segmentation effect of different component segmentation models can be obtained according to the target segmentation effect model. At the same time, it is possible to filter the images whose segmentation model fails to segment, which can further improve the accuracy of other subsequent models such as component recognition models.

In an embodiment, in step S503, the evaluation result is obtained according to the image block and the target segmentation effect evaluation model. Specifically, it may further include steps S5031-S5032:

S5031: Input the image block into the target segmentation effect evaluation model, and obtain test results corresponding to the target segmentation effect evaluation model.

Based on the above-mentioned step S4042, the test model with the top K scores in the accuracy rate is used as the target effect segmentation evaluation model. In one embodiment, the two test models with the top 2 accuracy scores can be selected as the target effect segmentation evaluation models, and the target segmentation effect evaluation can be obtained according to the two test models and image blocks with the top 2 accuracy scores. The test result corresponding to the model, specifically, when the picture to be tested is preprocessed into a picture block with a fixed size of 320*320 or 365*365, the picture block can be input into two test models for prediction. Then, the corresponding test results based on the two test models can be obtained respectively. It should be noted that the selection of the two test models with the top 2 accuracy scores as the target effect segmentation evaluation model is only used as an example. Of course, the three test models with the top 3 accuracy scores can also be selected as the target effect. The segmentation evaluation model, or the selection of the four test models with the top 4 accuracy scores as the target effect segmentation evaluation model, is not limited here.

S5032: Perform a probability average calculation on the test result to obtain an evaluation result corresponding to the target segmentation effect evaluation model.

Based on the test results corresponding to the two test models obtained in step S5031, in one embodiment, the corresponding test results can be calculated for probability average, and then the evaluation results corresponding to the target segmentation effect evaluation model can be obtained. Specifically, for example, the test result generated by one test model is [0.6, 0.5, 0.05, 0.05], and the evaluation result generated by the other test model is [0.4, 0.3, 0.05, 0.05], then the two are merged by the average probability The test result can be evaluated as [0.5, 0.4, 0.05, 0.05]. In this embodiment, it can be understood that if different test models are generated according to different model parameter training, different test results will be generated through different test models. In this embodiment, the method of calculating the probability average of the test results can reduce the evaluation. The results are affected by a single model to improve the accuracy of obtaining the evaluation results. It should be noted that in the above-mentioned embodiment, the selection of the two test models with the top 2 accuracy scores as the target effect segmentation evaluation model is only used as an example and is not limited. Of course, the three top 3 scores can also be selected. One test model or the top four test models, that is, the corresponding probability average calculation of the test results can be performed to obtain the corresponding evaluation results based on the target segmentation effect evaluation model.

It should be understood that the size of the sequence number of each step in the foregoing embodiment does not mean the order of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiment of the present application.

In one embodiment, a segmentation effect evaluation device based on deep learning is provided, and the functions implemented by the segmentation effect evaluation device correspond one-to-one with the steps of the segmentation effect evaluation method in the foregoing embodiment. Specifically, as shown in FIG. 6, the segmentation effect evaluation may include a first acquisition module 10, a label creation module 20, an annotation module 30, a second acquisition module 40, and an evaluation result module 50. Among them, the detailed description of each functional module is as follows:

The first acquisition module 10 is configured to acquire a vehicle picture, and acquire a segmented picture according to the evaluation result of the vehicle picture and the pre-trained component segmentation model;

The label creation module 20 is configured to create training labels according to the characteristics of the segmented image of the evaluation result, and the training labels of the evaluation result include labels with good segmentation effect and labels with poor segmentation effect;

The tagging module 30 is used to tag the segmented pictures of the evaluation result according to the tags with good segmentation effect of the evaluation results and the tags with poor segmentation effect of the evaluation result to obtain the tagged pictures;

The second acquisition module 40 is used to construct a deep learning model, and acquire a target segmentation effect evaluation model according to the evaluation result of the deep learning model and the marked image of the evaluation result;

The evaluation result module 50 is used to obtain the picture to be tested for which the component segmentation has been performed, and obtain the evaluation result of the picture to be tested for the evaluation result according to the evaluation result to be tested picture and the evaluation result target segmentation effect evaluation model.

Preferably, the evaluation result deep learning model is the MobilenetV2 model, and the evaluation result second obtaining module 40 is also used for:

Divide the marked images of the evaluation results into training set and test set;

Perform data amplification on the evaluation result training set and the evaluation result test set to obtain an amplified training set and an amplified test set;

Input the evaluation result augmentation training set into the evaluation result MobilenetV2 model for training, and obtain the training model;

The evaluation result amplification test set is input to the evaluation result training model for testing, and the target segmentation effect evaluation model is obtained according to the accuracy of the test.

For the specific definition of the segmentation effect evaluation device, please refer to the above definition of the segmentation effect evaluation method, which will not be repeated here. Each module in the above-mentioned segmentation effect evaluation device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules may be embedded in the form of hardware or independent of the processor in the computer equipment, or may be stored in the memory of the computer equipment in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

In one embodiment, as shown in FIG. 7, a computer device is provided. Specifically, the computer device 60 of this embodiment includes a processor 61, a memory 62, and computer-readable instructions 63 that are stored in the memory 62 and can run on the processor 61. When the processor 61 executes the computer-readable instruction 63, the steps in the segmentation effect evaluation method of the foregoing embodiment are implemented. To avoid repetition, details are not described herein again. Alternatively, when the processor 61 executes the computer-readable instruction 63, the following steps are implemented:

In one embodiment, one or more readable storage media storing computer readable instructions are provided. The readable storage media provided in this embodiment include non-volatile readable storage media and volatile readable storage. In the medium, when the computer-readable instructions are executed by one or more processors, the one or more processors execute the following steps:

Understandably, the computer-readable storage medium may include: any entity or device capable of carrying the computer-readable instruction code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory ( ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), electrical carrier signal and telecommunication signal, etc.

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing relevant hardware through computer-readable instructions. The computer-readable instructions can be stored in a non-volatile computer. In a readable storage medium, when the computer-readable instructions are executed, they may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database, or other media used in the embodiments provided in this application may include non-volatile and/or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not a limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Channel (SynchlinK) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

The blockchain referred to in this application is a new application mode of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information for verification. The validity of the information (anti-counterfeiting) and the generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

Those skilled in the art can clearly understand that for the convenience and conciseness of description, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned functions can be allocated to different functional modules and modules as required. Sub-module and unit completion, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, a person of ordinary skill in the art should understand that it can still implement the foregoing The technical solutions recorded in the examples are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in Within the scope of protection of this application. To

To

Claims

A method for evaluating segmentation effect based on deep learning, wherein the method includes:

Obtain a picture of a vehicle, and obtain a segmented picture according to the vehicle picture and the pre-trained component segmentation model;

Create training tags based on the features of the segmented pictures, the training tags include tags with good segmentation effect and tags with poor segmentation effect; wherein, the segmentation effect is good means that the segmented picture has a predetermined shape and/or uniform color. Feature, the poor segmentation effect means that the segmented picture has the feature of flower dots and/or color crossing;

Labeling the segmented picture according to the label with good segmentation effect and the label with poor segmentation effect, to obtain an annotated picture;

Constructing a deep learning model, and obtaining a target segmentation effect evaluation model according to the deep learning model and the labeled picture;

Obtain a picture to be tested that has undergone component segmentation, and obtain an evaluation result of the picture to be tested according to the picture to be tested and the target segmentation effect evaluation model.
5. The segmentation effect evaluation method according to claim 1, wherein the deep learning model is a MobilenetV2 model, and the obtaining a target segmentation effect evaluation model according to the deep learning model and the labeled picture comprises:

Divide the labeled pictures into a training set and a test set;

Performing data amplification on the training set and the test set to obtain an amplified training set and an amplified test set;

Input the augmented training set into the MobilenetV2 model for training, and obtain a training model;

Inputting the amplification test set to the training model for testing;

Obtain the target segmentation effect evaluation model according to the test accuracy.
3. The segmentation effect evaluation method according to claim 2, wherein the obtaining the target segmentation effect evaluation model according to the test accuracy includes:

Adjust the model parameters according to the accuracy of the test, and obtain the test model with the top K scores of accuracy;

The test model with the top K rankings in the accuracy score is used as the target segmentation effect evaluation model.
The segmentation effect evaluation method according to claim 3, wherein said obtaining the evaluation result of the picture to be tested according to the picture to be tested and the target segmentation effect evaluation model comprises:

Zooming the picture to be tested to a preset size to obtain a zoomed picture;

Performing edge cropping on the zoomed picture to obtain an image block used to input the target segmentation effect evaluation model to obtain an evaluation result;

Obtaining the evaluation result according to the image block and the target segmentation effect evaluation model.
5. The segmentation effect evaluation method according to claim 4, wherein said obtaining said evaluation result according to said image block and said target segmentation effect evaluation model comprises:

Input the image block into the target segmentation effect evaluation model, and obtain a test result corresponding to the target segmentation effect evaluation model;

A probability average calculation is performed on the test result to obtain an evaluation result corresponding to the target segmentation effect evaluation model.
The segmentation effect evaluation method according to claim 2, wherein the data amplification includes random cropping, random horizontal flipping, and random rotation.
A device for evaluating segmentation effects based on deep learning, wherein the device includes:

The first acquisition module is configured to acquire a vehicle picture, and acquire a segmented picture according to the vehicle picture and the pre-trained component segmentation model;

The label creation module is used to create training labels based on the features of the segmented picture. The training labels include labels with good segmentation effect and labels with poor segmentation effect. The good segmentation effect means that the segmented picture has a shape conforming to a preset shape. And/or the feature of uniform color, the poor segmentation effect means that the segmented picture has the feature of flower dots and/or color crossing;

An annotation module, configured to annotate the segmented picture according to the label with good segmentation effect and the label with poor segmentation effect, to obtain the annotated picture;

The second acquisition module is configured to construct a deep learning model, and acquire a target segmentation effect evaluation model according to the deep learning model and the labeled picture;

The evaluation result module is used to obtain the picture to be tested for which the component segmentation has been performed, and obtain the evaluation result of the picture to be tested according to the picture to be tested and the target segmentation effect evaluation model.
8. The segmentation effect evaluation device according to claim 7, wherein the deep learning model is a MobilenetV2 model, and the second acquisition module is further used for:

Divide the labeled pictures into a training set and a test set;

Performing data amplification on the training set and the test set to obtain an amplified training set and an amplified test set;

Input the augmented training set into the MobilenetV2 model for training, and obtain a training model;

The amplification test set is input to the training model for testing, and the target segmentation effect evaluation model is obtained according to the accuracy of the test.
A computer device includes a memory, a processor, and computer-readable instructions that are stored in the memory and can run on the processor, wherein the processor implements the following steps when the processor executes the computer-readable instructions:

Acquiring a vehicle picture, and acquiring a segmented picture according to the vehicle picture and the pre-trained component segmentation model;

Create training tags based on the features of the segmented pictures, the training tags include tags with good segmentation effect and tags with poor segmentation effect; wherein, the segmentation effect is good means that the segmented picture has a predetermined shape and/or uniform color. Feature, the poor segmentation effect means that the segmented picture has the feature of flower dots and/or color crossing;

Labeling the segmented picture according to the label with good segmentation effect and the label with poor segmentation effect, to obtain an annotated picture;

Constructing a deep learning model, and obtaining a target segmentation effect evaluation model according to the deep learning model and the labeled picture;

Obtain a picture to be tested that has undergone component segmentation, and obtain an evaluation result of the picture to be tested according to the picture to be tested and the target segmentation effect evaluation model.
9. The computer device according to claim 9, wherein the deep learning model is a MobilenetV2 model, and acquiring a target segmentation effect evaluation model according to the deep learning model and the labeled picture comprises the following steps:

Divide the labeled pictures into a training set and a test set;

Performing data amplification on the training set and the test set to obtain an amplified training set and an amplified test set;

Input the augmented training set into the MobilenetV2 model for training, and obtain a training model;

Inputting the amplification test set to the training model for testing;

Obtain the target segmentation effect evaluation model according to the test accuracy.
10. The computer device according to claim 10, wherein said obtaining the target segmentation effect evaluation model according to the test accuracy includes the following steps:

Adjust the model parameters according to the accuracy of the test, and obtain the test model with the top K scores of accuracy;

The test model with the top K rankings in the accuracy score is used as the target segmentation effect evaluation model.
11. The computer device according to claim 11, wherein said obtaining the evaluation result of the picture to be tested according to the picture to be tested and the target segmentation effect evaluation model comprises the following steps:

Zooming the picture to be tested to a preset size to obtain a zoomed picture;

Performing edge cropping on the zoomed picture to obtain an image block used to input the target segmentation effect evaluation model to obtain an evaluation result;

Obtaining the evaluation result according to the image block and the target segmentation effect evaluation model.
The computer device according to claim 12, wherein said obtaining said evaluation result according to said image block and said target segmentation effect evaluation model comprises the following steps:

Input the image block into the target segmentation effect evaluation model, and obtain a test result corresponding to the target segmentation effect evaluation model;

A probability average calculation is performed on the test result to obtain an evaluation result corresponding to the target segmentation effect evaluation model.
10. The computer device of claim 10, wherein the data augmentation includes random cropping, random horizontal flipping, and random rotation.
One or more readable storage media storing computer readable instructions, where when the computer readable instructions are executed by one or more processors, the one or more processors execute the following steps:

Acquiring a vehicle picture, and acquiring a segmented picture according to the vehicle picture and the pre-trained component segmentation model;

Create training tags based on the features of the segmented pictures, the training tags include tags with good segmentation effect and tags with poor segmentation effect; wherein, the segmentation effect is good means that the segmented picture has a predetermined shape and/or uniform color. Feature, the poor segmentation effect means that the segmented picture has the feature of flower dots and/or color crossing;

Labeling the segmented picture according to the label with good segmentation effect and the label with poor segmentation effect, to obtain an annotated picture;

Constructing a deep learning model, and obtaining a target segmentation effect evaluation model according to the deep learning model and the labeled picture;

Obtain a picture to be tested that has undergone component segmentation, and obtain an evaluation result of the picture to be tested according to the picture to be tested and the target segmentation effect evaluation model.
15. The readable storage medium according to claim 15, wherein the deep learning model is a MobilenetV2 model, and acquiring a target segmentation effect evaluation model according to the deep learning model and the labeled picture comprises the following steps:

Divide the labeled pictures into a training set and a test set;

Performing data amplification on the training set and the test set to obtain an amplified training set and an amplified test set;

Input the augmented training set into the MobilenetV2 model for training, and obtain a training model;

Inputting the amplification test set to the training model for testing;

Obtain the target segmentation effect evaluation model according to the test accuracy.
15. The readable storage medium of claim 16, wherein the obtaining the target segmentation effect evaluation model according to the test accuracy includes the following steps:

Adjust the model parameters according to the accuracy of the test, and obtain the test model with the top K scores of accuracy;

The test model with the top K rankings in the accuracy score is used as the target segmentation effect evaluation model.
17. The readable storage medium according to claim 17, wherein the obtaining the evaluation result of the picture to be tested according to the picture to be tested and the target segmentation effect evaluation model comprises the following steps:

Zooming the picture to be tested to a preset size to obtain a zoomed picture;

Performing edge cropping on the zoomed picture to obtain an image block used to input the target segmentation effect evaluation model to obtain an evaluation result;

Obtaining the evaluation result according to the image block and the target segmentation effect evaluation model.
18. The readable storage medium according to claim 18, wherein said obtaining said evaluation result according to said image block and said target segmentation effect evaluation model comprises the following steps:

Input the image block into the target segmentation effect evaluation model, and obtain a test result corresponding to the target segmentation effect evaluation model;

A probability average calculation is performed on the test result to obtain an evaluation result corresponding to the target segmentation effect evaluation model.
15. The readable storage medium of claim 16, wherein the data augmentation includes random cropping, random horizontal flipping, and random rotation.