CN113947571A

CN113947571A - Training method of vehicle damage detection model and vehicle damage identification method

Info

Publication number: CN113947571A
Application number: CN202111163402.XA
Authority: CN
Inventors: 于越; 谭啸; 孙昊
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2021-09-30
Filing date: 2021-09-30
Publication date: 2022-01-18

Abstract

The invention provides a training method of a vehicle damage detection model and a vehicle damage identification method, relates to the field of artificial intelligence, and particularly relates to the technical field of computer vision and deep learning. The specific implementation scheme is as follows: the method comprises the steps of obtaining label images and label-free images from a plurality of images to be processed containing the vehicle, training a first initial model based on the label images to obtain a first target model, wherein the first initial model is used for processing the images with labels, predicting the label-free images based on the first target model to obtain pseudo labels corresponding to the label-free images, and training a second initial model based on the label images, the label-free images and the pseudo labels corresponding to the label-free images to obtain a second target model, wherein the second target model is used for carrying out damage detection on the vehicle, and the second initial model is used for processing the label-free images. The method and the device at least solve the technical problem that the vehicle damage monitoring model generated in the prior art is poor in robustness.

Description

Training method of vehicle damage detection model and vehicle damage identification method

Technical Field

The disclosure relates to the field of artificial intelligence, in particular to the technical field of computer vision and deep learning, and specifically relates to a training method of a vehicle damage detection model and a vehicle damage identification method.

Background

At present, when a vehicle damage is detected through a vehicle damage detection model, a label image is usually required to be used for training the vehicle damage detection model in advance.

However, manual labeling is required to obtain the label image, and the process often requires a lot of manpower and is very costly. In addition, due to factors such as different positions of damaged vehicle parts such as scratch and the like, different vehicle types, different external environments and the like, a vehicle damage detection model using only a small amount of labeled data has a serious overfitting phenomenon, and has poor performance in new scenes and new vehicle types.

Disclosure of Invention

The disclosure provides a training method of a vehicle damage detection model and a vehicle damage identification method, which at least solve the technical problem that in the prior art, the generated network model is poor in robustness due to the fact that the number of data samples with labels is too small.

According to an aspect of the present disclosure, there is provided a training method of a vehicle damage detection model, including: the method comprises the steps of obtaining a label image and a non-label image from a plurality of images to be processed containing a vehicle, wherein the label image at least comprises vehicle damage data used for marking the damage condition of the vehicle, training a first initial model based on the label image to obtain a first target model, the first initial model is used for processing the image with a label, predicting the non-label image based on the first target model to obtain a pseudo label corresponding to the non-label image, and training a second initial model according to the label image, the non-label image and the pseudo label corresponding to the non-label image to obtain a second target model, the second target model is used for carrying out damage detection on the vehicle, and the second initial model is used for processing the non-label image.

Further, the training method of the vehicle damage detection model further comprises the following steps: the label-free image is predicted based on the first target model, a plurality of labels corresponding to the label-free image are obtained, confidence degrees of the labels are calculated, and therefore the labels are screened according to the confidence degrees of the labels and confidence degree threshold values, and the pseudo labels are obtained.

Further, the training method of the vehicle damage detection model further comprises the following steps: and acquiring the label with the confidence coefficient larger than the confidence coefficient threshold value from the plurality of labels to obtain a pseudo label.

Further, the training method of the vehicle damage detection model further comprises the following steps: and mixing the label-free image with the pseudo label and the label image to obtain a mixed image set, and training the second initial model based on the mixed image set to obtain a second target model.

Further, the training method of the vehicle damage detection model further comprises the following steps: after the second initial model is trained according to the label image, the non-label image and the pseudo label corresponding to the non-label image to obtain a second target model, a first model parameter corresponding to the first target model is obtained, a first loss value corresponding to the non-label image is calculated based on a preset loss function, and therefore a second model parameter corresponding to the second target model is adjusted according to the first loss value, the first model parameter is adjusted based on the second model parameter, and the adjusted first target model is obtained.

Further, the training method of the vehicle damage detection model further comprises the following steps: after the first model parameters are adjusted based on the second model parameters to obtain the adjusted first target model, calculating a first loss value corresponding to the label image and a second loss value corresponding to the label-free image, and performing weighted summation processing on the first loss value and the second loss value to obtain a third loss value, so that when the third loss value meets the preset condition, the first target model and the second target model are stopped from being updated.

Further, the training method of the vehicle damage detection model further comprises the following steps: and after the updating of the first target model and the second target model is stopped, acquiring a target image corresponding to the target vehicle, and processing the image to be processed based on the second target model to obtain damage data corresponding to the target vehicle.

Further, the training method of the vehicle damage detection model further comprises the following steps: the method comprises the steps of obtaining a plurality of images to be processed, determining label images from the plurality of images to be processed, obtaining labeling results of the label images, obtaining vehicle damage data, removing the label images from the images to be processed, and obtaining non-label images, wherein the number of the label images is smaller than that of the non-label images.

According to another aspect of the present disclosure, there is also provided a training apparatus for a vehicle damage detection model, including: the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring a label image and a non-label image from a plurality of images to be processed containing the vehicle, and the label image at least comprises vehicle damage data used for marking the vehicle damage condition; the first training module is used for training a first initial model based on the label image to obtain a first target model, wherein the first initial model is used for processing the image with the label; the prediction module is used for predicting the label-free image based on the first target model to obtain a pseudo label corresponding to the label-free image; and the second training module is used for training the second initial model according to the label image, the non-label image and the pseudo label corresponding to the non-label image to obtain a second target model, wherein the second target model is used for carrying out damage detection on the vehicle, and the second initial model is used for processing the non-label image.

According to another aspect of the present disclosure, there is also provided a vehicle damage identification method, including: acquiring an image to be identified, wherein the image to be identified at least comprises a vehicle to be identified; identifying an image to be identified based on a vehicle damage detection model to obtain an identification result, wherein the vehicle damage detection model is obtained based on a label image, a non-label image and a pseudo label training corresponding to the non-label image, and the pseudo label is a label obtained by predicting the non-label image based on a first target model obtained by the label image training; and determining the damage information of the vehicle to be identified according to the identification result.

Further, the vehicle damage identification method further includes: and determining the damage position and/or the damage degree of the vehicle to be identified according to the identification result.

Further, the vehicle damage identification method further includes: and after determining the damage information of the vehicle to be identified according to the identification result, determining a maintenance strategy for maintaining the vehicle to be identified according to the damage position and/or the damage degree.

According to another aspect of the present disclosure, there is also provided a vehicle damage recognition apparatus including: the device comprises an acquisition module, a recognition module and a recognition module, wherein the acquisition module is used for acquiring an image to be recognized, and the image to be recognized at least comprises a vehicle to be recognized; the identification module is used for identifying the image to be identified based on the vehicle damage detection model to obtain an identification result, wherein the vehicle damage detection model is obtained based on a label image, a non-label image and a pseudo label training corresponding to the non-label image, and the pseudo label is a label obtained by predicting the non-label image based on a first target model obtained by the label image training; and the determining module is used for determining the damage information of the vehicle to be identified according to the identification result.

According to another aspect of the present disclosure, there is also provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the training method for the vehicle damage detection model.

According to another aspect of the present disclosure, there is also provided a non-transitory computer readable storage medium storing computer instructions for causing a computer to execute the training method of the vehicle damage detection model described above.

According to another aspect of the present disclosure, there is also provided a computer program product comprising a computer program which, when executed by a processor, implements the above-mentioned training method of the vehicle damage detection model

In the disclosure, a method of predicting an unlabeled image based on a first target model to obtain a pseudo label corresponding to the unlabeled image is adopted, so that a second initial model is trained according to the labeled image, the unlabeled image and the pseudo label corresponding to the unlabeled image, a labeled image and a unlabeled image are obtained from a plurality of images to be processed including a vehicle, wherein the labeled image at least comprises vehicle damage data for marking the damage condition of the vehicle, the first initial model is trained based on the labeled image to obtain the first target model, the first initial model is used for processing an image with a label, and the unlabeled image is predicted based on the first target model to obtain a pseudo label corresponding to the unlabeled image, so that the second initial model is trained according to the labeled image, the unlabeled image and the pseudo label corresponding to the unlabeled image, and obtaining a second target model, wherein the second target model is used for carrying out damage detection on the vehicle, and the second initial model is used for processing the image without the label.

In the process, the pseudo label corresponding to the unlabeled image is obtained by predicting the unlabeled image based on the first target model, and the second initial model is trained according to the labeled image, the unlabeled image and the pseudo label corresponding to the unlabeled image to obtain the second target model, so that the aim of obtaining a vehicle damage detection model by training a large number of unlabeled images and a small number of labeled images is fulfilled, and the technical problem that the generated vehicle damage detection model is poor in robustness due to the fact that the number of data samples with labels is too small in the prior art is solved. Moreover, since the vehicle damage detection model is generated based on a large amount of image data in this process, the vehicle damage detection model of the present disclosure has a better detection effect than a vehicle damage detection model trained only with the tag image.

Therefore, the scheme provided by the disclosure achieves the purpose of avoiding the need of generating a large amount of label images by manpower, thereby realizing the technical effect of saving labor cost, and further solving the technical problem that the generated vehicle damage monitoring model is poor in robustness due to the fact that the number of data samples with labels is too small in the prior art.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

fig. 1 is a flowchart of a training method of a vehicle damage detection model according to embodiment 1 of the present disclosure;

fig. 2 is a flowchart of a training method of a vehicle damage detection model according to embodiment 1 of the present disclosure;

fig. 3 is a schematic diagram of a training device for a vehicle damage detection model according to embodiment 2 of the present disclosure;

FIG. 4 is a block diagram of an electronic device for implementing a method of training a vehicle impairment detection model according to an embodiment of the present disclosure;

fig. 5 is a flow chart of a vehicle impairment identification method according to embodiment 3 of the present disclosure;

fig. 6 is a schematic diagram of a vehicle damage identification device according to embodiment 4 of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the personal information of the related user all accord with the regulations of related laws and regulations, and do not violate the good customs of the public order.

Example 1

In accordance with an embodiment of the present disclosure, there is provided an embodiment of a training method for a vehicle damage detection model, it is noted that the steps illustrated in the flowchart of the drawings may be executed in a computer system such as a set of computer executable instructions, and that while a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be executed in an order different than that herein.

Fig. 1 is a flowchart of a training method of a vehicle damage detection model according to an embodiment of the disclosure, as shown in fig. 1, the method includes the following steps:

step S102, acquiring a label image and a non-label image from a plurality of images to be processed containing vehicles.

In an optional embodiment, the tag image at least includes vehicle damage data for marking a vehicle damage condition, for example, a vehicle damage condition such as a scratch of a vehicle door, a drop of a vehicle lamp, or a damage of a vehicle window of a vehicle is displayed in the image to be processed, and the tag image is an image obtained by marking a position of the vehicle damage and a degree of the vehicle damage. The electronic device can obtain the label image through a manual labeling mode, and can also obtain the existing label image through network downloading directly.

Optionally, the label-free image is an image without labeling the damage condition of the vehicle. The operator can acquire the label image and the non-label image in various modes such as photographing, network downloading or software synthesis.

And step S104, training the first initial model based on the label image to obtain a first target model.

In an alternative embodiment, the first initial model may be a neural network model that acts as a teacher, such as the fast rcnn (fast convolutional neural network) detection model. The first initial model can process the label images according to a normal detection model training process, so that training is completed, and the first target model is obtained, wherein excessive label images are not needed for training the first initial model. For example, the ratio of the label image to the non-label image in the current image to be processed is 1: 10, the operator can select all the label images in the image to be processed to train the first initial model. It should be noted that the above ratio is only an example, and the number of unlabeled images and the number of labeled images are not particularly limited in the embodiments of the present disclosure.

In the process, the image to be processed is divided into the label image and the non-label image, and the first initial model is trained based on the label image, so that the detection accuracy of the generated first target model can be improved.

And S106, predicting the unlabeled image based on the first target model to obtain a pseudo label corresponding to the unlabeled image.

In an alternative embodiment, a large number of unlabeled images can be predicted through the first target model, so that pseudo labels corresponding to the unlabeled images are obtained. The pseudo label is a virtual label for the vehicle damage condition in the unlabeled image, and is different from a manual label, and the pseudo label is generated by automatic label of the first target model, for example, if the first target model identifies that the vehicle in the unlabeled image has the conditions of vehicle door scratch, vehicle lamp falling and vehicle window damage, the first target model can mark the vehicle door, the vehicle lamp and the vehicle window with the vehicle damage, that is, the pseudo label of the unlabeled image is generated.

In the process, the first target model is generated based on the label image training, so that the accuracy of the prediction result of the first target model is high, a large number of label-free images are predicted based on the first target model, and the pseudo labels corresponding to the label-free images can be automatically generated, so that a large number of manual labeling works are avoided, and the effect of saving labor cost is realized.

And S108, training the second initial model according to the label image, the non-label image and the pseudo label corresponding to the non-label image to obtain a second target model.

In an alternative embodiment, the second target model is used for damage detection of the vehicle and the second initial model is used for processing the unlabeled image. Wherein the second initial model may be a neural network model acting as a student role. Before training the second initial model, the electronic device may perform mixing processing on the unlabeled image with the pseudo label and the labeled image to obtain a mixed image set, and train the second initial model based on the mixed image set to obtain a second target model. The electronic equipment can record and store the corresponding relation between the pseudo label and the label-free image.

It should be noted that, since the second initial model is trained based on the labeled images and a large number of unlabeled images, the training samples are sufficient, and therefore, the trained second target model has good robustness. In the process, the purpose of obtaining a vehicle damage detection model through training by fully utilizing a large number of label-free images and a small number of label images is achieved, and the technical problem that the generated vehicle damage detection model is poor in robustness due to the fact that the number of data samples with labels is too small in the prior art is solved.

Based on the contents of the above steps S102 to S108, in the solution of the embodiment of the present disclosure, a method of predicting an unlabeled image based on a first target model to obtain a pseudo label corresponding to the unlabeled image is adopted, so as to train a second initial model according to the labeled image, the unlabeled image, and the pseudo label corresponding to the unlabeled image, and obtain a labeled image and an unlabeled image from a plurality of images to be processed including a vehicle, wherein the labeled image at least includes vehicle damage data for marking vehicle damage conditions, and train the first initial model based on the labeled image to obtain a first target model, wherein the first initial model is used for processing an image with a label, and predicts the unlabeled image based on the first target model to obtain a pseudo label corresponding to the unlabeled image, so as to obtain the labeled image and the unlabeled image, and a pseudo label corresponding to the label-free image is trained on the second initial model to obtain a second target model, wherein the second target model is used for carrying out damage detection on the vehicle, and the second initial model is used for processing the label-free image.

It is easy to note that, in the above process, the pseudo label corresponding to the unlabeled image is obtained by predicting the unlabeled image based on the first target model, and the second initial model is trained according to the labeled image, the unlabeled image, and the pseudo label corresponding to the unlabeled image to obtain the second target model, so that the purpose of obtaining a vehicle damage detection model by training by fully utilizing a large number of unlabeled images and a small number of labeled images is achieved, and the technical problem that the robustness of the generated vehicle damage detection model is poor due to the fact that the number of data samples with labels is too small in the prior art is solved. Moreover, since the vehicle damage detection model is generated based on a large amount of image data in the process, compared with the vehicle damage detection model obtained by training only with the label image, the vehicle damage detection model finally obtained by the embodiment of the disclosure has a better detection effect.

In an optional embodiment, the electronic device predicts the unlabeled image based on the first target model to obtain a plurality of labels corresponding to the unlabeled image, and calculates confidence degrees of the labels, so as to screen the labels according to the confidence degrees of the labels and a confidence degree threshold value to obtain a pseudo label.

Optionally, for a large amount of label-free data, the electronic device may predict through the trained first target model, and calculate the confidence degrees for the plurality of labels corresponding to the obtained label-free image. The confidence threshold value can be set by an operator in advance according to the first target model, and can be set by a user, and the confidence threshold value is generally set according to prediction experience and scene requirements, for example, the confidence value is set to 0.7 according to the prediction experience, if the usage scene is a scene of vehicle factory inspection with high precision requirement, the confidence threshold value can be properly adjusted to be higher, for example, 0.8, and if the usage scene is a scene of vehicle scrapping inspection with low precision requirement, the confidence threshold value can be properly adjusted to be lower, for example, 0.6.

It should be noted that the labels can be screened by calculating the confidence degrees of the non-label images corresponding to the labels and setting a confidence degree threshold value, so that the confidence degree of the finally obtained pseudo labels is improved, and the detection accuracy of the finally trained vehicle damage detection model is improved.

In an alternative embodiment, the first target model may obtain a label with a confidence level greater than a confidence threshold from the plurality of labels, resulting in a pseudo label.

Optionally, the first target model predicts the unlabeled image, and there may be a label with an inaccurate prediction result in the obtained multiple labels, and since the accuracy of the prediction result of the label can be characterized by the confidence, the label with the confidence greater than the confidence threshold can be used as a pseudo label, and the label with the confidence lower than the confidence threshold does not participate in the subsequent training of the second initial model.

Through the process, the credibility of the pseudo label is guaranteed, and therefore the detection accuracy of the vehicle damage detection model obtained through final training is improved.

In an alternative embodiment, the electronic device may perform blending processing on the unlabeled image with the pseudo label and the labeled image to obtain a blended image set, and train the second initial model based on the blended image set to obtain a second target model.

Optionally, the electronic device may record and store an association relationship between the pseudo tag and the non-tag image while acquiring the pseudo tag, so that the non-tag image and the tag image corresponding to the pseudo tag are mixed based on the pseudo tag, and a mixed image set is obtained.

Further, the electronic device may train the second initial model based on the mixed image set, and optionally, a GPU (graphics processing unit) may be used in the training process to accelerate training and testing of the second neural initial model.

In the process, through hybrid processing, the purpose of fully utilizing a large number of label-free images and a small number of label images and training to obtain a vehicle damage detection model is achieved, the technical problem that in the prior art, due to the fact that the number of data samples with labels is too small, the generated vehicle damage monitoring model is poor in robustness is solved, and the effect of saving manual labeling cost is achieved.

In an optional embodiment, the electronic device trains the second initial model according to the label image, the non-label image, and the pseudo label corresponding to the non-label image to obtain a second target model, and then obtains a first model parameter corresponding to the first target model. And calculating a second loss value corresponding to the label-free image based on a preset loss function, so as to adjust a second model parameter corresponding to the second target model according to the second loss value, and adjust the first model parameter based on the second model parameter to obtain the adjusted first target model.

Optionally, after obtaining the second object model, the electronic device may obtain a first model parameter (which may be represented by Pa) corresponding to the first object model. Meanwhile, the electronic device calculates a second loss value (which may be represented by L2) corresponding to the unlabeled image based on the preset loss function, and adjusts a second model parameter (which may be represented by Pb) corresponding to the second target model according to the second loss value, and finally, the electronic device may adjust the first model parameter by using the second model parameter, where the adjusted first model parameter may be represented by Pa _ new.

Further, the specific adjustment formula can refer to formula one, which is as follows:

Pa_new＝(beta)*Pa+(1-beta)*Pb

wherein Pa _ new is a first model parameter corresponding to the adjusted first target model, Pa is a first model parameter corresponding to the first target model before adjustment, Pb is a second model parameter corresponding to the second target model after self-adjustment according to the second loss value, it should be noted that beta in the formula one is a hyper-parameter, and can be set by an operator in a self-defined manner, for example, beta is set to 0.999 by default.

In the process, the first model parameters corresponding to the first target model can be continuously updated by using an exponential weighted average mode, so that the prediction performance of the first target model can be continuously optimized, and the effect of mutually promoting the first target model and the second target model is realized.

In an optional embodiment, after the electronic device adjusts the first model parameter based on the second model parameter to obtain the adjusted first target model, the electronic device may calculate a first loss value corresponding to the labeled image and a second loss value corresponding to the unlabeled image, and perform weighted summation on the first loss value and the second loss value to obtain a third loss value, so that when the third loss value meets a preset condition, the electronic device stops updating the first target model and the second target model.

Optionally, after obtaining the adjusted first target model, the electronic device may calculate loss values corresponding to the label image and the unlabeled image respectively, for example, the loss value corresponding to the label image is recorded as a first loss value, and the loss value corresponding to the unlabeled image is recorded as a second loss value. After the first loss value and the second loss value are obtained, the electronic device performs weighted summation processing on the first loss value and the second loss value, and the specific calculation process can refer to a formula two:

L＝L1+alpha*L2

wherein L1 in formula two represents the first loss value, L2 represents the second loss value, and alpha is a loss coefficient for controlling the ratio of the two loss values, which can be set by an operator according to experience, for example, the default value of the loss coefficient is set to 0.5. Further, the calculation result after weighted summation can be recorded as a third loss value (i.e., L in the formula).

Optionally, the first loss value and the second loss value may be calculated according to a conventional calculation method for detecting a loss value of a network model, but it should be noted that the data sources for calculating the first loss value and the second loss value are different, the data for calculating the first loss value is derived from a labeled image, and the data for calculating the second loss value is derived from an unlabeled image.

Further, the electronic device may repeat the process of updating the first target model and the second target model until the third loss value meets the preset condition, and then stop updating to complete the training of the first target model and the second target model. The preset condition may be set to recognize that the third loss value is a minimum value, so that when the third loss value is minimized, whether the first target model and the second target model after updating the parameters converge is determined, and a final vehicle damage detection model is output when converging. The operator can also set the maximum repetition times, and the electronic device notifies the first target model and the second target model to be updated when the repetition times reach the maximum repetition times.

It should be noted that, when the first target model is initially predicted, a predicted result may have errors, and the number of unlabeled images is greater than that of labeled images, so that the detection accuracy of the first target model and the second target model can be further optimized by calculating loss values of the first target model and the second target model respectively and performing weighted summation on the two loss values, thereby achieving an effect of improving the detection capability of the two target models.

In an optional embodiment, after stopping updating the first target model and the second target model, the electronic device obtains a target image corresponding to the target vehicle; and processing the image to be processed based on the second target model to obtain damage data corresponding to the target vehicle.

Alternatively, the target vehicle may be a vehicle with a damaged body, for example, a scratch on the body of the vehicle. The target image is an image including a target vehicle.

Optionally, after the electronic device obtains the second target model after the update is stopped, the obtained second target model may be used as a vehicle damage detection model for executing a vehicle damage detection task. For example, when a target vehicle in the target image has a vehicle damage condition of door scratch, the second target model is used for identifying and detecting the target image, the second target model can automatically identify that the door of the target vehicle has the damage of door scratch, and meanwhile, the second target model can explain information such as the position, area and depth of the damage, identify the damage degree and mark the damage level, such as severe damage, moderate damage and slight damage.

In the process, the vehicle damage is detected through the second target model, so that the effects of improving the detection efficiency and reducing the manual detection cost are achieved, and the problem of instability in manual detection is solved.

In an optional embodiment, the electronic device acquires a plurality of images to be processed, determines a label image from the plurality of images to be processed, thereby acquires a labeling result of the label image, obtains vehicle damage data, and further removes the label image from the images to be processed, thereby obtaining a non-label image, wherein the number of the label images is less than the number of the non-label images.

Optionally, the electronic device may obtain a plurality of images to be processed through network downloading, photographing, or software synthesis, and determine the tag image from the images to be processed, where an operator may select a partial image from the images to be processed, obtain the tag image through manual labeling, or specifically obtain a part of the existing tag image when obtaining the images to be processed.

Optionally, an operator may label the damaged portion of the vehicle by performing a frame on an image to be processed including the vehicle to obtain a labeling result of the label image, where images other than the label image in the image to be processed are non-label images, and the number of the label images is less than the number of the non-label images.

In the process, because the number of the label images is smaller than that of the label-free images, operators do not need to label a large number of images to be processed manually, only a small number of label images and a large number of label-free images are needed to complete the training of the vehicle damage detection model, and the problem that the vehicle damage detection model obtained through training is seriously over-fitted is solved.

Fig. 2 is a flowchart of a training method for a vehicle damage detection model according to an embodiment of the present disclosure, so as to further describe a process of training the vehicle damage detection model in the embodiment of the present disclosure, and as shown in fig. 2, the embodiment of the present disclosure includes a first target model and a second target model with two model results being the same, where a teacher model a is the first target model and a student model B is the second target model, and finally, the embodiment of the present disclosure uses the student model B after training as a vehicle damage detection model for actually performing a vehicle damage detection task.

Optionally, as shown in fig. 2, the electronic device first trains a teacher model a through a small number of labeled images with labeled data, then predicts a large number of remaining unlabeled images by using the teacher model a, and screens out a result with a confidence level higher than a confidence level threshold in a prediction result as a pseudo label of the unlabeled image. And then mixing the label images and the label-free images with the pseudo labels to train a student model B together, updating parameters of the student model B in the training process by an exponential weighted average method to enable the two models to mutually promote, and finally obtaining a vehicle damage detection model with better detection effect compared with the vehicle damage detection model only trained by using the label images.

It should be noted that through the above process, the electronic device can make full use of a large number of label-free images to perform model training on the premise of using only a small number of label images, and a large number of labeled manpower and material resources can be saved. And because training data of different scenes and different vehicle types can be added in the training stage, the robustness of the vehicle damage detection model is greatly enhanced, and the problem of overfitting of a common detection model under the condition of insufficient labeled data is solved.

Therefore, the scheme provided by the disclosure achieves the purpose of avoiding the need of generating a large amount of label images by manpower, thereby realizing the technical effect of saving labor cost, and further solving the technical problem that the generated vehicle damage detection model is poor in robustness due to the fact that the number of data samples with labels is too small in the prior art.

Example 2

According to an embodiment of the present disclosure, an embodiment of a training apparatus for a vehicle damage detection model is also provided, where fig. 3 is a schematic diagram of a training apparatus for a vehicle damage detection model according to embodiment 2 of the present disclosure.

As shown in fig. 3, the training apparatus includes: an obtaining module 301, configured to obtain a tag image and a non-tag image from a plurality of images to be processed including a vehicle, where the tag image at least includes vehicle damage data for marking a vehicle damage condition; a first training module 303, configured to train a first initial model based on the labeled image to obtain a first target model, where the first initial model is used to process an image with a label; the prediction module 305 is configured to predict the unlabeled image based on the first target model to obtain a pseudo label corresponding to the unlabeled image; the second training module 307 is configured to train the second initial model according to the label image, the unlabeled image, and the pseudo label corresponding to the unlabeled image, to obtain a second target model, where the second target model is used to perform damage detection on the vehicle, and the second initial model is used to process the unlabeled image.

It should be noted that the acquiring module 301, the first training module 303, the predicting module 305, and the second training module 307 correspond to steps S102 to S108 in the foregoing embodiment, and the four modules are the same as the corresponding steps in the implementation example and the application scenario, but are not limited to the disclosure in embodiment 1.

Optionally, the prediction module further includes: the device comprises a first prediction module, a calculation model and a screening module. The first prediction module is used for predicting the label-free image based on the first target model to obtain a plurality of labels corresponding to the label-free image; a calculation module for calculating confidence levels of the plurality of tags; and the screening module is used for screening the plurality of labels according to the confidence degrees of the plurality of labels and the confidence degree threshold value to obtain the pseudo labels.

Optionally, the screening module further includes: the first obtaining module is used for obtaining the label with the confidence coefficient larger than the confidence coefficient threshold value from the plurality of labels to obtain the pseudo label.

Optionally, the second training module further includes: the system comprises a mixing module and a third training module, wherein the mixing module is used for mixing the label-free image with the pseudo label and the label image to obtain a mixed image set; and the third training module is used for training the second initial model based on the mixed image set to obtain a second target model.

Optionally, the second training module further includes: the device comprises a second acquisition module, a first calculation module, a first adjustment module and a second adjustment module. The second obtaining module is used for obtaining a first model parameter corresponding to the first target model; the first calculation module is used for calculating a first loss value corresponding to the label-free image based on a preset loss function; the first adjusting module is used for adjusting a second model parameter corresponding to the second target model according to the first loss value; and the second adjusting module is used for adjusting the first model parameter based on the second model parameter to obtain the adjusted first target model.

Optionally, the training device for the vehicle damage detection model further includes: a second calculation module, a summation module, and a stopping module. The second calculation module is used for calculating a first loss value corresponding to the label image and a second loss value corresponding to the non-label image; the summing module is used for carrying out weighted summation processing on the first loss value and the second loss value to obtain a third loss value; and the stopping module is used for stopping updating the first target model and the second target model when the third loss value meets the preset condition.

Optionally, the training device for the vehicle damage detection model further includes: a third acquisition module and a processing module. The third acquisition module is used for acquiring a target image corresponding to the target vehicle; and the processing module is used for processing the image to be processed based on the second target model to obtain the damage data corresponding to the target vehicle.

Optionally, the obtaining module further includes: the device comprises a fourth acquisition module, a determination module, a fifth acquisition module and a removal module. The fourth acquisition module is used for acquiring a plurality of images to be processed; the determining module is used for determining a label image from a plurality of images to be processed; the fifth acquisition module is used for acquiring the labeling result of the label image to obtain vehicle damage data; and the removing module is used for removing the label images from the image to be processed to obtain the label-free images, wherein the number of the label images is less than that of the label-free images.

Example 3

Fig. 5 is a flowchart of a vehicle damage identification method according to an embodiment of the disclosure, as shown in fig. 5, the method includes the following steps:

step S502, acquiring an image to be identified.

In an optional embodiment, the image to be recognized at least includes a vehicle to be recognized, where the vehicle to be recognized may be a vehicle in which a vehicle has damage, for example, a vehicle damage condition such as a scratch of a door of the vehicle, a drop of a lamp of the vehicle, or a damage of a window of the vehicle is displayed in the image to be processed. In addition, the mode of acquiring the image to be recognized can be various modes such as photographing, network downloading or software synthesis.

And step S504, identifying the image to be identified based on the vehicle damage detection model to obtain an identification result.

In an optional embodiment, the vehicle damage detection model is obtained by training pseudo labels corresponding to the label images, the non-label images and the non-label images, and the pseudo labels are labels obtained by predicting the non-label images based on the first target model obtained by training the label images.

Alternatively, embodiments of the present disclosure may provide a first initial model that acts as a teacher, e.g., a fast rcnn (fast convolutional neural network) detection model. The first initial model can process the label images according to a normal detection model training process, so that training is completed, and the first target model is obtained, wherein excessive label images are not needed for training the first initial model. For example, the ratio of the label image to the non-label image in the current image to be processed is 1: 10, the operator can select all the label images in the image to be processed to train the first initial model. It should be noted that the above ratio is only an example, and the number of unlabeled images and the number of labeled images are not particularly limited in the embodiments of the present disclosure.

Optionally, the pseudo tag is a virtual tag for a vehicle damage condition in the unlabeled image, and is different from a manual tagging, and the pseudo tag is generated by automatically tagging the first target model, for example, if the first target model identifies that a vehicle in the unlabeled image has a situation of vehicle door scratch, vehicle lamp falling, and vehicle window damage, the first target model may mark the vehicle door, the vehicle lamp, and the vehicle window with the vehicle damage condition, that is, a pseudo tag of the unlabeled image is generated.

And step S506, determining damage information of the vehicle to be identified according to the identification result.

In an optional embodiment, the damage information of the vehicle to be recognized can be determined according to the recognition result of the vehicle damage detection model for recognizing the object to be recognized, for example, the vehicle damage condition that the vehicle to be recognized in the image to be recognized has a scratch on the vehicle door, the image to be recognized is recognized and detected through the vehicle damage detection model, the vehicle damage detection model can automatically recognize that the vehicle door of the vehicle to be recognized has damage caused by scratch on the vehicle door, meanwhile, the vehicle damage detection model can also explain the information of the position, area, depth and the like of the damage, and identify the damage degree, and identify the damage level, for example, severe damage, moderate damage and slight damage.

In the process, the damage information of the vehicle to be identified is determined according to the identification result, so that the effects of improving the detection efficiency and reducing the manual detection cost are achieved, and the problem of instability in manual detection is solved.

In an optional embodiment, the vehicle damage identification model can also determine the damage position and/or the damage degree of the vehicle to be identified according to the identification result.

Optionally, the identification result of the vehicle damage detection model for identifying the object to be identified may determine damage information of the vehicle to be identified, for example, a vehicle damage condition that a vehicle door of the vehicle to be identified in the image to be identified is scratched exists, the image to be identified is identified and detected by the vehicle damage detection model, the vehicle damage detection model may automatically identify that the vehicle door of the vehicle to be identified has damage that the vehicle door of the vehicle to be identified is scratched, and meanwhile, the vehicle damage detection model may further explain information such as a position, an area, and a depth of the damage, and identify a damage degree, and identify a damage level, for example, a severe damage, a moderate damage, and a slight damage.

In an alternative embodiment, the vehicle damage identification model may determine a repair strategy for repairing the vehicle to be identified according to the damage location and/or the damage degree.

Optionally, after the damage position and/or the damage degree of the vehicle to be identified are/is determined, the vehicle damage identification model may propose a maintenance strategy so that a maintenance worker can refer to the vehicle damage identification model, for example, a vehicle lamp in front of the left side of the vehicle to be identified falls off and is seriously damaged, the vehicle damage identification model may propose a maintenance strategy for replacing the vehicle lamp, and the vehicle damage identification model may also recommend a brand and a model of the vehicle lamp that can be used for the vehicle by obtaining a historical maintenance record and analyzing big data, and meanwhile, the vehicle damage identification model may also give reference opinions such as a corresponding maintenance duration and a maintenance step. Through the process, the maintenance efficiency of maintenance personnel can be effectively improved.

Therefore, the scheme provided by the embodiment of the disclosure achieves the purpose of avoiding the need of generating a large amount of label images by manpower, thereby solving the technical problem that the generated vehicle damage detection model has poor robustness due to the fact that the number of data samples with labels is too small in the prior art, and further achieving the effects of improving the vehicle damage detection efficiency and the maintenance efficiency.

It should be noted that the training method of the vehicle damage detection model in this embodiment is the same as the training method of the second target model in embodiment 1, and related contents have been described in embodiment 1 and are not described again here.

Example 4

According to the embodiment of the present disclosure, an embodiment of a vehicle damage identification device is also provided, wherein fig. 6 is a schematic diagram of a vehicle damage identification device according to embodiment 4 of the present disclosure.

As shown in fig. 6, the training apparatus includes: an acquisition module 601, a recognition module 603, and a determination module 605. The system comprises an acquisition module 601, a recognition module and a recognition module, wherein the acquisition module 601 is used for acquiring an image to be recognized, and the image to be recognized at least comprises a vehicle to be recognized; the identification module 603 is configured to identify an image to be identified based on a vehicle damage detection model to obtain an identification result, where the vehicle damage detection model is obtained by training a label image, a non-label image, and a pseudo label corresponding to the non-label image, and the pseudo label is a label obtained by predicting the non-label image based on a first target model obtained by training the label image; and the determining module 605 is configured to determine the damage information of the vehicle to be identified according to the identification result.

Optionally, the determining module further includes: and the first determining module is used for determining the damage position and/or the damage degree of the vehicle to be identified according to the identification result.

Optionally, the vehicle damage identification device further includes: and the second determining module is used for determining a maintenance strategy for maintaining the vehicle to be identified according to the damage position and/or the damage degree.

Example 5

According to another aspect of the embodiments of the present disclosure, there is also provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the training method of the vehicle damage detection model and the vehicle damage identification method in the above-described embodiments 1 and 3.

Example 6

According to another aspect of the embodiments of the present disclosure, there is also provided a computer program product, which includes a computer program that, when executed by a processor, implements the training method of the vehicle damage detection model and the vehicle damage identification method in the above-mentioned embodiments 1 and 3.

Example 7

According to another aspect of the embodiments of the present disclosure, there is also provided an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the training method of the vehicle damage detection model and the vehicle damage identification method in embodiment 1.

Fig. 4 shows a schematic block diagram of an example electronic device 800 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 4, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The calculation unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.

A number of components in the device 800 are connected to the I/O interface 805, including: an input unit 806, such as a keyboard, a mouse, or the like; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, or the like; and a communication unit 809 such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

Computing unit 801 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and the like. The calculation unit 801 executes the respective methods and processes described above, the training method of the vehicle damage detection model, and the vehicle damage identification method. For example, in some embodiments, the training method of the vehicle damage detection model and the vehicle damage identification method may be implemented as computer software programs tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and/or installed onto device 800 via ROM 802 and/or communications unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the vehicle damage detection model training method and the vehicle damage identification method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the training method of the vehicle damage detection model and the vehicle damage identification method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device. Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.

Claims

1. A method of training a vehicle impairment detection model, comprising:

acquiring a label image and a non-label image from a plurality of images to be processed containing vehicles, wherein the label image at least comprises vehicle damage data used for marking the vehicle damage condition;

training a first initial model based on the label image to obtain a first target model, wherein the first initial model is used for processing the image with the label;

predicting the label-free image based on the first target model to obtain a pseudo label corresponding to the label-free image;

and training a second initial model according to the label image, the non-label image and a pseudo label corresponding to the non-label image to obtain a second target model, wherein the second target model is used for carrying out damage detection on the vehicle, and the second initial model is used for processing the non-label image.

2. The method of claim 1, wherein predicting the unlabeled image based on the first target model to obtain a pseudo label corresponding to the unlabeled image comprises:

predicting the label-free image based on the first target model to obtain a plurality of labels corresponding to the label-free image;

calculating confidence levels of the plurality of tags;

and screening the plurality of labels according to the confidence degrees of the plurality of labels and the confidence degree threshold value to obtain the pseudo labels.

3. The method of claim 2, wherein the filtering the plurality of labels according to the confidence levels of the plurality of labels and a label threshold to obtain the pseudo label comprises:

and acquiring the label with the confidence coefficient larger than the confidence coefficient threshold value from the plurality of labels to obtain the pseudo label.

4. The method of claim 1, wherein training a second initial model according to the labeled image, the unlabeled image, and a pseudo label corresponding to the unlabeled image to obtain a second target model comprises:

mixing the label-free image with the pseudo label and the label image to obtain a mixed image set;

and training the second initial model based on the mixed image set to obtain the second target model.

5. The method of claim 1, after training a second initial model according to the labeled image, the unlabeled image, and a pseudo label corresponding to the unlabeled image to obtain a second target model, the method further comprising:

acquiring a first model parameter corresponding to the first target model;

calculating a first loss value corresponding to the label-free image based on a preset loss function;

adjusting a second model parameter corresponding to the second target model according to the first loss value;

and adjusting the first model parameter based on the second model parameter to obtain an adjusted first target model.

6. The method of claim 5, after adjusting the first model parameters based on the second model parameters, resulting in an adjusted first target model, the method further comprising:

calculating a first loss value corresponding to the label image and a second loss value corresponding to the label-free image;

carrying out weighted summation processing on the first loss value and the second loss value to obtain a third loss value;

and when the third loss value meets a preset condition, stopping updating the first target model and the second target model.

7. The method of claim 6, after stopping updating the first and second target models, the method further comprising:

acquiring a target image corresponding to a target vehicle;

and processing the image to be processed based on the second target model to obtain damage data corresponding to the target vehicle.

8. The method of claim 1, obtaining a tagged image and a non-tagged image from a plurality of images to be processed containing a vehicle, comprising:

acquiring a plurality of images to be processed;

determining the label image from a plurality of images to be processed;

acquiring a labeling result of the label image to obtain the vehicle damage data;

and removing the label images from the image to be processed to obtain the label-free images, wherein the number of the label images is less than that of the label-free images.

9. A training apparatus for a vehicle damage detection model, comprising:

the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring a label image and a non-label image from a plurality of images to be processed containing the vehicle, and the label image at least comprises vehicle damage data used for marking the vehicle damage condition;

the first training module is used for training a first initial model based on the label image to obtain a first target model, wherein the first initial model is used for processing the image with the label;

the prediction module is used for predicting the label-free image based on the first target model to obtain a pseudo label corresponding to the label-free image;

and the second training module is used for training a second initial model according to the label image, the unlabeled image and the pseudo label corresponding to the unlabeled image to obtain a second target model, wherein the second target model is used for carrying out damage detection on the vehicle, and the second initial model is used for processing the unlabeled image.

10. A vehicle impairment identification method comprising:

acquiring an image to be identified, wherein the image to be identified at least comprises a vehicle to be identified;

identifying the image to be identified based on a vehicle damage detection model to obtain an identification result, wherein the vehicle damage detection model is obtained based on a label image, a non-label image and a pseudo label training corresponding to the non-label image, and the pseudo label is a label obtained by predicting the non-label image based on a first target model obtained by the label image training;

and determining the damage information of the vehicle to be identified according to the identification result.

11. The method of claim 10, wherein determining impairment information for the vehicle to be identified from the identification comprises:

and determining the damage position and/or the damage degree of the vehicle to be identified according to the identification result.

12. The method according to claim 11, after determining damage information of the vehicle to be identified from the identification result, the method further comprising:

and determining a maintenance strategy for maintaining the vehicle to be identified according to the damage position and/or the damage degree.

13. A vehicle damage identification device comprising:

the device comprises an acquisition module, a recognition module and a recognition module, wherein the acquisition module is used for acquiring an image to be recognized, and the image to be recognized at least comprises a vehicle to be recognized;

the identification module is used for identifying the image to be identified based on a vehicle damage detection model to obtain an identification result, wherein the vehicle damage detection model is obtained by training a label image, a non-label image and a pseudo label corresponding to the non-label image, and the pseudo label is a label obtained by predicting the non-label image based on a first target model obtained by training the label image;

and the determining module is used for determining the damage information of the vehicle to be identified according to the identification result.

14. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of training a vehicle damage detection model of any of claims 1 to 8 and the method of vehicle damage identification of any of claims 10 to 12.

15. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the training method of the vehicle damage detection model according to any one of claims 1 to 8 and the vehicle damage identification method according to any one of claims 10 to 12.

16. A computer program product comprising a computer program which, when executed by a processor, implements the method of training a vehicle impairment detection model according to any one of claims 1 to 8 and the method of vehicle impairment recognition according to any one of claims 10 to 12.