WO2021135499A1 - Damage detection model training and vehicle damage detection methods, device, apparatus, and medium - Google Patents

Abstract

Damage detection model training and vehicle damage detection methods, a device, an apparatus, and a medium. The training method comprises: acquiring a damage sample set (S10); inputting damage sample images into a damage detection model containing a first parameter, extracting, by means of the damage detection model, damage features from the damage sample images, and generating an intermediate convolution feature map (S20); inputting the intermediate convolution feature map into a mask prediction branch model containing a second parameter (S30); outputting, by means of the damage detection model, a training result according to the damage features, and acquiring a mask result by means of the mask prediction branch model (S40); calculating a first loss value and a second loss value (S50); determining a total loss value according to the first loss value and the second loss value (S60); and if the total loss value does not meet a preset convergence condition, iteratively updating the first parameter of the damage detection model and the second parameter of the mask prediction branch model until the total loss value meets the preset convergence condition, and recording the converged damage detection model as a trained damage detection model (S70). The invention enables fast identification of damage types and damage regions. The invention further relates to the blockchain technique, and enables damage sample images to be stored in a blockchain.

Description

Damage detection model training, vehicle damage detection methods, devices, equipment and media

This application claims the priority of the Chinese patent application filed with the Chinese Patent Office on June 8, 2020, the application number is 202010514057.9, and the invention title is "damage detection model training, vehicle damage detection methods, devices, equipment and media", all of which The content is incorporated in this application by reference.

Technical field

This application relates to the field of artificial intelligence classification models, and in particular to a damage detection model training, a vehicle damage detection method, device, computer equipment, and storage medium.

Background technique

The inventor found that after a traffic accident occurs in a vehicle, some parts of the vehicle will leave traces of damage, such as damage, scratches, etc. At present, insurance companies generally manually identify the images taken by the owner or business personnel of the vehicle damage after the traffic accident , That is, to manually identify and determine the damage type and damaged area of the damaged part of the vehicle in the image. In this way, due to the influence of inconsistent standard understanding and insufficient observation experience, the artificially recognized damage type and damaged area may not match; for example: Because it is difficult to distinguish between dents and scratches through visual images, damage assessment personnel can easily determine the type of damage caused by the dent as the type of scratch damage. The miscalculation caused by the above conditions will greatly reduce the accuracy of the damage assessment; While it may cause cost losses for the insurance company, it will also reduce the satisfaction of car owners or customers; in addition, the manual loss determination workload is huge and the loss determination efficiency is low. When a certain loss determination accuracy needs to be met, Will further increase the workload and reduce work efficiency.

Summary of the invention

This application provides a damage detection model training, a vehicle damage detection method, device, computer equipment, and storage medium, which can accurately and quickly identify the damage type and damage area in the image containing the damage location, and improve the alignment The accuracy and reliability of the determination of the loss type and the area of the fixed loss reduce the cost and improve the training efficiency.

A damage detection model training method includes:

Acquire a damage sample set; the damage sample set includes damage sample images, and one damage sample image is associated with a damage label group; the damage label group includes at least one damage label type and a mask corresponding to the damage label type Mark the map and at least one rectangular frame area;

The damage sample image is input into a damage detection model containing the first parameter, and the damage feature in the damage sample image is extracted through the damage detection model and an intermediate convolution feature map is generated; the damage detection model is based on the YOLOV3 model framework Deep convolutional neural network model;

Input the intermediate convolution feature map into the mask prediction branch model containing the second parameter;

The damage detection model outputs the training result according to the damage feature, and at the same time obtains the mask result through the mask prediction branch model; the training result includes at least one sample damage type and at least one sample damage rectangular area; the mask The code result is obtained and output according to the damage mask feature extracted from the intermediate convolution feature map, and the mask result includes at least one mask damage type and a mask tensor map corresponding to the mask damage type ；

Input all the damage label types, all the rectangular frame areas, all the sample damage types and all the sample damage rectangular areas of the damage sample image into the first loss model to obtain the first loss value, and at the same time Input all the damage label types, all the mask annotation maps, all the mask damage types, and all the mask tensor maps of the damage sample image into a second loss model to obtain a second loss value;

Determine a total loss value according to the first loss value and the second loss value;

When the total loss value does not reach the preset convergence condition, iteratively update the first parameter of the damage detection model and the second parameter of the mask prediction branch model until the total loss value reaches the preset When the convergence condition is set, the damage detection model after convergence is recorded as the training completed damage detection model.

A vehicle damage detection method, including:

Receive car damage detection instructions and obtain car damage images;

The car damage image is input into the damage detection model trained as described in the above damage detection model training method, the damage feature is extracted from the damage detection model, and the final result output by the damage detection model according to the damage feature is obtained; the final The result includes the damage type and the damage area, and the final result characterizes the damage type and the damage area of all the damage positions in the car damage image.

A damage detection model training device includes:

The acquisition module is used to acquire a damage sample set; the damage sample set includes damage sample images, and one damage sample image is associated with a damage label group; the damage label group includes at least one damage label type and the damage label The mask marking map corresponding to the type and at least one rectangular frame area;

The input module is configured to input the damage sample image into a damage detection model containing the first parameter, and extract damage features in the damage sample image from the damage detection model and generate an intermediate convolution feature map; the damage detection model It is a deep convolutional neural network model based on the YOLOV3 model framework;

A branching module, configured to input the intermediate convolution feature map into a mask prediction branch model containing a second parameter;

The output module is configured to output the training result according to the damage feature through the damage detection model, and at the same time obtain the mask result through the mask prediction branch model; the training result includes at least one sample damage type and at least one sample damage rectangle Region; the mask result is obtained and output according to the damage mask feature extracted from the intermediate convolution feature map, the mask result includes at least one mask damage type and the corresponding mask damage type Mask tensor map;

The loss module is used to input all the damage label types, all the rectangular frame areas, all the sample damage types, and all the sample damage rectangular areas of the damaged sample image into the first loss model to obtain the first loss Value, and input all the damage label types, all the mask annotation maps, all the mask damage types, and all the mask tensor maps of the damage sample image into the second loss model at the same time to obtain the second loss model. Loss value

A determining module, configured to determine a total loss value according to the first loss value and the second loss value;

The convergence module is configured to iteratively update the first parameter of the damage detection model and the second parameter of the mask prediction branch model when the total loss value does not reach the preset convergence condition, until the total loss value When the preset convergence condition is reached, the damage detection model after convergence is recorded as a damage detection model that has been trained.

A vehicle damage detection device, including:

The receiving module is used to receive the car damage detection instruction and obtain the car damage image;

The detection module is used to input the car damage image into the damage detection model trained by the above damage detection model training method, extract damage features from the damage detection model, and obtain the final output of the damage detection model according to the damage feature Result; the final result includes the damage type and the damage area, and the final result characterizes the damage type and the damage area of all the damage positions in the car damage image.

A computer device includes a memory, a processor, and computer-readable instructions that are stored in the memory and can run on the processor, and the processor implements the following steps when the processor executes the computer-readable instructions:

Acquire a damage sample set; the damage sample set includes damage sample images, and one damage sample image is associated with a damage label group; the damage label group includes at least one damage label type and a mask corresponding to the damage label type Mark the map and at least one rectangular frame area;

The damage sample image is input into a damage detection model containing the first parameter, and the damage feature in the damage sample image is extracted through the damage detection model and an intermediate convolution feature map is generated; the damage detection model is based on the YOLOV3 model framework Deep convolutional neural network model;

Input the intermediate convolution feature map into the mask prediction branch model containing the second parameter;

The damage detection model outputs the training result according to the damage feature, and at the same time obtains the mask result through the mask prediction branch model; the training result includes at least one sample damage type and at least one sample damage rectangular area; the mask The code result is obtained and output according to the damage mask feature extracted from the intermediate convolution feature map, and the mask result includes at least one mask damage type and a mask tensor map corresponding to the mask damage type ；

Input all the damage label types, all the rectangular frame areas, all the sample damage types and all the sample damage rectangular areas of the damage sample image into the first loss model to obtain the first loss value, and at the same time Input all the damage label types, all the mask annotation maps, all the mask damage types, and all the mask tensor maps of the damage sample image into a second loss model to obtain a second loss value;

Determine a total loss value according to the first loss value and the second loss value;

When the total loss value does not reach the preset convergence condition, iteratively update the first parameter of the damage detection model and the second parameter of the mask prediction branch model until the total loss value reaches the preset When the convergence condition is set, the damage detection model after convergence is recorded as the training completed damage detection model.

A computer device includes a memory, a processor, and computer-readable instructions that are stored in the memory and can run on the processor, and the processor further implements the following steps when the processor executes the computer-readable instructions:

Receive car damage detection instructions and obtain car damage images;

The car damage image is input to the damage detection model trained by the damage detection model training method, the damage feature is extracted from the damage detection model, and the final result output by the damage detection model according to the damage feature is obtained; the final result Including the damage type and the damage area, the final result represents the damage type and the damage area of all the damage positions in the car damage image.

One or more readable storage media storing computer readable instructions, when the computer readable instructions are executed by one or more processors, the one or more processors execute the following steps:

Acquire a damage sample set; the damage sample set includes damage sample images, and one damage sample image is associated with a damage label group; the damage label group includes at least one damage label type and a mask corresponding to the damage label type Mark the map and at least one rectangular frame area;

The damage sample image is input into a damage detection model containing the first parameter, and the damage feature in the damage sample image is extracted through the damage detection model and an intermediate convolution feature map is generated; the damage detection model is based on the YOLOV3 model framework Deep convolutional neural network model;

Input the intermediate convolution feature map into the mask prediction branch model containing the second parameter;

The damage detection model outputs the training result according to the damage feature, and at the same time obtains the mask result through the mask prediction branch model; the training result includes at least one sample damage type and at least one sample damage rectangular area; the mask The code result is obtained and output according to the damage mask feature extracted from the intermediate convolution feature map, and the mask result includes at least one mask damage type and a mask tensor map corresponding to the mask damage type ；

Input all the damage label types, all the rectangular frame areas, all the sample damage types and all the sample damage rectangular areas of the damage sample image into the first loss model to obtain the first loss value, and at the same time Input all the damage label types, all the mask annotation maps, all the mask damage types, and all the mask tensor maps of the damage sample image into a second loss model to obtain a second loss value;

Determine a total loss value according to the first loss value and the second loss value;

When the total loss value does not reach the preset convergence condition, iteratively update the first parameter of the damage detection model and the second parameter of the mask prediction branch model until the total loss value reaches the preset When the convergence condition is set, the damage detection model after convergence is recorded as the training completed damage detection model.

One or more readable storage media storing computer readable instructions, when the computer readable instructions are executed by one or more processors, the one or more processors further execute the following steps:

Receive car damage detection instructions and obtain car damage images;

The car damage image is input to the damage detection model trained by the damage detection model training method, the damage feature is extracted from the damage detection model, and the final result output by the damage detection model according to the damage feature is obtained; the final result Including the damage type and the damage area, the final result represents the damage type and the damage area of all the damage positions in the car damage image.

The damage detection model training method, device, computer equipment and storage medium provided in this application train the damage detection model based on the YOLOV3 model architecture by acquiring damage sample images containing damage label groups, and extract the damage features of the damage sample images The training result and the intermediate convolution feature map are obtained, and the damage mask feature is extracted from the intermediate convolution feature map through the mask prediction branch model to obtain the mask result. According to the damage label group, the training result and the mask As a result, the total loss value is determined, and the damage detection model is continuously iteratively trained by judging whether the total loss value reaches the preset convergence condition, and the damage detection model after convergence is recorded as the training damage detection model. Therefore, it provides A model training method is proposed. By increasing the mask prediction branch model for training, the number of sample collections can be reduced and the recognition accuracy and reliability can be improved, and the damage type in the image containing the damage location can be accurately and quickly identified. And the damage area, improve the accuracy and reliability of determining the type and area of the damage, reduce the cost, and improve the training efficiency.

The vehicle damage detection method, device, computer equipment, and storage medium provided in the present application acquire a vehicle damage image, input the vehicle damage image into the above-mentioned trained damage detection model, extract damage features through the damage detection model, and obtain all The damage detection model outputs a final result including damage type and damage area according to the damage feature; the final result represents the damage type and damage area of all damage positions in the car damage image, thus improving recognition Speed, thereby improving identification efficiency, reducing costs, and improving customer satisfaction.

The details of one or more embodiments of the present application are presented in the following drawings and description, and other features and advantages of the present application will become apparent from the description, drawings and claims.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

FIG. 1 is a schematic diagram of an application environment of a damage detection model training method or a car damage detection method in an embodiment of the present application;

2 is a flowchart of a method for training a damage detection model in an embodiment of the present application;

3 is a flowchart of step S10 of the damage detection model training method in an embodiment of the present application;

4 is a flowchart of step S40 of the damage detection model training method in an embodiment of the present application;

FIG. 5 is a flowchart of step S401 of the damage detection model training method in an embodiment of the present application;

Fig. 6 is a flowchart of a vehicle damage detection method in an embodiment of the present application;

Fig. 7 is a schematic block diagram of a damage detection model training device in an embodiment of the present application;

Fig. 8 is a schematic block diagram of a vehicle damage detection device in an embodiment of the present application;

Fig. 9 is a schematic diagram of a computer device in an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The damage detection model training method provided by this application can be applied in the application environment as shown in Fig. 1, where the client (computer equipment) communicates with the server through the network. Among them, the client (computer equipment) includes, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, cameras, and portable wearable devices. The server can be implemented as an independent server or a server cluster composed of multiple servers.

In an embodiment, as shown in FIG. 2, a method for training a recognition model is provided, and the technical solution mainly includes the following steps S10-S70:

S10. Acquire a damage sample set; the damage sample set includes damage sample images, and one damage sample image is associated with a damage label group; the damage label group includes at least one damage label type corresponding to the damage label type The mask marks the map and at least one rectangular frame area.

Understandably, the damage sample set includes a plurality of damage sample images, the damage sample set is a collection of all the damage sample images, and the damage sample images may be historically collected and contain a vehicle in a traffic accident. The damaged vehicle image or photo can also be a fused image according to requirements. One damage sample image corresponds to one damage label group, and the damage label group includes the damage label type and the mask label. Figure and the rectangular frame area, the damage label types include scratches, scratches, dents, wrinkles, dead folds, tears, missing, etc. 7 types of damage, the mask marked image shows through each damage label type The corresponding mask value replaces each pixel value in the area range of the damage location, that is, according to the type of damage label corresponding to each damage location, the mask value corresponding to the damage label type is used to fill the area of the damage location. For each pixel value, the pixel value with the same mask value is translated to a channel image of the same size as the damage sample image to form 7 channel images containing the mask value corresponding to the damage label type. The rectangular frame area is the coordinate area that can cover the damage location through a rectangular frame with the smallest area.

In one embodiment, as shown in FIG. 3, before the step S10, that is, before the damage sample set is obtained, the method includes:

S101: Obtain a sample image and a public data image; the sample image is a shot image containing a damage location, and the public data image is an image randomly selected from a KITTI data set.

Understandably, the sample image is an image taken in history and contains the location of damage left by the vehicle after a traffic accident, the public data image is an image randomly extracted from the KITTI data set, and the KITTI data The set is a collection of images related to the public smart vehicle. The size of the public data image is converted to the same size as the sample image through the resize method. The resize method can be set according to requirements, such as resize The methods are nearest neighbor interpolation algorithm, bilinear interpolation algorithm, bicubic interpolation algorithm, interpolation algorithm based on pixel region relationship, Lanzos interpolation interpolation algorithm, and so on.

S102: Perform fusion processing on the sample image and the public data image by using a mixup method to obtain a fused sample image.

Understandably, the mixup method is to perform weighting processing and fusion processing for each pixel value in the sample and image and the corresponding pixel value in the public data image through a preset ratio to generate the fusion The pixel value of the sample image, the fusion processing is to weight each pixel value in the sample and image with the corresponding pixel value in the public data image and then sum up to obtain the corresponding in the fused sample image The pixel value.

S103: Determine the fused sample image as a damaged sample image corresponding to the sample image, and store the damaged sample image in a blockchain.

Understandably, the fused sample image is marked as the damaged sample image, and the damaged sample image is stored in the blockchain.

It should be emphasized that, in order to further ensure the privacy and security of the damaged sample image, the damaged sample image can also be stored in a node of the blockchain.

Among them, the blockchain referred to in this application is a new application mode of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information for verification. The validity of the information (anti-counterfeiting) and the generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer. The decentralized and fully distributed DNS service provided by the blockchain can realize the query and resolution of domain names through the point-to-point data transmission service between various nodes in the network, which can be used to ensure that the operating system and firmware of an important infrastructure are not available. If it is tampered with, it can monitor the status and integrity of the data, find bad tampering, and ensure that the transmitted data has not been tampered with. Store the damaged sample image in the blockchain, which can ensure the privacy and security of the damaged sample image Sex.

This application uses the mixup method to perform fusion processing on the sample image and the public data image (randomly extracted from the KITTI data set) to obtain a fusion sample image, and determine the fusion sample image as the damaged sample corresponding to the sample image And storing the damaged sample image in the blockchain can improve the security of the damaged sample image, prevent it from being tampered with, prevent the damaged sample image from overfitting in the subsequent recognition process, and improve the recognition accuracy.

S20. Input the damage sample image to a damage detection model containing the first parameter, extract damage features in the damage sample image through the damage detection model and generate an intermediate convolution feature map; the damage detection model is based on YOLOV3 The deep convolutional neural network model of the model architecture.

Understandably, the damage detection model is a YOLOV3 model-based deep convolutional neural network model that recognizes the sample damage type and the sample damage rectangular area in the damage sample image, that is, the network structure of the damage detection model and the YOLOV3 model The network structure is the same. The damage features include seven types of damage including scratches, scratches, dents, folds, deadfolds, tears, and missing. The first parameters of the damage detection model can be performed according to requirements. Setting, for example, the first parameter can obtain all the parameters of the YOLOV3 model through the transfer learning method, or all can be set to a preset value.

S30: Input the intermediate convolution feature map into a mask prediction branch model containing a second parameter.

Understandably, the mask prediction branch model is a preset convolutional neural network model, and the second parameter of the mask prediction branch model can be set according to requirements, for example, the second parameter is random The parameter value.

S40. Output a training result according to the damage feature through the damage detection model, and at the same time obtain a mask result through the mask prediction branch model; the training result includes at least one sample damage type and at least one sample damage rectangular area; The mask result is obtained and output according to the damage mask feature extracted from the intermediate convolution feature map, and the mask result includes at least one mask damage type and a mask corresponding to the mask damage type.量图。 Volume chart.

Understandably, the damage detection model predicts according to the extracted damage feature to obtain the training result. The training result includes the sample damage type and the sample damage rectangular area, and the sample damage rectangular area is the same as the sample damage rectangular area. The area coordinate range corresponding to the damage type, through the mask prediction branch model, the extraction of damage mask features can be increased, and the identification of the mask damage type can be enhanced, and the damage mask feature is the mask value corresponding to the damage label type Related features, that is, the addition of a mask prediction branch for recognition can improve the accuracy and precision of damage recognition. The mask result includes the mask damage type and the mask tensor map. The code tensor map is a feature vector map that identifies the damage location of the same damage type in the damage sample image, and also refers to the feature vector map corresponding to the masked damage type.

Among them, the sample damage types include 7 types of damage, including scratches, scratches, dents, folds, dead folds, tears, and missing. The mask damage types include scratches, scratches, dents, folds, and dead folds. 7 types of injuries including, tearing and missing.

In an embodiment, as shown in FIG. 4, in the step S40, that is, obtaining the mask result through the mask prediction branch model includes:

S401. Input the intermediate convolution feature map to an expansion module in the mask prediction branch model, and perform damage mask feature extraction and expansion processing on the intermediate convolution feature map through the expansion module to obtain a multi-channel feature Figure.

Understandably, the expansion module is to obtain the multi-channel feature map by extracting the damage mask feature from the 32×32 sized feature vector graph, and the multi-channel feature The graph contains multiple 256×256 feature vector graphs (also tensor graphs in the full text).

In one embodiment, as shown in FIG. 5, in the step S401, the intermediate convolution feature map is input to the expansion module in the mask prediction branch model, and the intermediate convolution is performed by the expansion module. The feature map is expanded to obtain a multi-channel feature map, including:

S40101. Input the intermediate convolutional feature map to the first convolutional layer in the expansion module, and perform the damage mask feature extraction on the intermediate convolutional feature map through the first convolutional layer to obtain the first convolutional feature map. A feature map;

Understandably, the expansion module includes a first convolutional layer, a first sampling layer, a second convolutional layer, a second sampling layer, a third convolutional layer, and a third sampling layer, and the first convolutional layer includes A 256-channel 3×3 convolution kernel and a 128-channel 1×1 convolution kernel, the first convolution layer convolves the intermediate convolution feature map through a 256-channel 3×3 convolution kernel After the integration, a 128-channel 1×1 convolution kernel is used for convolution, so as to extract the damage mask feature.

S40102, performing up-sampling processing on the first feature map 1 through the first sampling layer in the expansion module to obtain a first sampling map;

Understandably, the up-sampling process is to perform size expansion and filling processing on a feature vector image to a preset size. The first sampling image is a 64×64 feature vector image, and the first sampling layer can be updated. Obtain the damage mask feature with a high probability, thereby preventing overfitting and improving generalization.

S40103. Input the first sampling image to a second convolutional layer in the expansion module, and perform the damage mask feature extraction on the first sampling image through the second convolutional layer to obtain a second feature Figure;

Understandably, the second convolutional layer includes a 128-channel 3×3 convolution kernel and a 64-channel 1×1 convolution kernel. The second convolution layer passes through a The 128-channel 3×3 convolution kernel performs convolution and then passes through a 64-channel 1×1 convolution kernel to perform convolution, so as to extract the damage mask feature.

S40104, performing up-sampling processing on the second feature map through the second sampling layer in the expansion module to obtain a second sampling map;

Understandably, the up-sampling process is to perform size expansion and filling processing on a feature vector image to a preset size, the second sampling image is a 128×128 feature vector image, and the second sampling layer can be updated. Obtain the damage mask feature with a high probability, thereby preventing overfitting and improving generalization.

S40105. Input the second sampling image to a third convolutional layer in the expansion module, and perform the damage mask feature extraction on the second sampling image through the third convolutional layer to obtain a third feature Figure;

Understandably, the third convolution layer includes a 64-channel 3×3 convolution kernel and a 32-channel 1×1 convolution kernel, and the third convolution layer passes through a The 64-channel 3×3 convolution kernel performs convolution and then passes through a 32-channel 1×1 convolution kernel to perform convolution, so as to further extract the damage mask feature.

S40106: Perform up-sampling processing on the third feature map through the third sampling layer in the expansion module to obtain a multi-channel feature map.

Understandably, the multi-channel feature map is a 256×256 feature vector map, and the third sampling layer can further obtain the damage mask feature, thereby preventing overfitting and improving generalization.

S402. Input the multi-channel feature map to a classification module in the mask prediction branch model, and perform classification and prediction processing on the multi-channel feature map through the classification module to obtain the corresponding intermediate convolution feature map. Mask prediction result.

Understandably, the mask damage prediction type includes 7 damage types such as scratches, scratches, dents, wrinkles, dead-folds, tears, and missing. The classification module in the branch model is predicted by the mask. The multi-channel feature map is classified, that is, the feature vector map in the multi-channel feature map is classified to obtain the feature vector map corresponding to all mask prediction damage types, and the feature vector corresponding to the mask prediction damage type is obtained Figure, predicts the mask prediction tensor map corresponding to the mask prediction damage type, the mask prediction tensor diagram is a channel containing the predicted pixel value corresponding to each pixel and is related to the mask prediction damage type The feature vector map of the mask, the mask prediction result includes the mask prediction damage type and the mask prediction tensor map.

S403: Determine the mask result corresponding to the damaged sample image according to the mask prediction result corresponding to the intermediate convolution feature map.

Understandably, according to the comparison between the mask prediction result and the preset probability value, the mask prediction tensor map conforming to the probability value is retained, and all the mask prediction tensor maps after retention are determined as The mask tensor map corresponding to the damage sample image, so that the mask prediction damage type corresponding to the mask prediction tensor map is determined to be the damage sample according to the retained mask prediction tensor map The mask damage type corresponding to the image, and all the mask tensor maps and the corresponding mask damage types are determined as the mask result of the damage sample image.

This application performs damage mask feature extraction and expansion processing on the intermediate convolution feature map through the expansion module in the mask prediction branch model to obtain a multi-channel feature map; and then predicts the classification in the branch model through the mask The module classifies and predicts the multi-channel feature map to obtain the mask prediction result corresponding to the intermediate convolution feature map; determines the damage sample image according to the mask prediction result corresponding to the intermediate convolution feature map Corresponding to the mask result, in this way, a mask prediction branch model is provided to extract the damage mask feature and obtain the mask result, which provides a method to improve the accuracy of the subsequent training of the damage detection model, and reduces the damage detection. The training time of the model and the number of samples, thereby reducing the cost.

S50. Input all the damage label types, all the rectangular frame areas, all the sample damage types and all the sample damage rectangular areas of the damaged sample image into a first loss model to obtain a first loss value, and at the same time Input all the damage label types, all the mask annotation maps, all the mask damage types, and all the mask tensor maps of the damage sample image into a second loss model to obtain a second loss value.

Understandably, the first loss model includes the first loss function, and all the damage label types, all the rectangular box areas, all the sample damage types, and all the sample damage rectangular areas are input into the The first loss function, the first loss value is calculated by the cross-entropy method; the second loss value model includes the second loss function, and all the damage label types and all the damage label types of the damage sample image are calculated. The mask annotation map, all the mask damage types, and all the mask tensor maps are input to the second loss function, and the second loss value is calculated by a cross-entropy method.

S60: Determine a total loss value according to the first loss value and the second loss value.

Understandably, the first loss value and the second loss value are input to a loss model containing a total loss function. The total loss function in the loss model can be set according to requirements, and the loss model is for generating the A model of the total loss value, and the total loss value is calculated by the total loss function.

In an embodiment, the step 60, that is, the determining the total loss value according to the first loss value and the second loss value, includes:

S601. Input the first loss value and the second loss value into a preset loss model, and calculate the total loss value through the total loss function in the loss model; the total loss function is:

L=w ₁ ×X1+w ₂ ×X2

among them,

X1 is the first loss value;

X2 is the second loss value;

w ₁ is the weight of the first loss value;

w ₂ is the weight of the second loss value.

S70: When the total loss value does not reach a preset convergence condition, iteratively update the first parameter of the damage detection model and the second parameter of the mask prediction branch model until the total loss value reaches the When the convergence condition is preset, the damage detection model after convergence is recorded as the damage detection model that has been trained.

Understandably, the convergence condition may be a condition that the value of the total loss value is small and will not drop after 9000 calculations, that is, the value of the total loss value is small and will not decrease after 9000 calculations. When it will no longer fall, stop training, and record the damage detection model after convergence as the completed damage detection model; the convergence condition can also be the condition that the total loss value is less than the set threshold, that is, the damage detection model When the total loss value is less than the set threshold, the training is stopped, and the damage detection model after convergence is recorded as the training completed damage detection model.

In this way, when the total loss value does not reach the preset convergence condition, the first parameter of the damage detection model and the second parameter of the mask prediction branch model are continuously updated and iterated, so that the accurate results can be continuously moved closer. Make the recognition accuracy higher and higher.

In an embodiment, after the step S60, that is, after the total loss value is determined according to the first loss value and the second loss value, the method further includes:

S80: When the total loss value reaches a preset convergence condition, record the converged damage detection model as a trained damage detection model.

Understandably, when the total loss value reaches the preset convergence condition, it indicates that the total loss value has reached the optimal result. At this time, the damage detection model has converged, and the damage detection model after convergence will be The record is a trained damage detection model. In this way, according to the damage sample image in the damage sample set, the trained damage detection model is obtained through continuous training, which can improve the accuracy and reliability of recognition.

This application obtains a damage sample set; the damage sample set includes damage sample images, and one damage sample image is associated with a damage label group; the damage label group includes at least one damage label type corresponding to the damage label type And at least one rectangular frame area; input the damage sample image into a damage detection model containing the first parameter, and extract the damage feature in the damage sample image through the damage detection model and generate an intermediate convolution feature Figure; The damage detection model is a deep convolutional neural network model based on the YOLOV3 model framework; the intermediate convolution feature map is input to the mask prediction branch model containing the second parameter; the damage detection model is based on the damage The feature output contains the training result of the sample damage type and the sample damage rectangular area, and at the same time, the mask result containing the sample damage type and the sample damage rectangular area is obtained through the mask prediction branch model; the mask result is based on the intermediate The damage mask feature extracted from the convolution feature map is obtained and output. The mask result includes at least one mask damage type and a mask tensor map corresponding to the mask damage type; All the damage label types, all the rectangular frame areas, all the sample damage types, and all the sample damage rectangular areas are input into the first loss model to obtain the first loss value, and at the same time, all the damage samples in the damaged sample image are input to the first loss model. Input the damage label type, all the mask annotation graphs, all the mask damage types, and all the mask tensor graphs into the second loss model to obtain a second loss value; according to the first loss value and the The second loss value determines the total loss value; when the total loss value does not reach the preset convergence condition, iteratively update the first parameter of the damage detection model and the second parameter of the mask prediction branch model, Until the total loss value reaches the preset convergence condition, the damage detection model after convergence is recorded as the trained damage detection model.

This application realizes that by acquiring damage sample images containing damage label groups, training a damage detection model based on the YOLOV3 model architecture, extracting the damage features of the damage sample image and obtaining the training result and the intermediate convolution feature map, through the The mask prediction branch model extracts the damage mask feature from the intermediate convolution feature map to obtain the mask result, and determines the total loss value according to the damage label group, the training result and the mask result, and determines the total loss value Whether the preset convergence condition is reached, the damage detection model is continuously iteratively trained, and the damage detection model after convergence is recorded as the training damage detection model. Therefore, a model training method is provided to predict the branch model by adding a mask Training can reduce the number of sample collections and improve the accuracy and reliability of recognition. It can accurately and quickly identify the damage type and damage area in the image containing the damage location, and improve the damage assessment type and damage area. The accuracy and reliability of the determination reduces the cost and improves the training efficiency.

The vehicle damage detection method provided in this application can be applied in the application environment as shown in Fig. 1, in which the client (computer equipment) communicates with the server through the network. Among them, the client (computer equipment) includes, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, cameras, and portable wearable devices. The server can be implemented as an independent server or a server cluster composed of multiple servers.

In an embodiment, as shown in FIG. 6, a vehicle damage detection method is provided, and the technical solution mainly includes the following steps S100-S200:

S100, receiving a car damage detection instruction, and acquiring a car damage image;

Understandably, after a traffic accident, the vehicle will leave traces of damage. The staff of the insurance company will take photos related to the traffic accident. These photos include photos of the vehicle damage. The staff upload the photos of the vehicle damage to the server. To trigger the vehicle damage detection instruction to obtain the vehicle damage image contained in the vehicle damage detection instruction, where the vehicle damage image is a photograph of the vehicle damage taken.

S200. Input the car damage image into the above-mentioned trained damage detection model, extract damage features from the damage detection model, and obtain a final result output by the damage detection model according to the damage feature; the final result includes the damage type And the damage area, the final result characterizes the damage type and damage area of all the damage positions in the car damage image.

Understandably, it is only necessary to input the car damage image into a trained damage detection model, and extract the damage feature through the damage detection model. The damage detection model is based on the damage feature in the car damage image. The final result is output, and the final result characterizes the damage type and damage area of all damage positions in the car damage image. In this process, the mask prediction branch model does not need to be used, which speeds up the recognition speed. Improved recognition efficiency.

This application acquires a car damage image, inputs the car damage image into the above-mentioned trained damage detection model, extracts damage features from the damage detection model, and obtains the damage type output by the damage detection model according to the damage feature And the final result of the damage area; the final result characterizes the damage type and damage area of all the damage locations in the car damage image, so that the recognition speed is improved, thereby improving the recognition efficiency, reducing the cost, and improving the customer Satisfaction.

In one embodiment, a damage detection model training device is provided, and the damage detection model training device corresponds to the damage detection model training method in the above-mentioned embodiment in a one-to-one correspondence. As shown in FIG. 7, the damage detection model training device includes an acquisition module 11, an input module 12, a branch module 13, an output module 14, a loss module 15, a determination module 16 and a convergence module 17. The detailed description of each functional module is as follows:

The acquisition module 11 is configured to acquire a damage sample set; the damage sample set includes damage sample images, and one damage sample image is associated with a damage label group; the damage label group includes at least one damage label type that is related to the damage The mask label corresponding to the label type and at least one rectangular frame area;

The input module 12 is configured to input the damage sample image into a damage detection model containing the first parameter, and extract damage features in the damage sample image through the damage detection model and generate an intermediate convolution feature map; the damage detection The model is a deep convolutional neural network model based on the YOLOV3 model framework;

The branch module 13 is configured to input the intermediate convolution feature map into a mask prediction branch model containing the second parameter;

The output module 14 is configured to output the training result according to the damage feature through the damage detection model, and at the same time obtain the mask result through the mask prediction branch model; the training result includes at least one sample damage type and at least one sample damage Rectangular area; the mask result is obtained and output according to the damage mask feature extracted from the intermediate convolution feature map, the mask result includes at least one mask damage type and corresponding to the mask damage type The mask tensor map;

The loss module 15 is configured to input all the damage label types, all the rectangular frame areas, all the sample damage types, and all the sample damage rectangular areas of the damaged sample image into a first loss model to obtain a first loss model. Loss value, and input all the damage label types, all the mask annotation maps, all the mask damage types and all the mask tensor maps of the damage sample image into the second loss model at the same time, to obtain the first loss model Two loss value;

The determining module 16 is configured to determine a total loss value according to the first loss value and the second loss value;

The convergence module 17 is configured to iteratively update the first parameter of the damage detection model and the second parameter of the mask prediction branch model when the total loss value does not reach the preset convergence condition, until the total loss When the value reaches the preset convergence condition, the damage detection model after convergence is recorded as the training completed damage detection model.

In an embodiment, the determining module 16 includes:

The calculation unit is configured to input the first loss value and the second loss value into a preset loss model, and calculate the total loss value through the total loss function in the loss model; the total loss function is :

L=w ₁ ×X1+w ₂ ×X2

among them,

X1 is the first loss value;

X2 is the second loss value;

w ₁ is the weight of the first loss value;

w ₂ is the weight of the second loss value.

In an embodiment, the acquisition module 11 includes:

An acquiring unit for acquiring a sample image and a public data image; the sample image is a photographed image containing the damage location, and the public data image is an image randomly extracted from the KITTI data set;

The fusion unit is configured to perform fusion processing on the sample image and the public data image by a mixup method to obtain a fusion sample image;

The determining unit is configured to determine the fused sample image as the damaged sample image corresponding to the sample image, and store the damaged sample image in the blockchain.

In an embodiment, the output module 14 includes:

The branching unit is configured to input the intermediate convolution feature map into an expansion module in the mask prediction branch model, and perform damage mask feature extraction and expansion processing on the intermediate convolution feature map through the expansion module to obtain Multi-channel feature map;

The prediction unit is configured to input the multi-channel feature map into the classification module in the mask prediction branch model, and perform classification and prediction processing on the multi-channel feature map through the classification module to obtain the intermediate convolution feature The mask prediction result corresponding to the graph;

The output unit is configured to determine the mask result corresponding to the damaged sample image according to the mask prediction result corresponding to the intermediate convolution feature map.

In an embodiment, the branch unit includes:

The first convolution subunit is configured to input the intermediate convolution feature map into the first convolution layer in the expansion module, and perform the damage on the intermediate convolution feature map through the first convolution layer Mask feature extraction to obtain the first feature map;

The first sampling subunit is configured to perform up-sampling processing on the first feature map 1 through the first sampling layer in the expansion module to obtain a first sampling map;

The second convolution subunit is configured to input the first sample image into the second convolution layer in the expansion module, and perform the damage mask on the first sample image through the second convolution layer Feature extraction to obtain a second feature map;

The second sampling subunit is configured to perform up-sampling processing on the second feature map through the second sampling layer in the expansion module to obtain a second sampling map;

The third convolution subunit is used to input the second sample image into the third convolution layer in the expansion module, and perform the damage mask on the second sample image through the third convolution layer Feature extraction to obtain the third feature map;

The third sampling subunit is configured to perform up-sampling processing on the third feature map through the third sampling layer in the expansion module to obtain a multi-channel feature map.

For the specific definition of the damage detection model training device, please refer to the above definition of the damage detection model training method, which will not be repeated here. Each module in the above-mentioned damage detection model training device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules may be embedded in the form of hardware or independent of the processor in the computer equipment, or may be stored in the memory of the computer equipment in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

In one embodiment, a vehicle damage detection device is provided, and the vehicle damage detection device corresponds to the vehicle damage detection method in the above-mentioned embodiment in a one-to-one correspondence. As shown in FIG. 8, the vehicle damage detection device includes an acquisition module 101 and a detection module 102. The detailed description of each functional module is as follows:

The receiving module 101 is configured to receive a car damage detection instruction and obtain a car damage image;

The detection module 102 is configured to input the car damage image into the damage detection model trained as the above damage detection model training method, extract damage features from the damage detection model, and obtain the output of the damage detection model according to the damage feature. Final result; the final result includes damage type and damage area, and the final result characterizes the damage type and damage area of all damage locations in the car damage image.

For the specific limitation of the vehicle damage detection device, please refer to the above limitation of the vehicle damage detection method, which will not be repeated here. The various modules in the vehicle damage detection device described above can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules may be embedded in the form of hardware or independent of the processor in the computer equipment, or may be stored in the memory of the computer equipment in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. 9. The computer equipment includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a readable storage medium and an internal memory. The readable storage medium stores an operating system, computer readable instructions, and a database. The internal memory provides an environment for the operation of the operating system and computer readable instructions in the readable storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer-readable instruction is executed by the processor to realize a damage detection model training method or a vehicle damage detection method. The readable storage medium provided in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.

In one embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and capable of running on the processor. When the processor executes the computer-readable instructions, the damage in the above-mentioned embodiment is realized. The detection model training method, or the processor executes the computer-readable instructions to implement the vehicle damage detection method in the above embodiment.

In one embodiment, one or more readable storage media storing computer readable instructions are provided. The readable storage media provided in this embodiment include non-volatile readable storage media and volatile readable storage. Medium; the readable storage medium stores computer readable instructions, and when the computer readable instructions are executed by one or more processors, the one or more processors implement the damage detection model training method in the above-mentioned embodiments, or the computer When the program is executed by the processor, the vehicle damage detection method in the foregoing embodiment is implemented.

A person of ordinary skill in the art can understand that all or part of the processes in the methods of the above-mentioned embodiments can be implemented by instructing relevant hardware through computer-readable instructions. The computer-readable instructions can be stored in a non-volatile computer. In a readable storage medium or a volatile readable storage medium, when the computer readable instruction is executed, it may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database, or other media used in the embodiments provided in this application may include non-volatile and/or volatile memory. Non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not a limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Channel (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Those skilled in the art can clearly understand that for the convenience and conciseness of description, only the division of the above functional units and modules is used as an example. In practical applications, the above functions can be allocated to different functional units and modules as required. Module completion, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, a person of ordinary skill in the art should understand that it can still implement the foregoing The technical solutions recorded in the examples are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in Within the scope of protection of this application.

Claims (20)

1. A damage detection model training method, which includes:

   Acquire a damage sample set; the damage sample set includes damage sample images, and one damage sample image is associated with a damage label group; the damage label group includes at least one damage label type and a mask corresponding to the damage label type Mark the map and at least one rectangular frame area;

   The damage sample image is input into a damage detection model containing the first parameter, and the damage feature in the damage sample image is extracted through the damage detection model and an intermediate convolution feature map is generated; the damage detection model is based on the YOLOV3 model framework Deep convolutional neural network model;

   Input the intermediate convolution feature map into the mask prediction branch model containing the second parameter;

   The damage detection model outputs the training result according to the damage feature, and at the same time obtains the mask result through the mask prediction branch model; the training result includes at least one sample damage type and at least one sample damage rectangular area; the mask The code result is obtained and output according to the damage mask feature extracted from the intermediate convolution feature map, and the mask result includes at least one mask damage type and a mask tensor map corresponding to the mask damage type ；

   Input all the damage label types, all the rectangular frame areas, all the sample damage types and all the sample damage rectangular areas of the damage sample image into the first loss model to obtain the first loss value, and at the same time Input all the damage label types, all the mask annotation maps, all the mask damage types, and all the mask tensor maps of the damage sample image into a second loss model to obtain a second loss value;

   Determine a total loss value according to the first loss value and the second loss value;

   When the total loss value does not reach the preset convergence condition, iteratively update the first parameter of the damage detection model and the second parameter of the mask prediction branch model until the total loss value reaches the preset When the convergence condition is set, the damage detection model after convergence is recorded as the training completed damage detection model.
2. The damage detection model training method according to claim 1, wherein the determining the total loss value according to the first loss value and the second loss value comprises:

   The first loss value and the second loss value are input into a preset loss model, and the total loss value is calculated by the total loss function in the loss model; the total loss function is:

   L=w ₁ ×X1+w ₂ ×X2

   among them,

   X1 is the first loss value;

   X2 is the second loss value;

   w ₁ is the weight of the first loss value;

   w ₂ is the weight of the second loss value.
3. The damage detection model training method according to claim 1, wherein before obtaining the damage sample set, it comprises:

   Acquire sample images and public data images; the sample images are images taken with damage locations, and the public data images are randomly selected images in the KITTI data set;

   Fusion processing the sample image and the public data image by using a mixup method to obtain a fused sample image;

   The fusion sample image is determined as the damaged sample image corresponding to the sample image, and the damaged sample image is stored in the blockchain.
4. The damage detection model training method according to claim 1, wherein said obtaining a mask result through the mask prediction branch model comprises:

   Input the intermediate convolution feature map to an expansion module in the mask prediction branch model, and perform damage mask feature extraction and expansion processing on the intermediate convolution feature map through the expansion module to obtain a multi-channel feature map;

   The multi-channel feature map is input to the classification module in the mask prediction branch model, and the multi-channel feature map is classified and predicted by the classification module to obtain the mask corresponding to the intermediate convolution feature map forecast result;

   According to the mask prediction result corresponding to the intermediate convolution feature map, the mask result corresponding to the damaged sample image is determined.
5. The damage detection model training method according to claim 4, wherein the intermediate convolution feature map is input to an expansion module in the mask prediction branch model, and the intermediate convolution feature map is processed by the expansion module. Expand processing to obtain multi-channel feature maps, including:

   The intermediate convolution feature map is input to the first convolution layer in the expansion module, and the damage mask feature extraction is performed on the intermediate convolution feature map through the first convolution layer to obtain the first feature Figure;

   Up-sampling the first feature map 1 through the first sampling layer in the expansion module to obtain a first sampling map;

   Input the first sampling image to the second convolutional layer in the expansion module, and perform the damage mask feature extraction on the first sampling image through the second convolutional layer to obtain a second feature image;

   Performing up-sampling processing on the second feature map through the second sampling layer in the expansion module to obtain a second sampling map;

   Input the second sampling image to the third convolutional layer in the expansion module, and perform the damage mask feature extraction on the second sampling image through the third convolutional layer to obtain a third feature image;

   Up-sampling processing is performed on the third feature map through the third sampling layer in the expansion module to obtain a multi-channel feature map.
6. A vehicle damage detection method, which includes:

   Receive car damage detection instructions and obtain car damage images;

   The car damage image is input into the damage detection model trained by the damage detection model training method according to any one of claims 1 to 5, the damage feature is extracted from the damage detection model, and the damage detection model is obtained according to the damage The final result of the feature output; the final result includes the damage type and the damage area, and the final result represents the damage type and the damage area of all the damage positions in the car damage image.
7. A damage detection model training device, which includes:

   The acquisition module is used to acquire a damage sample set; the damage sample set includes damage sample images, and one damage sample image is associated with a damage label group; the damage label group includes at least one damage label type and the damage label The mask marking map corresponding to the type and at least one rectangular frame area;

   The input module is configured to input the damage sample image into a damage detection model containing the first parameter, and extract damage features in the damage sample image from the damage detection model and generate an intermediate convolution feature map; the damage detection model It is a deep convolutional neural network model based on the YOLOV3 model framework;

   A branching module, configured to input the intermediate convolution feature map into a mask prediction branch model containing a second parameter;

   The output module is configured to output the training result according to the damage feature through the damage detection model, and at the same time obtain the mask result through the mask prediction branch model; the training result includes at least one sample damage type and at least one sample damage rectangle Region; the mask result is obtained and output according to the damage mask feature extracted from the intermediate convolution feature map, the mask result includes at least one mask damage type and the corresponding mask damage type Mask tensor map;

   The loss module is used to input all the damage label types, all the rectangular frame areas, all the sample damage types, and all the sample damage rectangular areas of the damaged sample image into the first loss model to obtain the first loss Value, and input all the damage label types, all the mask annotation maps, all the mask damage types, and all the mask tensor maps of the damage sample image into the second loss model at the same time to obtain the second loss model. Loss value

   A determining module, configured to determine a total loss value according to the first loss value and the second loss value;

   The convergence module is configured to iteratively update the first parameter of the damage detection model and the second parameter of the mask prediction branch model when the total loss value does not reach the preset convergence condition, until the total loss value When the preset convergence condition is reached, the damage detection model after convergence is recorded as a damage detection model that has been trained.
8. A vehicle damage detection device, which includes:

   The receiving module is used to receive the car damage detection instruction and obtain the car damage image;

   The detection module is configured to input the car damage image into the damage detection model trained by the damage detection model training method according to any one of claims 1 to 5, and extract damage features from the damage detection model to obtain the damage detection The model outputs a final result according to the damage feature; the final result includes a damage type and a damage area, and the final result represents the damage type and damage area of all damage locations in the car damage image.
9. A computer device includes a memory, a processor, and computer-readable instructions that are stored in the memory and can run on the processor, wherein the processor implements the following steps when the processor executes the computer-readable instructions:

   Acquire a damage sample set; the damage sample set includes damage sample images, and one damage sample image is associated with a damage label group; the damage label group includes at least one damage label type and a mask corresponding to the damage label type Mark the map and at least one rectangular frame area;

   The damage sample image is input into a damage detection model containing the first parameter, and the damage feature in the damage sample image is extracted through the damage detection model and an intermediate convolution feature map is generated; the damage detection model is based on the YOLOV3 model framework Deep convolutional neural network model;

   Input the intermediate convolution feature map into the mask prediction branch model containing the second parameter;

   The damage detection model outputs the training result according to the damage feature, and at the same time obtains the mask result through the mask prediction branch model; the training result includes at least one sample damage type and at least one sample damage rectangular area; the mask The code result is obtained and output according to the damage mask feature extracted from the intermediate convolution feature map, and the mask result includes at least one mask damage type and a mask tensor map corresponding to the mask damage type ；

   Input all the damage label types, all the rectangular frame areas, all the sample damage types and all the sample damage rectangular areas of the damage sample image into the first loss model to obtain the first loss value, and at the same time Input all the damage label types, all the mask annotation maps, all the mask damage types, and all the mask tensor maps of the damage sample image into a second loss model to obtain a second loss value;

   Determine a total loss value according to the first loss value and the second loss value;

   When the total loss value does not reach the preset convergence condition, iteratively update the first parameter of the damage detection model and the second parameter of the mask prediction branch model until the total loss value reaches the preset When the convergence condition is set, the damage detection model after convergence is recorded as the training completed damage detection model.
10. 9. The computer device according to claim 9, wherein said determining the total loss value according to the first loss value and the second loss value comprises:

    The first loss value and the second loss value are input into a preset loss model, and the total loss value is calculated by the total loss function in the loss model; the total loss function is:

    L=w ₁ ×X1+w ₂ ×X2

    among them,

    X1 is the first loss value;

    X2 is the second loss value;

    w ₁ is the weight of the first loss value;

    w ₂ is the weight of the second loss value.
11. 9. The computer device according to claim 9, wherein, before acquiring the damage sample set, the processor further implements the following steps when executing the computer-readable instructions:

    Acquire sample images and public data images; the sample images are images taken with damage locations, and the public data images are randomly selected images in the KITTI data set;

    Fusion processing the sample image and the public data image by using a mixup method to obtain a fused sample image;

    The fusion sample image is determined as the damaged sample image corresponding to the sample image, and the damaged sample image is stored in the blockchain.
12. The computer device according to claim 9, wherein said obtaining the mask result through the mask prediction branch model comprises:

    Input the intermediate convolution feature map to an expansion module in the mask prediction branch model, and perform damage mask feature extraction and expansion processing on the intermediate convolution feature map through the expansion module to obtain a multi-channel feature map;

    The multi-channel feature map is input to the classification module in the mask prediction branch model, and the multi-channel feature map is classified and predicted by the classification module to obtain the mask corresponding to the intermediate convolution feature map forecast result;

    According to the mask prediction result corresponding to the intermediate convolution feature map, the mask result corresponding to the damaged sample image is determined.
13. The computer device according to claim 12, wherein the intermediate convolution feature map is input to an expansion module in the mask prediction branch model, and the intermediate convolution feature map is expanded by the expansion module, Obtain multi-channel feature maps, including:

    The intermediate convolution feature map is input to the first convolution layer in the expansion module, and the damage mask feature extraction is performed on the intermediate convolution feature map through the first convolution layer to obtain the first feature Figure;

    Up-sampling the first feature map 1 through the first sampling layer in the expansion module to obtain a first sampling map;

    Input the first sampling image to the second convolutional layer in the expansion module, and perform the damage mask feature extraction on the first sampling image through the second convolutional layer to obtain a second feature image;

    Performing up-sampling processing on the second feature map through the second sampling layer in the expansion module to obtain a second sampling map;

    Input the second sampling image to the third convolutional layer in the expansion module, and perform the damage mask feature extraction on the second sampling image through the third convolutional layer to obtain a third feature image;

    Up-sampling processing is performed on the third feature map through the third sampling layer in the expansion module to obtain a multi-channel feature map.
14. A computer device includes a memory, a processor, and computer readable instructions stored in the memory and capable of running on the processor, wherein the processor further implements the following steps when executing the computer readable instructions :

    Receive car damage detection instructions and obtain car damage images;

    The car damage image is input to the damage detection model trained by the damage detection model training method, the damage feature is extracted from the damage detection model, and the final result output by the damage detection model according to the damage feature is obtained; the final result Including the damage type and the damage area, the final result represents the damage type and the damage area of all the damage positions in the car damage image.
15. One or more readable storage media storing computer readable instructions, where when the computer readable instructions are executed by one or more processors, the one or more processors execute the following steps:

    Acquire a damage sample set; the damage sample set includes damage sample images, and one damage sample image is associated with a damage label group; the damage label group includes at least one damage label type and a mask corresponding to the damage label type Mark the map and at least one rectangular frame area;

    The damage sample image is input into a damage detection model containing the first parameter, and the damage feature in the damage sample image is extracted through the damage detection model and an intermediate convolution feature map is generated; the damage detection model is based on the YOLOV3 model framework Deep convolutional neural network model;

    Input the intermediate convolution feature map into the mask prediction branch model containing the second parameter;

    The damage detection model outputs the training result according to the damage feature, and at the same time obtains the mask result through the mask prediction branch model; the training result includes at least one sample damage type and at least one sample damage rectangular area; the mask The code result is obtained and output according to the damage mask feature extracted from the intermediate convolution feature map, and the mask result includes at least one mask damage type and a mask tensor map corresponding to the mask damage type ；

    Input all the damage label types, all the rectangular frame areas, all the sample damage types and all the sample damage rectangular areas of the damage sample image into the first loss model to obtain the first loss value, and at the same time Input all the damage label types, all the mask annotation maps, all the mask damage types, and all the mask tensor maps of the damage sample image into a second loss model to obtain a second loss value;

    Determine a total loss value according to the first loss value and the second loss value;

    When the total loss value does not reach the preset convergence condition, iteratively update the first parameter of the damage detection model and the second parameter of the mask prediction branch model until the total loss value reaches the preset When the convergence condition is set, the damage detection model after convergence is recorded as the training completed damage detection model.
16. 15. The readable storage medium of claim 15, wherein the determining the total loss value according to the first loss value and the second loss value comprises:

    The first loss value and the second loss value are input into a preset loss model, and the total loss value is calculated by the total loss function in the loss model; the total loss function is:

    L=w ₁ ×X1+w ₂ ×X2

    among them,

    X1 is the first loss value;

    X2 is the second loss value;

    w ₁ is the weight of the first loss value;

    w ₂ is the weight of the second loss value.
17. 15. The readable storage medium of claim 15, wherein, before obtaining the damage sample set, when the computer-readable instructions are executed by one or more processors, the one or more processors are caused to perform the following steps:

    Acquire sample images and public data images; the sample images are images taken with damage locations, and the public data images are randomly selected images in the KITTI data set;

    Fusion processing the sample image and the public data image by using a mixup method to obtain a fused sample image;

    The fusion sample image is determined as the damaged sample image corresponding to the sample image, and the damaged sample image is stored in the blockchain.
18. 15. The readable storage medium according to claim 15, wherein the obtaining the mask result through the mask prediction branch model comprises:

    Input the intermediate convolution feature map to an expansion module in the mask prediction branch model, and perform damage mask feature extraction and expansion processing on the intermediate convolution feature map through the expansion module to obtain a multi-channel feature map;

    The multi-channel feature map is input to the classification module in the mask prediction branch model, and the multi-channel feature map is classified and predicted by the classification module to obtain the mask corresponding to the intermediate convolution feature map forecast result;

    According to the mask prediction result corresponding to the intermediate convolution feature map, the mask result corresponding to the damaged sample image is determined.
19. The readable storage medium according to claim 18, wherein the intermediate convolution feature map is input to an expansion module in the mask prediction branch model, and the intermediate convolution feature map is expanded by the expansion module Processing to obtain a multi-channel feature map, including:

    The intermediate convolution feature map is input to the first convolution layer in the expansion module, and the damage mask feature extraction is performed on the intermediate convolution feature map through the first convolution layer to obtain the first feature Figure;

    Up-sampling the first feature map 1 through the first sampling layer in the expansion module to obtain a first sampling map;

    Input the first sampling image to the second convolutional layer in the expansion module, and perform the damage mask feature extraction on the first sampling image through the second convolutional layer to obtain a second feature image;

    Performing up-sampling processing on the second feature map through the second sampling layer in the expansion module to obtain a second sampling map;

    Input the second sampling image to the third convolutional layer in the expansion module, and perform the damage mask feature extraction on the second sampling image through the third convolutional layer to obtain a third feature image;

    Up-sampling of the third feature map is performed through the third sampling layer in the expansion module to obtain a multi-channel feature map.
20. One or more readable storage media storing computer readable instructions, where when the computer readable instructions are executed by one or more processors, the one or more processors further execute the following steps:

    Receive car damage detection instructions and obtain car damage images;

    The car damage image is input to the damage detection model trained by the damage detection model training method, the damage feature is extracted from the damage detection model, and the final result output by the damage detection model according to the damage feature is obtained; the final result Including the damage type and the damage area, the final result represents the damage type and the damage area of all the damage positions in the car damage image.

Images