WO2023035538A1

WO2023035538A1 - Vehicle damage detection method, device, apparatus and storage medium

Info

Publication number: WO2023035538A1
Application number: PCT/CN2022/072367
Authority: WO
Inventors: 方起明; 刘莉红; 刘玉宇
Original assignee: 平安科技（深圳）有限公司
Priority date: 2021-09-08
Filing date: 2022-01-17
Publication date: 2023-03-16
Also published as: CN113743407A; CN113743407B

Abstract

A vehicle damage detection method, a device, an apparatus and a storage medium, the method comprising: obtaining a standard data set, wherein the standard data set comprises vehicle data for a plurality of different vehicle types with various standard damage label information; acquiring a target image, performing pre-identification on the target image, and for each site pre-identified to be a damage region, marking with a candidate region box, wherein the target image comprises a plurality of vehicle images which have not yet undergone damage labeling; in accordance with the positional relationships between various candidate region boxes, identifying a target damage site corresponding to each candidate region box in the target image; respectively performing aggregation calculations on the candidate region boxes to obtain an aggregate embedded value and an aggregation confidence score for when a candidate region box is aggregated to a target damage site; in accordance with the aggregate embedded value and the aggregation confidence score, merging each of the candidate region boxes corresponding to the same target damage site, thus obtaining prototype characterization information corresponding to the target damage site; using a detection model to respectively perform inter-domain alignment on the prototype characterization information and on the standard damage label information of each vehicle type, and using the standard damage label information having the smallest alignment distance as vehicle damage information corresponding to the prototype characterization information.

Description

Vehicle damage detection method, device, equipment and storage medium

This application claims the priority of the Chinese patent application with the application number 202111058959.7 filed on September 8, 2021, and the title of the invention is "vehicle damage detection method, device, equipment and storage medium", the entire content of which is incorporated by reference incorporated in this application.

technical field

The present application relates to the technical field of artificial intelligence, in particular to a method, device, equipment and storage medium for detecting vehicle damage.

Background technique

In the daily driving of a car, it is inevitable that the car will be damaged due to factors such as weather conditions, road environment and individual differences of drivers. Therefore, it is indispensable to determine the damaged part and degree of damage of the damaged vehicle, which will not only affect the determination of the subsequent vehicle maintenance plan, but also affect the determination of the amount of economic compensation for the parties involved in the accident.

In recent years, with the development of artificial intelligence technology, some institutions have abandoned the damage assessment method relying on manual judgment, and instead adopted methods such as artificial intelligence-based visual inspection to assess the damage of damaged vehicles. However, the inventor realized that methods based on artificial intelligence visual inspection need to obtain in advance a large number of marked vehicle pictures of different models for model training. If the training is used, the damage of the vehicle cannot be accurately identified, resulting in a decrease in the accuracy of vehicle damage detection.

technical problem

The main purpose of this application is to provide a detection method, device, equipment and storage medium for vehicle damage, aiming to solve the problem that in the prior art, the damage of the vehicle cannot be accurately identified without pre-training the vehicle models that need to be damaged, resulting in vehicle damage. A technical problem that reduces the accuracy of damage detection.

technical solution

The present application proposes a method for detecting vehicle damage, the method comprising: acquiring a standard data set, wherein the standard data set includes vehicle data of several different vehicle types with different standard damage label information; acquiring a target image, Perform pre-identification on the target image, and mark a region candidate frame for each part pre-recognized as a damaged region, wherein the target image includes several vehicle images that have not been marked with damage; according to different regions, the candidate frame The positional relationship between each region candidate frame is identified in the target image corresponding to the target lesion; the region candidate frame is aggregated and calculated respectively, and the region candidate frame is aggregated to the target lesion The aggregated embedding value and the aggregated confidence degree; according to the aggregated embedded value and the aggregated confidence degree, the region candidate frames corresponding to the same target lesion are merged to obtain the target lesion corresponding to Prototype characterization information: through the detection model, the prototype characterization information and the standard damage labeling information of each vehicle type are respectively inter-domain aligned, and the standard damage labeling information with the smallest alignment distance is used as the corresponding prototype characterization information Vehicle damage information.

The present application also proposes a vehicle damage detection device, including: a data set acquisition module, used to obtain a standard data set, wherein the standard data set includes several different types of vehicles with different standard damage labeling information Data; an image acquisition module, configured to acquire a target image, perform pre-identification on the target image, and mark a region candidate frame for each part pre-recognized as a damaged region, wherein the target image includes several unmarked damage The vehicle image; the target damage part recognition module, used to identify the target damage part corresponding to each of the region candidate frames in the target image according to the positional relationship between the different regions candidate frames; the aggregation calculation module, It is used to perform aggregation calculation on the region candidate frames respectively, to obtain the aggregation embedding value and the aggregation confidence when the region candidate frames are aggregated to the target damage site; Said aggregation confidence degree, merge each of the region candidate frames corresponding to the same target damage part, and obtain the prototype representation information corresponding to the target damage part; the domain alignment module is used to use the detection model to respectively combine the The prototype characterization information is inter-domain aligned with the standard damage labeling information of each vehicle type, and the standard damage labeling information with the smallest alignment distance is used as the vehicle damage information corresponding to the prototype characterization information.

The present application also proposes a computer device, including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned method for detecting vehicle damage are realized, including: obtaining the standard A data set, wherein the standard data set includes vehicle data of several different vehicle types with different standard damage labeling information; the target image is acquired, the target image is pre-identified, and each pre-identified as a damaged area The part mark region candidate frame, wherein, the target image includes several vehicle images without damage labeling; according to the positional relationship between the different region candidate frames, identify each region candidate frame in the target The corresponding target damage part in the image; perform aggregation calculation on the region candidate frames respectively, and obtain the aggregation embedding value and aggregation confidence when the region candidate frame is aggregated to the target damage part; according to the aggregation embedding value and the Said aggregation confidence, merge each of the region candidate frames corresponding to the same target damage part, and obtain the prototype representation information corresponding to the target damage part; through the detection model, respectively combine the prototype representation information with each The standard damage labeling information of the vehicle type is aligned between domains, and the standard damage labeling information with the smallest alignment distance is used as the vehicle damage information corresponding to the prototype characterization information.

The present application also proposes a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned vehicle damage detection method are implemented, including: acquiring a standard data set, wherein, The standard data set includes vehicle data of several different vehicle types with different standard damage labeling information; the target image is acquired, the target image is pre-recognized, and each part pre-recognized as a damaged region is marked as a region candidate frame, wherein the target image includes several vehicle images that have not been marked with damage; according to the positional relationship between the different candidate regions, identify the target corresponding to each candidate region frame in the target image Lesion site; perform aggregation calculation on the region candidate frames respectively, to obtain the aggregated embedding value and the aggregated confidence degree when the region candidate frame is aggregated to the target lesion site; according to the aggregated embedded value and the aggregated confidence degree, merging each of the region candidate frames corresponding to the same target damage part to obtain the prototype representation information corresponding to the target damage part; through the detection model, respectively combining the prototype representation information with each of the vehicle types The standard damage label information is aligned between domains, and the standard damage label information with the smallest alignment distance is used as the vehicle damage information corresponding to the prototype characterization information.

Beneficial effect

The vehicle damage detection method, device, equipment, and storage medium of the present application obtain a vehicle damage image that has not been marked with damage as a target image, and generate several area candidate frames for the target image, thereby realizing automatic detection of possible vehicle damage areas. Recognition; by identifying the positional relationship of each region candidate frame, it is determined whether different region candidate frames correspond to the same damaged target damage site, which improves the integrity of target damage site identification; through aggregation calculation of the region candidate frames, And obtain the prototype representation information corresponding to different target damage parts, thereby enhancing the robustness of region recognition and avoiding the problem of incorrect recognition of damaged regions caused by the labeling errors of individual region candidate frames; The vehicle damage information corresponding to the prototype representation information is output, which improves the accuracy of vehicle damage detection.

Description of drawings

FIG. 1 is a schematic flow chart of a vehicle damage detection method according to an embodiment of the present application;

2 is a schematic flow diagram of a vehicle damage detection method according to a specific embodiment of the present application;

Fig. 3 is a schematic structural block diagram of a vehicle damage detection device according to an embodiment of the present application;

FIG. 4 is a schematic block diagram of a computer device according to an embodiment of the present application.

The realization, functional features and advantages of the present application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

BEST MODE FOR CARRYING OUT THE INVENTION

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

Referring to Fig. 1, an embodiment of the present application provides a method for detecting vehicle damage in order to achieve the above-mentioned purpose of the invention, the method comprising:

S1: Obtain a standard data set, wherein the standard data set includes vehicle data of several different vehicle types with different standard damage label information;

S2: Acquire a target image, perform pre-recognition on the target image, and mark a region candidate frame for each part pre-recognized as a damaged region, wherein the target image includes several vehicle images that have not been marked with damage;

S3: Identify the target lesion corresponding to each of the candidate region frames in the target image according to the positional relationship between the different candidate region frames;

S4: Carry out aggregation calculation on the region candidate frames respectively, to obtain the aggregation embedding value and the aggregation confidence when the region candidate frames are aggregated to the target lesion site;

S5: According to the aggregated embedding value and the aggregated confidence, merge the region candidate frames corresponding to the same target lesion to obtain prototype representation information corresponding to the target lesion;

S6: Align the prototype characterization information with the standard damage labeling information of each vehicle type through the detection model, and use the standard damage labeling information with the smallest alignment distance as the vehicle damage corresponding to the prototype characterization information information.

In this embodiment, by acquiring a vehicle damage image without damage labeling as the target image, several region candidate frames are generated for the target image, thereby realizing the automatic identification of the possible damage region of the vehicle; Recognition, so as to determine whether different region candidate frames correspond to the same damage target damage part, which improves the integrity of target damage part recognition; through aggregation calculation of region candidate frames, and obtaining prototype representation information corresponding to different target damage parts, In this way, the robustness of area recognition is enhanced, and the problem of incorrect identification of damaged areas caused by labeling errors of individual area candidate frames is avoided; by inter-domain alignment of prototype representation information and standard damage label information, the vehicle damage information corresponding to prototype representation information is output, Therefore, the accuracy of vehicle damage detection is improved.

For step S1, this embodiment is usually applied in the field of vehicle damage detection and recording. In order to evaluate the vehicle damage, it is often necessary to collect and identify vehicle images; the embodiment of this application can acquire and process vehicle images based on artificial intelligence technology . Among them, artificial intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. . Specifically, the above-mentioned vehicle data with standard damage annotation information is usually a business scenario data set with a large number of annotations, such as vehicle data of different vehicle types with damage annotations, so that the above-mentioned different vehicle types with damage annotations The vehicle data serves as the source domain to facilitate domain alignment with the target domain formed from subsequent target images.

For step S2, the above-mentioned vehicle damage image without damage labeling may be a vehicle data set without labeling of difficult-to-recognize vehicle types. Specifically, in daily vehicle damage detection, because some vehicles are relatively old or damaged, it is difficult to obtain training data of their type. Therefore, the unlabeled vehicle data set that is difficult to distinguish the vehicle type can be used as the target domain, and the source domain formed by the vehicle data of the known vehicle type with the damage label is used for domain alignment, that is, the known vehicle type with the damage label Damage prediction is performed on unlabeled vehicle data of unknown vehicle types.

After the target image is acquired, the pre-trained image recognition model can be used to pre-recognize the damage of the target image of unknown vehicle type. Specifically, the target image can be image recognized, and the above-mentioned vehicles can be identified according to parameters such as color, shape, and texture. Parts in the damaged image that may be damaged regions are selected, and each part identified as a damaged region is framed to obtain several region candidate frames.

For step S3, since in the actual damage detection, the area candidate frames often have deviations, and the complete damage area cannot be completely and accurately framed, resulting in insufficient target instance information contained in the framed part. Therefore, this embodiment adopts Construct a graph of the positional relationship between the region candidate frames, and construct a graph (graph) data structure according to the positional relationship between each region candidate frame, which contains a collection of vertices connected by a series of edges , where a vertex is a region candidate frame, and the length of the edge connected between the vertices is the distance between two region candidate frames; this embodiment uses distance screening to distinguish whether different region candidate frames correspond to the same damage Part, specifically, delete the edge whose length is greater than the preset edge length threshold, that is, cancel the connection of the region candidate boxes at both ends of the edge, and finally retain the vertices corresponding to the connected edges, that is, the region candidate boxes with a closer positional relationship, At this time, the region candidate frames that are still connected to each other can be divided into regions to obtain several mutually independent regions, and each region contains several region candidate frames that are connected, and an independent region can be regarded as one of the above The target damage part, a target damage part is an image corresponding to an actual damage part on the target image.

For step S4, after corresponding the region candidate frames, although the integrity of the identified damage parts can be improved, due to the recognition deviation, some region candidate frames are often distributed around the damage parts, which leads to a single candidate frame The inaccuracy of characterizing an object. In order to achieve more accurate feature representation at the instance level, the candidate boxes belonging to the same target lesion should be aggregated into a complete marquee, so that the images in the complete marquee can form a relatively complete image of the lesion. In an embodiment, the aggregated embedding value represents the degree of influence of the region candidate frame on a certain target lesion, and the aggregate confidence represents the possibility that the region candidate frame belongs to the target lesion.

For step S5, by using the aggregated confidence as the weight of the aggregated embedding value, the weighted calculation of several area candidate frames corresponding to a target damage site is performed based on the above weights, and the weighted calculation result is the combined result of each area candidate frame, thus completing The feature representation of the region candidate frame is aggregated at the instance level. However, the image of the target damage part corresponding to the aggregation result often includes visual modality information. In order to facilitate the quantitative calculation of subsequent inter-domain alignment, the modes reflected by different target damage parts State information should be integrated into prototype representation information, so that it can be used as a substitute for each target damage site in subsequent inter-domain alignment. In this embodiment, prototype representation information is a data-based parameter of human visual features, used to represent Feature information of the image part corresponding to the damaged part.

For step S6, the source domain and the target domain are aligned through the preset vehicle damage detection model, wherein the source domain is the vehicle data of the known vehicle type with standard damage label information, and the target domain adopts the above step S5 The obtained prototype representation information is replaced, and after several source domains and several target domains are aligned, several interrelated source domain-target domain pairs can be obtained, among which, in the same source domain-target domain pair, the target domain The damage cause of the domain is the same as that of the source domain, and the smaller the alignment distance between the source domain and the target domain, the closer the two domains are, that is, the closer the vehicle types are, and vice versa. Therefore, this embodiment will align the source domain with the smallest distance The domain-target domain pair is the closest detection result; at this time, since the source domain is the vehicle data with standard damage label information, the standard damage label can be used as the target corresponding to the closest target domain to the source domain The cause of the damage of the damaged part, thus realizing the damage detection of foreign damaged vehicles that are difficult to obtain training samples.

In one embodiment, the identifying the target lesion S3 corresponding to each of the candidate area frames in the target image according to the positional relationship between the different candidate area frames includes:

S31: In the same target image, select two different candidate regions as the first recognition frame and the second recognition frame;

S32: Calculate an intersection ratio between the first recognition frame and the second recognition frame according to the positional relationship between the first recognition frame and the second recognition frame;

S33: If the intersection-over-union ratio is greater than a preset ratio threshold, determine that the target lesion corresponding to the first recognition frame and the second recognition frame are the same;

S34: Select two different candidate regions as the first recognition frame and the second recognition frame again, and perform the calculation of the intersection ratio and the determination of the ratio threshold until the target image Each of the region candidate frames completes the intersection-over-union ratio calculation and the ratio threshold determination with the rest of the region candidate frames.

In this embodiment, it is further determined whether different region candidate frames correspond to the same target lesion by way of cross-merge ratio, which improves the accuracy of identifying the target lesion.

For step S31, in the actual vehicle damage, there may be multiple smaller damages. At this time, if only the distance between two area candidate frames is used to determine whether they belong to the same template unit, it will often result in multiple smaller damages. Small independent lesions are identified as the same piece of lesions. Therefore, in this embodiment, it is further determined whether different region candidate frames correspond to the same damage site by way of cross-over-merge ratio.

For step S32, after two identification frames are selected, the intersection area and union area of the two are calculated, and the ratio of the intersection area to the union area is taken as the above-mentioned intersection-union ratio.

For step S33, it can be understood that since the intersection ratio is closer to 1, the probability of overlap between the two is greater. Therefore, when the intersection ratio is greater than the preset ratio threshold, it is identified that there is a larger overlap area between the two recognition frames , so it can be determined that the target lesion corresponding to the first recognition frame and the second recognition frame are the same.

For step S34, after the calculation of the current two recognition frames is completed, the rest of the region candidate frames are selected again to perform the above intersection calculation and judgment, until a judgment is completed between any two region candidate frames.

In one embodiment, the calculation method S4 of the aggregated embedded value includes:

S41: Construct an adjacency matrix between the region candidate frames through the intersection-over-union ratio;

S42: Obtain the feature embedding value of the region candidate frame, and calculate the aggregate embedding value corresponding to the feature embedding value by the following formula:

In the formula,

is the aggregation embedding value, F is the feature embedding value, A is the adjacency matrix, and D is the diagonal matrix of the adjacency matrix.

In this embodiment, an adjacency matrix is used to calculate aggregated embedding values between candidate frames in different regions, so that more accurate damage instance information can be expressed.

For step S41, the above-mentioned adjacency matrix can be obtained according to the above-mentioned graph (graph) data structure. The adjacency matrix usually includes a two-dimensional array, and the one-dimensional array in the two-dimensional array stores all vertex data in the graph (graph) data structure. The dimension array stores the data of the relationship (edge) between vertices, so as to obtain the distance between the quantified region candidate frames, and then determine the degree of aggregation between the region candidate frames.

For step S42, the feature embedding value of the above-mentioned region candidate frame can be calculated by a feature selection algorithm (Embedded), which can obtain the feature combination of the region candidate frame, find the optimal feature combination in the feature combination and return the feature embedding value As a result, the vector features used to describe different region candidate frames, that is, the above-mentioned feature embedding values can reduce the data dimension to a fixed-size vector feature representation for easy processing and calculation; Extract the feature embedding value of the image to facilitate the subsequent aggregation calculation.

In one embodiment, the calculation method S4 of the aggregation confidence includes:

S43: Obtain the classification confidence of the region candidate frame, and calculate the aggregation confidence corresponding to the classification confidence by the following formula:

In the formula,

is the aggregation confidence, P is the classification confidence, A is the adjacency matrix, and D is the diagonal matrix of the adjacency matrix.

In this embodiment, an adjacency matrix is used to calculate aggregated embedding values between candidate frames in different regions, so that more accurate damage instance information can be further expressed.

For step S43 , classification confidence levels are generated for the images framed by each candidate area frame, and the above classification confidence levels represent the possibility that a candidate area frame belongs to a predetermined target lesion. Specifically, the image classification model can be preset, and the image classification model can be used to determine whether the image in the region candidate frame is corresponding to a certain target damage part, and calculate the possibility that the region candidate frame is corresponding to the target damage part , that is, the above-mentioned classification confidence. The above-mentioned aggregation confidence is: under the spatial correlation provided by the adjacency matrix, the possibility that the region candidate frame belongs to the target lesion.

In one embodiment, referring to FIG. 2, the aggregated embedding value and the aggregated confidence are combined for each of the region candidate frames corresponding to the same target lesion to obtain the prototype corresponding to the target lesion. Characterization information S5, including:

S51: Using the aggregated confidence as the merging weight of the region candidate frame;

S52: Perform weighted average calculation on the aggregated embedding value according to the combination weight to obtain the prototype representation information.

In this embodiment, the weighted calculation is performed according to the combination weight to obtain the weighted prototype representation information, so as to obtain the prototype representation information that is more prominent for the region candidate frames with higher confidence.

For step S52, in order to highlight the modal information corresponding to the region candidate frames that are more important to a specific category, the application uses the aggregation confidence of each region candidate frame as the merging weight for merging, so that different region candidate frames are combined according to the aggregation confidence Merging is carried out to obtain the prototype representation information of the above-mentioned target damage site.

In one embodiment, said passing the detection model, performing inter-domain alignment S6 on the prototype representation information and the standard damage labeling information of each vehicle type, including:

S61: Using a built-in detection model with inter-class loss constraints, perform feature distribution alignment on the prototype representation information and the standard damage label information.

In this embodiment, feature distribution alignment is performed through inter-class loss constraints, so as to obtain a domain alignment result considering class imbalance.

For step S61 , the core idea is to minimize the intra-class loss (denoted as L _intra ) calculation by constraining the inter-class loss, so as to reduce the distance between two prototype representation information. In addition, the distance between different prototype representations is constrained by another inter-class loss (denoted as L _inter ). And considering the existence of category imbalance, the influence of different categories can be adjusted by setting the parameters of the loss between the two categories.

In one embodiment, the marking method S2 of the region candidate frame includes:

S21: Based on the Faster R-CNN target detection framework, perform feature extraction of foreground and background features on the region candidate network of the target image to generate region candidate frames.

In this embodiment, through the Faster R-CNN target detection framework, the feature extraction of foreground and background features is performed on the region candidate network of the target image, thereby generating accurate region candidate frames.

For step S21, an initial vehicle damage detection model based on Convolutional Neural Networks (CNN) can be established, and a Graph-induced Prototype Alignment (Graph-induced Prototype Alignment) framework can be used to unsupervised the initial vehicle damage detection model Adaptive learning to improve the accuracy of the initial vehicle damage detection model on target domain data.

Referring to Fig. 3, the present application also proposes a detection device for vehicle damage, including:

A data set acquisition module 100, configured to acquire a standard data set, wherein the standard data set includes vehicle data of several different vehicle types with different standard damage labeling information;

The image acquisition module 200 is configured to acquire a target image, perform pre-recognition on the target image, and mark a region candidate frame for each part pre-recognized as a damaged region, wherein the target image includes several parts that have not been marked with damage. vehicle image;

A target lesion identification module 300, configured to identify the target lesion corresponding to each of the candidate area frames in the target image according to the positional relationship between the different candidate area frames;

Aggregation calculation module 400, configured to perform aggregation calculation on the region candidate frames respectively, to obtain the aggregation embedding value and aggregation confidence when the region candidate frames are aggregated to the target lesion site;

The merge calculation module 500 is configured to merge the region candidate frames corresponding to the same target lesion according to the aggregated embedding value and the aggregate confidence to obtain prototype representation information corresponding to the target lesion ;

The domain alignment module 600 is configured to perform inter-domain alignment on the prototype representation information and the standard damage annotation information of each vehicle type through the detection model, and use the standard damage annotation information with the smallest alignment distance as the prototype representation The vehicle damage information corresponding to the information.

In one embodiment, the target lesion identification module 300 includes:

A candidate frame distinguishing unit, configured to select two different region candidate frames in the same target image as the first recognition frame and the second recognition frame;

an intersection ratio calculation unit, configured to calculate an intersection ratio between the first recognition frame and the second recognition frame according to the positional relationship between the first recognition frame and the second recognition frame;

A damage determination unit, configured to determine that the target damage site corresponding to the first recognition frame and the second recognition frame are the same if the intersection-over-union ratio is greater than a preset ratio threshold;

a threshold determination unit, configured to select two different candidate regions as the first identification frame and the second identification frame, and perform the intersection-over-union ratio calculation and the ratio threshold determination until the Each of the region candidate frames in the target image completes the intersection-over-union ratio calculation and the ratio threshold determination with the rest of the region candidate frames.

In one embodiment, the aggregation calculation module 400 includes:

a matrix component unit, configured to construct an adjacency matrix between the region candidate frames through the intersection-over-union ratio;

An embedding value calculation unit, configured to obtain the feature embedding value of the region candidate frame, and calculate the aggregate embedding value corresponding to the feature embedding value by the following formula:

In the formula,

In one embodiment, the aggregation calculation module 400 includes:

An aggregation confidence calculation unit, configured to obtain the classification confidence of the region candidate frame, and calculate the aggregation confidence corresponding to the classification confidence by the following formula:

In the formula,

In one embodiment, the combined calculation module 500 includes:

a merging weight calculation unit, configured to use the aggregation confidence as the merging weight of the region candidate frame;

The characterization information calculation unit is configured to perform weighted average calculation on the aggregated embedded values according to the merging weight to obtain the prototype characterization information.

In one embodiment, the domain alignment module 600 includes:

The feature alignment unit is configured to perform feature distribution alignment on the prototype representation information and the standard damage label information through a built-in detection model with inter-class loss constraints.

In one embodiment, the image acquisition module 200 is also used for:

The feature extraction unit is used to perform feature extraction of foreground and background features on the region candidate network of the target image based on the Faster R-CNN target detection framework to generate a region candidate frame.

Referring to FIG. 4 , an embodiment of the present application also provides a computer device, which may be a server, and its internal structure may be as shown in FIG. 4 . The computer device includes a processor, memory, network interface and database connected by a system bus. Among them, the processor designed by the computer is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs and databases. The memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer equipment is used to store data such as detection methods of vehicle damage. The network interface of the computer device is used to communicate with the external terminal through the network connection. When the computer program is executed by the processor, a method for detecting vehicle damage is realized. The vehicle damage detection method includes: obtaining a standard data set, wherein the standard data set includes several vehicle data with standard damage label information; obtaining a target image, and generating several region candidate frames for the target image , wherein, the target image includes several vehicle damage images that have not been marked with damage; according to the positional relationship between the different candidate regions, identify the target corresponding to each candidate region frame in the target image Lesion site; performing aggregation calculations on the region candidate frames respectively to obtain the aggregation embedding value and the aggregation confidence degree corresponding to the region candidate frame; according to the aggregation embedding value and the aggregation confidence degree, for the same target damage The region candidate frames corresponding to the parts are merged to obtain the prototype representation information of the cluster corresponding to the target damage part; through the detection model, the prototype representation information and the standard damage label information are inter-domain aligned, and the output The vehicle damage information corresponding to the prototype characterization information.

An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, a vehicle damage detection method is implemented, including the steps of: acquiring a standard data set, wherein the The standard data set includes vehicle data of several different vehicle types with different standard damage labeling information; the target image is acquired, the target image is pre-recognized, and each part pre-recognized as a damaged area is marked with a region candidate frame, Wherein, the target image includes several vehicle images that have not been marked with damage; according to the positional relationship between the different candidate regions, identify the target damage part corresponding to each candidate region frame in the target image ; Carry out aggregation calculations on the region candidate frames respectively, to obtain the aggregation embedding value and the aggregation confidence when the region candidate frames are aggregated to the target damage site; according to the aggregation embedding value and the aggregation confidence, for the same Merge each of the region candidate frames corresponding to a target damage part to obtain prototype representation information corresponding to the target damage part; through the detection model, respectively combine the prototype representation information with the standard damage of each vehicle type The annotation information is aligned between domains, and the standard damage annotation information with the smallest alignment distance is used as the vehicle damage information corresponding to the prototype representation information.

In the vehicle damage detection method performed above, in this embodiment, a vehicle damage image without damage labeling is obtained as a target image, and several area candidate frames are generated for the target image, thereby realizing automatic identification of possible damage areas of the vehicle; The positional relationship of each region candidate frame is identified to determine whether different region candidate frames correspond to the same damaged target damage part, which improves the integrity of target damage part identification; through aggregation calculation of region candidate frames, different targets are obtained Prototype representation information corresponding to the damaged parts, thereby enhancing the robustness of region recognition and avoiding the problem of incorrect recognition of damaged regions caused by labeling errors in individual region candidate frames; through inter-domain alignment of prototype representation information and standard damage labeling information, the output The vehicle damage information corresponding to the prototype representation information improves the accuracy of vehicle damage detection.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented through computer programs to instruct related hardware, and the computer programs can be stored in a non-volatile computer-readable memory In the medium, when the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Wherein, any references to memory, storage, database or other media provided in the present application and used in the embodiments may include non-volatile and/or volatile memory. Nonvolatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (SSRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, apparatus, article or method comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional same elements in the process, apparatus, article or method comprising the element.

The above are only preferred embodiments of the application, and are not intended to limit the patent scope of the application. Any equivalent structure or equivalent process conversion made by using the specification and drawings of the application, or directly or indirectly used in other related All technical fields are equally included in the patent protection scope of the present application.

Claims

A method for detecting vehicle damage, wherein the method includes:

Obtaining a standard data set, wherein the standard data set includes vehicle data of several different vehicle types with different standard damage labeling information;

Acquiring a target image, pre-identifying the target image, and marking a region candidate frame for each part pre-recognized as a damaged region, wherein the target image includes several vehicle images that have not been marked with damage;

Identifying a target lesion corresponding to each of the region candidate frames in the target image according to the positional relationship between the different region candidate frames;

Carrying out aggregation calculations on the region candidate frames respectively, to obtain an aggregation embedding value and an aggregation confidence when the region candidate frames are aggregated to the target lesion site;

According to the aggregated embedding value and the aggregated confidence, merge each of the region candidate frames corresponding to the same target lesion to obtain prototype representation information corresponding to the target lesion;

Through the detection model, the prototype characterization information is inter-domain aligned with the standard damage labeling information of each vehicle type, and the standard damage labeling information with the smallest alignment distance is used as the vehicle damage information corresponding to the prototype characterization information.
The vehicle damage detection method according to claim 1, wherein, according to the positional relationship between different said region candidate frames, identify the target damage part corresponding to each of said region candidate frames in said target image ,include:

In the same target image, select two different candidate regions as the first recognition frame and the second recognition frame;

calculating an intersection ratio between the first recognition frame and the second recognition frame according to the positional relationship between the first recognition frame and the second recognition frame;

If the intersection-over-union ratio is greater than a preset ratio threshold, it is determined that the target lesion corresponding to the first identification frame and the second identification frame are the same;

Selecting two different candidate frames of the region as the first recognition frame and the second recognition frame, and performing the calculation of the intersection ratio and the determination of the ratio threshold until each of the target images The region candidate frames and the other region candidate frames complete the calculation of the intersection ratio and the determination of the ratio threshold.
The detection method for vehicle damage according to claim 2, wherein the calculation method of the aggregated embedded value comprises:

Constructing an adjacency matrix between the region candidate frames through the intersection-over-union ratio;

Obtain the feature embedding value of the region candidate frame, and calculate the aggregate embedding value corresponding to the feature embedding value by the following formula:

In the formula,
is the aggregation embedding value, F is the feature embedding value, A is the adjacency matrix, and D is the diagonal matrix of the adjacency matrix.
The detection method of vehicle damage according to claim 3, wherein, the calculation method of the aggregation confidence includes:

Obtain the classification confidence of the region candidate frame, and calculate the aggregation confidence corresponding to the classification confidence by the following formula:

In the formula,
is the aggregation confidence, P is the classification confidence, A is the adjacency matrix, and D is the diagonal matrix of the adjacency matrix.
The vehicle damage detection method according to claim 1, wherein, according to the aggregated embedding value and the aggregated confidence, the region candidate frames corresponding to the same target damage site are combined to obtain The prototype representation information corresponding to the target damage site includes:

Using the aggregation confidence as the merging weight of the region candidate frame;

Perform weighted average calculation on the aggregated embedding value according to the combination weight to obtain the prototype representation information.
The method for detecting vehicle damage according to claim 1, wherein, through the detection model, respectively performing inter-domain alignment of the prototype representation information and the standard damage labeling information of each of the vehicle types, comprising:

Through the built-in detection model with inter-class loss constraints, the feature distribution alignment is performed on the prototype representation information and the standard damage label information.
The detection method of vehicle damage according to claim 1, wherein the marking method of the region candidate frame comprises:

Based on the Faster R-CNN target detection framework, the feature extraction of foreground and background features is performed on the region candidate network of the target image to generate the region candidate frame.
A vehicle damage detection device, including:

A data set acquisition module, configured to acquire a standard data set, wherein the standard data set includes vehicle data of several different vehicle types with different standard damage label information;

An image acquisition module, configured to acquire a target image, perform pre-identification on the target image, and mark an area candidate frame for each part pre-identified as a damaged area, wherein the target image includes several vehicles that have not been marked with damage image;

A target lesion identification module, configured to identify the target lesion corresponding to each of the candidate area frames in the target image according to the positional relationship between the different candidate area frames;

An aggregation calculation module, configured to perform aggregation calculation on the region candidate frames respectively, to obtain the aggregation embedding value and the aggregation confidence when the region candidate frames are aggregated to the target lesion site;

A merge calculation module, configured to merge each of the region candidate frames corresponding to the same target lesion according to the aggregate embedding value and the aggregate confidence, to obtain prototype representation information corresponding to the target lesion;

A domain alignment module, configured to perform inter-domain alignment on the prototype representation information and the standard damage annotation information of each vehicle type through the detection model, and use the standard damage annotation information with the smallest alignment distance as the prototype representation information Corresponding vehicle damage information.
A computer device, comprising a memory and a processor, the memory stores a computer program, wherein, when the processor executes the computer program, the steps of a method for detecting vehicle damage are realized;

Wherein, the detection method of described vehicle damage comprises:

Obtaining a standard data set, wherein the standard data set includes vehicle data of several different vehicle types with different standard damage labeling information;

Acquiring a target image, pre-identifying the target image, and marking a region candidate frame for each part pre-recognized as a damaged region, wherein the target image includes several vehicle images that have not been marked with damage;

Identifying a target lesion corresponding to each of the region candidate frames in the target image according to the positional relationship between the different region candidate frames;

Carrying out aggregation calculations on the region candidate frames respectively, to obtain an aggregation embedding value and an aggregation confidence when the region candidate frames are aggregated to the target lesion site;

According to the aggregated embedding value and the aggregated confidence, merge each of the region candidate frames corresponding to the same target lesion to obtain prototype representation information corresponding to the target lesion;

Through the detection model, the prototype characterization information is inter-domain aligned with the standard damage labeling information of each vehicle type, and the standard damage labeling information with the smallest alignment distance is used as the vehicle damage information corresponding to the prototype characterization information.
The computer device according to claim 9, wherein the identifying the target lesion corresponding to each of the candidate region frames in the target image according to the positional relationship between the different candidate region frames includes:

In the same target image, select two different candidate regions as the first recognition frame and the second recognition frame;

calculating an intersection ratio between the first recognition frame and the second recognition frame according to the positional relationship between the first recognition frame and the second recognition frame;

If the intersection-over-union ratio is greater than a preset ratio threshold, it is determined that the target lesion corresponding to the first identification frame and the second identification frame are the same;

Selecting two different candidate frames of the region as the first recognition frame and the second recognition frame, and performing the calculation of the intersection ratio and the determination of the ratio threshold until each of the target images The region candidate frames and the other region candidate frames complete the calculation of the intersection ratio and the determination of the ratio threshold.
The computer device according to claim 10, wherein the calculation method of the aggregated embedded value comprises:

Constructing an adjacency matrix between the region candidate frames through the intersection-over-union ratio;

Obtain the feature embedding value of the region candidate frame, and calculate the aggregate embedding value corresponding to the feature embedding value by the following formula:

In the formula,
is the aggregation embedding value, F is the feature embedding value, A is the adjacency matrix, and D is the diagonal matrix of the adjacency matrix.
The computer device according to claim 11, wherein the calculation method of the aggregation confidence degree comprises:

Obtain the classification confidence of the region candidate frame, and calculate the aggregation confidence corresponding to the classification confidence by the following formula:

In the formula,
is the aggregation confidence, P is the classification confidence, A is the adjacency matrix, and D is the diagonal matrix of the adjacency matrix.
The computer device according to claim 9, wherein, according to the aggregated embedding value and the aggregated confidence, the region candidate frames corresponding to the same target lesion are combined to obtain the target Prototype representation information corresponding to the damage site, including:

Using the aggregation confidence as the merging weight of the region candidate frame;

Perform weighted average calculation on the aggregated embedding value according to the combination weight to obtain the prototype representation information.
The computer device according to claim 9, wherein said passing through the detection model, respectively performing inter-domain alignment on said prototype representation information and standard damage labeling information of each said vehicle type, comprises:

Through the built-in detection model with inter-class loss constraints, the feature distribution alignment is performed on the prototype representation information and the standard damage label information.
The computer device according to claim 9, wherein the marking method of the region candidate frame comprises:

Based on the Faster R-CNN target detection framework, the feature extraction of foreground and background features is performed on the region candidate network of the target image to generate the region candidate frame.
A computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, the steps of a method for detecting vehicle damage are realized;

Wherein, the detection method of described vehicle damage comprises:

Obtaining a standard data set, wherein the standard data set includes vehicle data of several different vehicle types with different standard damage labeling information;

Acquiring a target image, pre-identifying the target image, and marking a region candidate frame for each part pre-recognized as a damaged region, wherein the target image includes several vehicle images that have not been marked with damage;

Identifying a target lesion corresponding to each of the region candidate frames in the target image according to the positional relationship between the different region candidate frames;

Carrying out aggregation calculations on the region candidate frames respectively, to obtain an aggregation embedding value and an aggregation confidence when the region candidate frames are aggregated to the target lesion site;

According to the aggregated embedding value and the aggregated confidence, merge each of the region candidate frames corresponding to the same target lesion to obtain prototype representation information corresponding to the target lesion;

Through the detection model, the prototype characterization information is inter-domain aligned with the standard damage labeling information of each vehicle type, and the standard damage labeling information with the smallest alignment distance is used as the vehicle damage information corresponding to the prototype characterization information.
The vehicle damage detection method according to claim 16, wherein, according to the positional relationship between the different candidate region frames, the target damage part corresponding to each candidate region frame in the target image is identified ,include:

In the same target image, select two different candidate regions as the first recognition frame and the second recognition frame;

calculating an intersection ratio between the first recognition frame and the second recognition frame according to the positional relationship between the first recognition frame and the second recognition frame;

If the intersection-over-union ratio is greater than a preset ratio threshold, it is determined that the target lesion corresponding to the first identification frame and the second identification frame are the same;

Selecting two different candidate frames of the region as the first recognition frame and the second recognition frame, and performing the calculation of the intersection ratio and the determination of the ratio threshold until each of the target images The region candidate frames and the other region candidate frames complete the calculation of the intersection ratio and the determination of the ratio threshold.
The vehicle damage detection method according to claim 17, wherein the calculation method of the aggregated embedded value comprises:

Constructing an adjacency matrix between the region candidate frames through the intersection-over-union ratio;

Obtain the feature embedding value of the region candidate frame, and calculate the aggregate embedding value corresponding to the feature embedding value by the following formula:

In the formula,
is the aggregation embedding value, F is the feature embedding value, A is the adjacency matrix, and D is the diagonal matrix of the adjacency matrix.
The detection method for vehicle damage according to claim 18, wherein, the calculation method of the aggregation confidence includes:

Obtain the classification confidence of the region candidate frame, and calculate the aggregation confidence corresponding to the classification confidence by the following formula:

In the formula,
is the aggregation confidence, P is the classification confidence, A is the adjacency matrix, and D is the diagonal matrix of the adjacency matrix.
The vehicle damage detection method according to claim 16, wherein, according to the aggregated embedded value and the aggregated confidence, the region candidate frames corresponding to the same target damage site are combined to obtain The prototype representation information corresponding to the target damage site includes:

Using the aggregation confidence as the merging weight of the region candidate frame;

Perform weighted average calculation on the aggregated embedding value according to the combination weight to obtain the prototype representation information.
The method for detecting vehicle damage according to claim 16, wherein, through the detection model, the inter-domain alignment of the prototype representation information and the standard damage labeling information of each of the vehicle types is performed, comprising:

Through the built-in detection model with inter-class loss constraints, the feature distribution alignment is performed on the prototype representation information and the standard damage label information.
The detection method of vehicle damage according to claim 16, wherein, the marking method of the region candidate frame comprises:

Based on the Faster R-CNN target detection framework, the feature extraction of foreground and background features is performed on the region candidate network of the target image to generate the region candidate frame.