CN109614935B

CN109614935B - Vehicle damage assessment method and device, storage medium and electronic equipment

Info

Publication number: CN109614935B
Application number: CN201811518123.9A
Authority: CN
Inventors: 马文伟; 刘设伟; 王强
Original assignee: Taikang Insurance Group Co Ltd; Taikang Online Property Insurance Co Ltd
Current assignee: Taikang Insurance Group Co Ltd; Taikang Online Property Insurance Co Ltd
Priority date: 2018-12-12
Filing date: 2018-12-12
Publication date: 2021-07-06
Anticipated expiration: 2038-12-12
Also published as: CN109614935A

Abstract

The invention provides a vehicle damage assessment method, which comprises the following steps: when a damage assessment platform receives a damage assessment request, determining a damaged vehicle image sequence corresponding to the damage assessment request; constructing a damaged vehicle three-dimensional model corresponding to the damaged vehicle according to the damaged vehicle image sequence; comparing the damaged vehicle three-dimensional model with a standard vehicle three-dimensional model corresponding to the type of the damaged vehicle, and determining deformation information of the damaged vehicle; and inputting the deformation information into a pre-established vehicle damage calculation model, and determining the damage grade of the damaged vehicle after the deformation information is processed by the vehicle damage calculation model. By applying the vehicle damage assessment method provided by the invention, the deformation information of the damaged vehicle can be rapidly calculated, so that the damage level of the damaged vehicle is determined, a large amount of human resources and time cost are saved, more services can be automatically processed in unit time by applying a vehicle damage calculation model based on machine learning, and the working efficiency is improved.

Description

Vehicle damage assessment method and device, storage medium and electronic equipment

Technical Field

The present invention relates to the field of loss assessment technologies, and in particular, to a method and an apparatus for vehicle loss assessment, a storage medium, and an electronic device.

Background

With the rapid increase of the automobile holding capacity, the traffic collision accident rapidly increases, and the vehicle damages more and more. When the insured vehicle has a traffic accident, the vehicle damage assessment is carried out on the insured vehicle and the vehicle damage settlement amount is determined, wherein the vehicle damage assessment is a key link of the settlement. The vehicle damage assessment is that comprehensive analysis is carried out on the vehicle collision and accident site through scientific and systematic specialized inspection, test and survey means according to the vehicle structure principle, and the damage assessment pricing of a scientific system is carried out on vehicle collision repair by using vehicle damage assessment data and maintenance data.

The inventor researches and discovers that the general car insurance claim settlement flow comprises the following steps: after a traffic accident occurs to a user, the user needs to make a call to report a case to an insurance company claim department, the insurance company sends a surveyor to a site for surveying and photographing, and a subsequent insurance company derogator can determine the damage degree of a vehicle and the amount of money to be claimed according to the damage condition of the photo evaluation. The whole car insurance claim settlement process is complex, consumes too long time and has low efficiency, thus being not beneficial to the quick realization of the car insurance claim settlement.

Disclosure of Invention

The invention aims to provide a vehicle damage assessment method, accelerate the vehicle damage assessment claim settlement process and solve the problem of low efficiency in the vehicle damage assessment process.

The invention also provides a vehicle damage assessment device used for ensuring the realization and application of the method in practice.

A vehicle damage assessment method comprising:

when a damage assessment request is received, determining a damaged vehicle image sequence corresponding to the damage assessment request;

constructing a damaged vehicle three-dimensional model corresponding to the damaged vehicle according to the damaged vehicle image sequence;

comparing the damaged vehicle three-dimensional model with a standard vehicle three-dimensional model corresponding to the type of the damaged vehicle, and determining deformation information of the damaged vehicle;

and inputting the deformation information into a pre-established vehicle damage calculation model, and determining the damage grade of the damaged vehicle after the deformation information is processed by the vehicle damage calculation model.

The method described above, optionally, the determining a damaged vehicle image sequence corresponding to the damage assessment request includes:

acquiring a damaged image of each damaged part in the damaged vehicle and a part identifier corresponding to each damaged part;

according to the part identification corresponding to each damaged part, searching a standard part image corresponding to each part identification in a pre-established standard vehicle image sequence corresponding to the vehicle type of the damaged vehicle;

and correspondingly replacing the searched standard component images corresponding to the component identifications with damaged images of all damaged components in the damaged vehicle to obtain the damaged vehicle image sequence.

The method described above, optionally, the process of establishing the standard vehicle image sequence includes:

collecting multi-angle vehicle videos of all vehicle types, and generating a vehicle video file corresponding to each vehicle type;

sampling a plurality of vehicle images in the vehicle video file corresponding to each vehicle type according to a preset sampling mode to form an initial image sequence corresponding to the vehicle type;

and performing pixel segmentation on each vehicle image in the initial image sequence corresponding to each vehicle type to obtain a part image of each vehicle part in each vehicle image, and forming the standard vehicle image sequence by using the obtained part images.

In the above method, optionally, the constructing a damaged vehicle stereo model corresponding to the damaged vehicle according to the damaged vehicle image sequence includes:

determining a corresponding spatial point cloud of each current component image in the damaged vehicle image sequence in a three-dimensional space coordinate system;

performing triangulation network processing on the space points in each space point cloud to obtain a space topological structure corresponding to each space point cloud;

and performing texture mapping on each space topological structure, and combining each space topological structure into a damaged vehicle three-dimensional model corresponding to the damaged vehicle.

Optionally, the determining the deformation information of the damaged vehicle by comparing the damaged vehicle stereo model with the standard vehicle stereo model corresponding to the vehicle type of the damaged vehicle includes:

determining a damaged shape distribution matrix corresponding to the damaged vehicle three-dimensional model and a standard shape distribution matrix corresponding to the standard vehicle three-dimensional model;

calculating an Euler distance between the damaged shape distribution matrix and the standard shape distribution matrix;

and calculating the similarity between the damaged vehicle three-dimensional model and the standard vehicle three-dimensional model according to the Euler distance, and taking the calculated similarity as the deformation information of the damaged vehicle.

The method described above, optionally, the building process of the vehicle damage calculation model includes:

acquiring deformation information of a plurality of determined damaged vehicles and a vehicle damage assessment grade corresponding to the deformation information of each determined damaged vehicle;

and inputting the deformation information of the plurality of determined damaged vehicles and the vehicle damage assessment grade corresponding to the deformation information of each determined damaged vehicle into a neural network model, and training the neural network model to obtain the vehicle damage calculation model.

The above method, optionally, further includes:

and searching the claim amount corresponding to the damage level of the damaged vehicle in a preset comparison table of the claim amount.

A vehicle damage assessment device comprising:

the model building unit is used for determining a damaged vehicle image sequence corresponding to the damage assessment request when the damage assessment request is received, and building a damaged vehicle three-dimensional model corresponding to a damaged vehicle according to the damaged vehicle image sequence;

the deformation calculation unit is used for comparing the damaged vehicle three-dimensional model with a standard vehicle three-dimensional model corresponding to the vehicle type of the damaged vehicle to determine the deformation information of the damaged vehicle;

and the damage analysis unit is used for inputting the deformation information into a pre-established vehicle damage calculation model, and determining the damage grade of the damaged vehicle after the deformation information is processed by the vehicle damage calculation model.

A storage medium comprising stored instructions, wherein the instructions, when executed, control a device in which the storage medium is located to perform the above-described vehicle damage assessment method.

An electronic device comprising a memory, and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by the one or more processors to perform the vehicle impairment method described above.

Compared with the prior art, the invention has the following advantages:

the invention provides a vehicle damage assessment method, which comprises the following steps: when a damage assessment request is received, determining a damaged vehicle image sequence corresponding to the damage assessment request; constructing a damaged vehicle three-dimensional model corresponding to the damaged vehicle according to the damaged vehicle image sequence; comparing the damaged vehicle three-dimensional model with a standard vehicle three-dimensional model corresponding to the type of the damaged vehicle, and determining deformation information of the damaged vehicle; and inputting the deformation information into a pre-established vehicle damage calculation model, and determining the damage grade of the damaged vehicle after the deformation information is processed by the vehicle damage calculation model. According to the vehicle damage assessment method, when a damage assessment platform receives a damage assessment request sent by a user, a damaged vehicle image sequence corresponding to the damage assessment request is determined, a three-dimensional model is constructed according to the damaged vehicle image sequence, a damaged vehicle three-dimensional model corresponding to a damaged vehicle is constructed, the damaged vehicle three-dimensional model is compared with a standard vehicle three-dimensional model corresponding to the damaged vehicle, deformation information of the damaged vehicle is determined, the deformation information is input into a pre-established vehicle damage calculation model based on machine learning, and after the vehicle damage calculation model is processed, the damage grade of the damaged vehicle is determined. By applying the vehicle damage assessment method provided by the invention, the deformation information of the damaged vehicle can be rapidly calculated, so that the damage level of the damaged vehicle is determined, a large amount of human resources and time cost are saved, more services can be automatically processed in unit time by applying a vehicle damage calculation model based on machine learning, and the working efficiency is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

FIG. 1 is a flow chart of a method of vehicle damage assessment provided by the present invention;

FIG. 2 is a flow chart of another method of the present invention for a vehicle damage assessment method;

FIG. 3 is an exemplary diagram of a vehicle damage assessment method provided by the present invention;

FIG. 4 is a diagram illustrating another exemplary method for determining damage to a vehicle according to the present invention;

FIG. 5 is a flow chart of yet another method of a vehicle damage assessment method provided by the present invention;

FIG. 6 is a flow chart of yet another method of a vehicle damage assessment method provided by the present invention;

FIG. 7 is a flow chart of yet another method of a vehicle damage assessment method provided by the present invention;

FIG. 8 is a schematic structural diagram of a vehicle damage assessment device according to the present invention;

fig. 9 is a schematic structural diagram of an electronic device provided in the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The invention is operational with numerous general purpose or special purpose computing device environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multi-processor apparatus, distributed computing environments that include any of the above devices or equipment, and the like.

The embodiment of the present invention provides a vehicle damage assessment method, which can be applied to system platforms of various devices, where the devices include a personal computer, a mobile phone, and the like, an execution subject of the device may be a system platform, and includes a computer terminal or processors of various mobile devices, and a method flowchart of the vehicle damage assessment method is shown in fig. 1, and specifically includes:

s101: when a damage assessment request is received, determining a damaged vehicle image sequence corresponding to the damage assessment request;

in the method provided by the embodiment of the invention, when a vehicle is damaged, a user sends a damage assessment request to a processor, and the processor determines a damaged vehicle image sequence corresponding to the damage assessment request when receiving the damage assessment request.

S102: constructing a damaged vehicle three-dimensional model corresponding to the damaged vehicle according to the damaged vehicle image sequence;

according to the method provided by the embodiment of the invention, the damaged vehicle image sequence can be an image sequence formed by images of all parts of a damaged vehicle, and a damaged vehicle three-dimensional model corresponding to the damaged vehicle is constructed according to the image sequence formed by images of all parts of the damaged vehicle.

S103: comparing the damaged vehicle three-dimensional model with a standard vehicle three-dimensional model corresponding to the type of the damaged vehicle, and determining deformation information of the damaged vehicle;

in the method provided by the embodiment of the invention, the damaged vehicle three-dimensional model corresponding to the damaged vehicle is compared with the standard vehicle three-dimensional model corresponding to the vehicle type of the damaged vehicle, and the deformation information of the damaged vehicle is determined.

S104: and inputting the deformation information into a pre-established vehicle damage calculation model, and determining the damage grade of the damaged vehicle after the deformation information is processed by the vehicle damage calculation model.

In the method provided by the embodiment of the invention, the deformation information of the damaged vehicle is input into a pre-established vehicle damage calculation model, the pre-established vehicle damage calculation model is based on a machine learning vehicle damage calculation model, and the damage grade of the damaged vehicle is determined after the processing of the machine learning based vehicle damage calculation model.

In the method provided by the embodiment of the invention, when the vehicle is damaged, a user sends a damage assessment request to the processor, so that the processor determines a damaged vehicle image sequence corresponding to the damage assessment request when receiving the damage assessment request, the damaged vehicle image sequence is an image sequence formed by images of all parts of a damaged vehicle, a damaged vehicle three-dimensional model corresponding to the damaged vehicle is constructed according to the image sequence formed by the images of all parts of the damaged vehicle, the damaged vehicle three-dimensional model corresponding to the damaged vehicle is compared with a standard vehicle three-dimensional model corresponding to the type of the damaged vehicle, deformation information of the damaged vehicle is determined, the deformation information of the damaged vehicle is input into a pre-established vehicle damage calculation model based on machine learning, and determining the damage grade of the damaged vehicle after the vehicle damage calculation model based on machine learning is processed. By applying the vehicle damage assessment method provided by the invention, the deformation information of the damaged vehicle can be rapidly calculated, so that the damage level of the damaged vehicle is determined, a large amount of human resources and time cost are saved, more services can be automatically processed in unit time by applying a vehicle damage calculation model based on machine learning, and the working efficiency is improved.

In the vehicle damage assessment method provided in the embodiment of the present invention, the process of determining the damaged vehicle image sequence corresponding to the damage assessment request may specifically include, as shown in fig. 2:

s201: acquiring a damaged image of each damaged part in the damaged vehicle and a part identifier corresponding to each damaged part;

the acquiring damaged images of each damaged part in the damaged vehicle comprises:

acquiring a damaged vehicle image containing damaged parts in the damaged vehicle;

and carrying out pixel segmentation on the damaged image of the vehicle to obtain a damaged image of each damaged part in the damaged vehicle.

In the method provided by the embodiment of the invention, a user uploads a damaged vehicle image of a damaged vehicle to a processor through a vehicle damage assessment interface, and interactively inputs a part identifier corresponding to each damaged part to the processor, the processor receives the damaged vehicle image of the damaged vehicle and the part identifier corresponding to each damaged part, and the damaged vehicle image is subjected to pixel segmentation by adopting a convolutional neural network Mask R-CNN method to obtain the damaged image of each damaged part in the damaged vehicle.

It should be noted that, the first time the user opens the vehicle damage assessment interface, the personal information needs to be registered, which includes: the three-dimensional vehicle model is called to be displayed on an interface according to the vehicle information such as the vehicle type, the vehicle color and the like, so that the interaction of subsequent users is facilitated.

The method for segmenting the damaged image of the vehicle by adopting the convolutional neural network Mask R-CNN method comprises the following specific steps:

firstly, inputting a damaged vehicle image of the damaged vehicle, and then preprocessing the damaged vehicle image, or directly inputting a preprocessed damaged vehicle image; then, inputting the preprocessed damaged picture of the vehicle into a pre-trained neural network ResNeXt to obtain a corresponding feature image feature map; then, setting a predetermined number of interested region ROIs for each feature point in the feature map, thereby obtaining a plurality of candidate ROIs; then, the candidate ROIs are sent to an RPN network for binary classification (foreground or background) and BB regression to obtain component images in the images, and then, a part of the candidate ROIs are filtered; then, performing regional feature clustering ROIAlign operation on the rest ROIs (i.e. corresponding the original image and the pixels of the feature map, and then corresponding the feature map and the fixed feature); finally, classification (N category classification), BB regression and MASK generation (FCN operation inside each ROI) are performed on these ROIs, and finally damaged images of each damaged component in the damaged vehicle are acquired.

S202: according to the part identification corresponding to each damaged part, searching a standard part image corresponding to each part identification in a pre-established standard vehicle image sequence corresponding to the vehicle type of the damaged vehicle;

wherein, the establishment process of the standard vehicle image sequence comprises the following steps:

In the method provided by the embodiment of the invention, all-directional 360-degree vehicle videos of all vehicle types are collected through terminal equipment, a vehicle video file corresponding to each vehicle type is generated, a plurality of vehicle images are generated by sampling in the vehicle video file corresponding to each vehicle type according to a sampling mode of every N frames to form an initial image sequence corresponding to the vehicle type, N is a positive integer, the value of N is determined according to actual requirements, pixel segmentation is carried out on each vehicle image in the initial image sequence corresponding to each vehicle type to obtain a standard component image of each vehicle component in each vehicle image, and the obtained standard component images are formed into the standard vehicle image sequence. The method for obtaining the component image of each vehicle component in each vehicle image by performing pixel segmentation on each vehicle image in the initial image sequence corresponding to each vehicle type is the same as the method for obtaining the damaged image of each damaged component in the damaged vehicle by performing pixel segmentation on the damaged image of the vehicle, and is not repeated here.

Preferably, a 2D vehicle image database is constructed based on the standard vehicle image sequence, and the standard vehicle image sequence stored in the standard 2D vehicle database may be stored correspondingly according to vehicle type, or may be stored according to image number, for example, the end of number 01 in the image sequence of each vehicle type is stored in the same table.

It should be noted that each image in the standard vehicle image sequence has a corresponding vehicle orientation label and a corresponding vehicle component label; the vehicle orientation information is obtained by performing camera matrix calculation by using a Bundle Adjustment algorithm by using a beam Adjustment method, so that the vehicle orientation information of each vehicle template image is obtained.

The concrete steps of the Bundle Adjustment algorithm are as follows:

bundle Adjustment, a beam Adjustment method, uses the LM algorithm to minimize the error between the observed and predicted image point coordinates. If the corresponding relation of the image feature points and the initial three-dimensional points are given, the BA can refine the 3D coordinates and the corresponding camera parameters corresponding to the feature points at the same time.

Assuming that there are n three-dimensional object points in space, and m pictures are now taken around these object points, the ith object point seen on the jth picture is x_ij. Bundle adjustment aims at optimizing the parameter estimation of the initial multiple cameras and structures in order to find reasonable parameters so that we can accurately calculate the spatial coordinates of n object points in m photos. More specifically, the vector a for each camera j_jRepresenting (internal and external) each three-dimensional object point i by a vector b_iAnd (4) showing. To simplify the problem, it is assumed that all object points are now visible in all pictures. The core problem of BA is to minimize the following reprojection error function (non-linearity):

function Q (a)_j,b_i) Representing object point b_iAt camera a_jThe projected coordinates of the lower part are also predicted values. The function d (x, y) represents the viewEuclidean distances between the measured image coordinates and the predicted image coordinates.

The vector P represents all the parameters of the m projection matrices and of the n three-dimensional object points:

the observed coordinates of all object points within all photographs are represented by vector X:

suppose P₀Is an initial parameter estimation (given initial three-dimensional object point coordinates and camera internal and external parameters), sigma_XA covariance matrix (default is an identity matrix, representing the irrelevance of each observation vector) representing the observation vector X. Using the projection function relationship X '═ f (p), the prediction vector X' can be calculated from each set of parameters:

wherein x_i'_j＝Q(a_j,b_i) Representing the predicted projection coordinates.

Thus, the BA problem becomes minimizing the Mahalanobis distance

Epsilon is the residual epsilon for P-X'. This minimization problem can be solved using the LM nonlinear least squares algorithm, with each iteration step size δ:

j is the Jacobian matrix for the projection relationship f, is the iteration step size, and minimizes the residual function by computing a reasonable P. The internal and external parameters of the camera corresponding to each image can be obtained through P, and the posture of the camera is calculated, so that the space direction information of the camera corresponding to the image, namely the vehicle orientation information, can be converted.

In the method provided by the embodiment of the invention, according to the part identifier corresponding to each damaged part, the vehicle part marking information corresponding to each image in the standard vehicle image sequence corresponding to the pre-established vehicle type of the damaged vehicle and the personal information registered by the user through the vehicle damage assessment interface, the standard part image corresponding to each part identifier is searched in the standard vehicle image sequence.

S203: and correspondingly replacing the searched standard component images corresponding to the component identifications with damaged images of all damaged components in the damaged vehicle to obtain the damaged vehicle image sequence.

In the method provided in this embodiment, based on the found standard component images corresponding to the component identifiers, the found standard component images corresponding to the component identifiers are replaced with damaged images of damaged components in the damaged vehicle, so as to obtain the damaged vehicle image sequence.

In the method provided by the embodiment of the present invention, with reference to fig. 3, an example of an acquisition process of an image of a left headlamp and an image of a left front wheel of popular treasure in a standard vehicle image sequence is as follows:

the left headlight is denoted by A, the left front wheel is denoted by B, and the process of acquiring the A image and the B image comprises the following steps:

acquiring a popular treasure omnibearing 360-degree vehicle video, generating a popular treasure vehicle video file, sampling in the popular treasure vehicle video file according to an N-frame sampling mode to generate an image containing an A component and an image containing a B component to form an initial image sequence containing the A component and an initial image sequence containing the B component, respectively performing pixel segmentation on the initial image sequence containing the A component and the initial image sequence containing the B component by adopting a convolutional neural network Mask R-CNN method to obtain a standard A component image and a standard B component image, storing the standard A component image and the standard B component image in a standard image sequence, constructing a 2D vehicle image database based on the standard vehicle image sequence, namely storing the standard A component image and the standard B component image in a 2D vehicle image database, and the 2D vehicle image database contains standard A part images with corresponding vehicle orientation labels and A part labels, and standard B part images with corresponding vehicle orientation labels and B part labels.

In the method provided by the embodiment of the present invention, with reference to fig. 4, the process of replacing the standard a component image with the a 'component image and replacing the standard B component image with the B' component image in the 2D database is exemplified as follows:

a user uploads a damaged image containing an A 'component and a damaged image containing a B' component from a damaged popular treasure to a processor through a vehicle damage interface, and interacts with a component identifier of the A 'and a component identifier of the B' input to the processor through a user interface, wherein the component identifier of the A 'can be the name of the A component, the component identifier of the B' can be the name of the B component, the processor receives the damaged images of all vehicles of the damaged vehicles and the component identifiers corresponding to all the damaged components, performs pixel segmentation on the damaged images of the vehicles by adopting a convolutional neural network Mask R-CNN method to obtain the image of the A 'component and the image of the B' component, searches a standard image of the A component and a standard image of the B component in a 2D vehicle image database respectively based on the component identifier of the A 'and the component identifier of the B', and replaces the standard image of the A 'component correspondingly into the image of the A' component, and the standard B part image is correspondingly replaced with a B' part image.

In the vehicle damage assessment method provided by the embodiment of the invention, a user uploads a damaged vehicle image of a damaged vehicle to a processor through a vehicle damage assessment interface, and inputs a component identifier corresponding to each damaged component to the processor through a user interface in an interactive manner, the processor receives the damaged vehicle image of the damaged vehicle and the component identifier corresponding to each damaged component, performs pixel segmentation on the damaged vehicle image by adopting a convolutional neural network Mask R-CNN method to obtain the damaged image of each damaged component in the damaged vehicle, searches a standard component image corresponding to each component identifier in the standard vehicle image sequence according to the component identifier corresponding to each damaged component and vehicle component label information corresponding to each image in the standard vehicle image sequence corresponding to a vehicle type of the damaged vehicle established in advance, and replacing the searched standard component images corresponding to the component identifications with damaged images of all damaged components in the damaged vehicle correspondingly based on the searched standard component images corresponding to the component identifications, so as to obtain the damaged vehicle image sequence, wherein a 2D vehicle image database is constructed based on the standard vehicle image sequence. By applying the vehicle damage assessment method provided by the invention, a fine 2D vehicle image database is constructed, and the standard component image corresponding to each component identifier can be quickly found based on the component identifier corresponding to the damaged component and the component marking information in the image sequence corresponding to each vehicle type in the 2D vehicle database, so that the efficiency is improved.

In the method for determining damage of a vehicle according to the embodiment of the present invention, a process of constructing a damaged vehicle three-dimensional model corresponding to a damaged vehicle according to the damaged vehicle image sequence may specifically include, as shown in fig. 5:

s301: determining a corresponding spatial point cloud of each current component image in the damaged vehicle image sequence in a three-dimensional space coordinate system;

in the method provided by this embodiment, a camera may be calibrated to obtain a spatial point cloud of the damaged vehicle image sequence, where the spatial point cloud includes a 3D point cloud, or a 3D point cloud obtained by scanning the image sequence, the obtained 3D point cloud is clustered by using CMVS, and a PMVS completes dense matching through matching, expansion, and filtering to generate a dense 3D point cloud at the same time, it needs to be stated that the matching calculation is performed by using part labeling information in a 2D vehicle image database as a constraint, rejecting some points with wrong matching, and improving matching accuracy; based on the corresponding relation between the 2D vehicle image feature points and the 3D point clouds, the coordinate information of the 2D vehicle image feature points and the component labeling coordinate information of the vehicle images, dividing the dense 3D point clouds according to components, and generating a space point cloud corresponding to each current component image in the damaged vehicle image sequence in a three-dimensional space coordinate system, namely a damaged component 3D point cloud corresponding to each component image in the damaged vehicle image sequence.

The specific segmentation method comprises the following steps:

the feature points on the image can be segmented according to the coordinate information of the components, the whole 3D point cloud can be clustered into a plurality of point clouds according to the corresponding relation between the 2D feature points and the 3D points, each point cloud corresponds to an independent object, if the point clouds are clustered into the point clouds corresponding to the vehicle head, the point clouds can be segmented into one component finally, the color labeling can be carried out on the point clouds, the color of the point clouds of the same component is the same, and the color of the point clouds of different components is different.

S302: performing triangulation network processing on the space points in each space point cloud to obtain a space topological structure corresponding to each space point cloud;

in the method provided by the embodiment of the invention, triangulation is performed on the 3D points in the 3D point cloud of the damaged part by adopting a Delaunay triangulation algorithm, and a spatial topological structure corresponding to each spatial point cloud is obtained.

The specific steps of the Delaunay triangulation algorithm are as follows:

a. and (3) convex hull generation:

1) the following four points were obtained: min (x-y), min (x + y), max (x-y) and max (x + y) are sequentially put into an array to form an initial convex hull;

2) setting subsequent points of a point I on the convex hull as J, calculating the distance from all points on the right side of the vector line segment IJ to the point IJ, and solving a point K with the largest distance;

3) inserting K between I and J, and assigning K to J; 4) repeating the steps 2 and 3 until the point is concentrated without

Up to the point to the right of IJ; 5) assigning J to I, taking its subsequent point, repeating 2, 3, 4 steps while

And after traversing the convex hull for one time, completing the convex hull generation.

b. Triangulation of convex hull by circular cutting boundary method: in the convex hull array, a triangle formed by two adjacent convex edge covers is searched each time, any other point on the convex hull is not contained in the triangle and on the boundary, then the point is removed to obtain a new convex hull linked list, the process is repeated, and finally the triangulation of the point in the convex hull array is successful.

c. Discrete interpolation:

1) establishing a circumscribed circle of the triangle, and finding out all triangles of which the circumscribed circle comprises points to be inserted to form an insertion area;

2) deleting the common side of the triangle in the insertion area to form a polygon formed by the vertexes of the influencing triangle;

3) connecting the insertion points with all the vertexes of the polygon to form a new Delaunay triangle;

4) repeat 1, 2, 3 until all discrete points on the non-convex hull are inserted.

S303: and performing texture mapping on each space topological structure, and combining each space topological structure into a damaged vehicle three-dimensional model corresponding to the damaged vehicle.

In the method provided by the embodiment of the invention, 3D point clouds of the damaged vehicle image sequence are obtained, clustering is carried out on the obtained 3D point clouds by using CMVS, dense matching is completed by PMVS through matching, expansion and filtering, dense 3D point clouds are generated at the same time, the dense 3D point clouds are divided according to components, damaged component 3D point clouds corresponding to each component image in the damaged vehicle image sequence are generated, triangulation is carried out on 3D points in the damaged component 3D point clouds by using a Delaunay triangulation algorithm, a space topological structure corresponding to each space point cloud is obtained, texture mapping is carried out on each space topological structure, and each space topological structure is combined into a damaged vehicle stereo model corresponding to the damaged vehicle.

In the vehicle damage assessment method provided in the embodiment of the present invention, the process of comparing the damaged vehicle stereo model with the standard vehicle stereo model corresponding to the vehicle type of the damaged vehicle to determine the deformation information of the damaged vehicle may specifically include, as shown in fig. 6:

s401: determining a damaged shape distribution matrix corresponding to the damaged vehicle three-dimensional model and a standard shape distribution matrix corresponding to the standard vehicle three-dimensional model;

s402: calculating an Euler distance between the damaged shape distribution matrix and the standard shape distribution matrix;

s403: and calculating the similarity between the damaged vehicle three-dimensional model and the standard vehicle three-dimensional model according to the Euler distance, and taking the calculated similarity as the deformation information of the damaged vehicle.

IN the method provided by the embodiment of the invention, based on the damaged vehicle three-dimensional model, a Monte Carlo method is adopted to randomly generate sampling sample points, Euclidean distances and IP values between point pairs of the sample points are calculated, the positions of intersection points of connecting lines of the sample point pairs and the damaged vehicle three-dimensional model are judged, whether the sample point pairs belong to IN class or UNIN class is judged according to the positions of the intersection points, the point pairs corresponding to each type are counted, and a damaged shape distribution matrix is generated according to the point pairs corresponding to each type; the calculation method of the standard shape distribution matrix corresponding to the standard vehicle three-dimensional model is the same as that of the damaged shape distribution matrix, and is not repeated here; calculating Euler distance between the damaged shape distribution matrix and the standard shape distribution matrix, calculating similarity between the damaged vehicle three-dimensional model and the standard vehicle three-dimensional model according to the Euler distance, and taking the similarity obtained by calculation as deformation information of the damaged vehicle, wherein the deformation information is a deformation coefficient and a deformation area.

The specific algorithm for calculating the deformation information of the damaged vehicle is as follows:

1) the Monte Carlo method is adopted to randomly generate sampling sample points, and Monte Carlo can rapidly obtain a series of sampling points uniformly distributed on the three-dimensional model according to the equal area principle, and has high robustness.

2) And calculating Euclidean distances and IP values of the sample point pairs, judging the intersection condition of the connecting line of the sample point pairs and the model one by one, and judging the type of the connecting line.

a. Let any two sample points be P1 and P2, the segment S between the two points is represented as:

S(t)＝P₁+t(P₂-P₁),0＜t＜1 (1)

b. assuming that the three vertices of the triangular patch are V1, V2, and V3, the coordinates of any point Q in the triangular patch can be expressed as:

c. substituting equation (1) into equation (2) yields:

integrated to obtain:

solving equation (4) with the rule of claime to obtain the intersection point.

d. And judging the type of the point pair according to the position and the number of the intersection points.

3) Generating a shape distribution matrix

The value range of the IP value is [0,1 ]]The range is equally divided into S₁Unit, the interval of each unit is 1/S₁The IP range of the ith unit is

The Euler distance has the characteristics of translation and rotation invariance, but is influenced by the three-dimensional model proportion transformation, and needs to be subjected to proportion normalization: if necessary, the distance range is divided into S₂And thus a distribution normalized by the average distance. Respectively counting all sample point pairs belonging to IN class and UNIN class according to the point pair shape function values obtained by calculation IN the step 2) and the classes thereof to generate two S₁×S₂Shape distribution matrix of

s_ijThe total number of pairs of sample points whose distance value belongs to cell j for which the IP value belongs to cell i. The improved shape histogram is a curved surface, and s is determined by taking the IP value as the x axis and the distance value as the y axis_ijIs a point (x)_i,y_i) The value of the curve of (a).

4) Computing similarity of models

The distance of the shape distribution matrix is calculated using L2-norm (Euler distance), and the similarity of the model is evaluated based on this.

Setting the model A corresponding to a damaged vehicle three-dimensional model, setting the model B corresponding to a standard vehicle three-dimensional model, and setting the shape distribution matrix as follows:

the euler distance is defined as:

the similarity of the models is:

wherein D_maxTo obtain the maximum Euler distance

In the method provided by the embodiment of the invention, the standard vehicle three-dimensional model is constructed based on a standard vehicle image sequence, and the construction process of the standard vehicle three-dimensional model is the same as that of the damaged vehicle three-dimensional model, and is not repeated here. Preferably, the standard vehicle stereo model is a standard 3D vehicle model, and a standard 3D vehicle model database is constructed based on the standard 3D vehicle model, wherein each 3D vehicle model in the standard 3D vehicle database corresponds to a vehicle component tag, and each component corresponds to an image index in the 2D vehicle database.

In the method provided by the embodiment of the invention, according to the part identifier corresponding to each damaged part, a standard part image corresponding to each part identifier is searched in a pre-established standard vehicle image sequence corresponding to the vehicle type of the damaged vehicle, and the standard part image corresponding to each part identifier is searched in a 2D vehicle database according to the part identifier corresponding to each damaged part and the incidence relation between standard 2D vehicle image data and standard 3D vehicle model data.

In the method for determining vehicle damage provided in the embodiment of the present invention, as shown in fig. 7, the process of establishing the vehicle damage calculation model may specifically include:

s501: acquiring deformation information of a plurality of determined damaged vehicles and a vehicle damage assessment grade corresponding to the deformation information of each determined damaged vehicle;

s502: and inputting the deformation information of the plurality of determined damaged vehicles and the vehicle damage assessment grade corresponding to the deformation information of each determined damaged vehicle into a neural network model, and training the neural network model to obtain the vehicle damage calculation model.

In the method provided by the embodiment of the invention, the existing vehicle insurance claim data and vehicle insurance making rules are collected, and a large amount of sample data is generated through the vehicle insurance claim data and the vehicle insurance making rules, wherein each sample data comprises: vehicle type, color, damaged part, damaged area, damaged coefficient, vehicle damage grade, claim settlement amount and the like; inputting data such as vehicle types, colors, damaged parts, damaged areas and damaged coefficients in the sample data into a neural network model, training the neural network model, outputting the training neural network model as a vehicle damage grade corresponding to the input sample data, and searching a claim amount corresponding to the damage grade of the damaged vehicle in a preset claim amount comparison table.

The invention adopts a Classification Regression Tree (CRT Classification Regression Tree) to train the damage model.

The concrete steps of building the tree by the CRT are as follows:

and (3) recursively building a tree: and building the tree in a recursive mode, adopting 2/3 samples of all samples to build the tree, firstly finding a division value, if return-1 does not exist, then judging whether the tree is a leaf node or not, and not dividing the tree for the leaf node according to the division value.

The CART tree building principle is as follows: first, the evaluation value of CART is considered as a constant of the sample space, i.e., as an average value of the respective variables. When the observed value of the response variable changes, the evaluation value of CART can be expressed as:

wherein R denotes the sample space, I_R(x) Is an index function of R.

The sample space is then divided into two parts R₁And R₂Selecting a specific regression variable X_j. If X is_jFor a continuous random variable, a specified scalar α is selected and defined as:

R₁＝{x∈R:x_j≤α},R₂＝{x∈R:x_j＞α}

then, continue to separately R₁And R₂The segmentation is done in the above manner until the number of observation samples becomes small (typically 5).

CART pruning algorithm: to avoid excessively large trees growing, the trees are pruned. Setting:

let 0 ≦ α ≦ infinity be the complex parameter, as follows:

R(τ)_α＝R(τ)+αLeaves(τ)

selecting an optimal tree: the above merit determination is achieved by a cross-validation evaluation method. For example, in a 10-fold cross validation check. 90% of samples are taken to generate CART, a sub-tree queue is generated through pruning, and then the square of the redundancy of each sub-tree in the queue is calculated. The test was then repeated 10 times using 10% of the remaining samples as a test set, with each time an evaluation test being made for a different portion of the sample.

By applying the vehicle damage assessment method provided by the embodiment of the invention, the personal information and the deformation information of the user are input into the vehicle damage calculation model, the vehicle damage calculation model can rapidly output the damage grade and the claim settlement amount, more service problems are automatically processed in unit time, and the working efficiency is improved.

Corresponding to the method illustrated in fig. 1, an embodiment of the present invention further provides a vehicle damage assessment apparatus, which is used for implementing the method illustrated in fig. 1 specifically, where the vehicle damage assessment apparatus provided in the embodiment of the present invention may be applied to a computer terminal or various mobile devices, and a schematic structural diagram of the vehicle damage assessment apparatus is illustrated in fig. 8, and specifically includes:

the model building unit 601 is configured to, when a damage assessment request is received, determine a damaged vehicle image sequence corresponding to the damage assessment request, and build a damaged vehicle stereo model corresponding to a damaged vehicle according to the damaged vehicle image sequence;

a deformation calculation unit 602, configured to compare the damaged vehicle stereo model with a standard vehicle stereo model corresponding to the vehicle type of the damaged vehicle, and determine deformation information of the damaged vehicle;

and the damage analysis unit 603 is configured to input the deformation information into a pre-established vehicle damage calculation model, and determine the damage level of the damaged vehicle after the deformation information is processed by the vehicle damage calculation model.

In an embodiment of the present invention, based on the foregoing scheme, the model building unit 601 is configured to:

In an embodiment of the present invention, based on the foregoing solution, the deformation calculation unit 602 is configured to:

In an embodiment of the present invention, based on the foregoing scheme, the damage analysis unit 603 is configured to:

In an embodiment of the present invention, based on the foregoing solution, the vehicle damage assessment apparatus further includes: and the searching unit is used for searching the claim amount corresponding to the damage grade of the damaged vehicle in a preset claim amount comparison table after the damage grade of the damaged vehicle is determined.

According to the vehicle damage assessment device provided by the embodiment of the invention, when a vehicle is damaged, a user sends a damage assessment request to the processor, so that the processor determines a damaged vehicle image sequence corresponding to the damage assessment request when receiving the damage assessment request, the damaged vehicle image sequence is an image sequence formed by images of all parts of a damaged vehicle, a damaged vehicle three-dimensional model corresponding to the damaged vehicle is constructed according to the image sequence formed by the images of all parts of the damaged vehicle, the damaged vehicle three-dimensional model corresponding to the damaged vehicle is compared with a standard vehicle three-dimensional model corresponding to the type of the damaged vehicle, deformation information of the damaged vehicle is determined, the deformation information of the damaged vehicle is input into a pre-established vehicle damage calculation model based on machine learning, and determining the damage grade of the damaged vehicle after the vehicle damage calculation model based on machine learning is processed. The vehicle damage assessment device provided by the invention can be used for rapidly calculating the deformation information of the damaged vehicle so as to determine the damage level of the damaged vehicle, so that a large amount of human resources and time cost are saved, and more services can be automatically processed in unit time by applying a vehicle damage calculation model based on machine learning, so that the working efficiency is improved.

The embodiment of the present invention further provides a storage medium, where the storage medium includes a stored instruction, where when the instruction runs, the apparatus where the storage medium is located is controlled to perform the following operations:

The method described above is optionally characterized in that the process of establishing the vehicle damage calculation model includes:

The above method, optionally, further includes:

An electronic device is provided in an embodiment of the present invention, and its structural diagram is shown in fig. 9, which specifically includes a memory 701 and one or more instructions 702, where the one or more instructions 702 are stored in the memory 701, and are configured to be executed by one or more processors 703 to perform the following operations according to the one or more instructions 702:

In the method provided by the embodiment of the present invention, the implementation of the vehicle damage assessment method is exemplified as follows:

utilizing terminal equipment to respectively carry out omnibearing 360-degree video acquisition on the popular treasure and the popular new treasure HS so as to generate vehicle video files corresponding to the popular treasure and the popular new treasure HS, sampling the vehicle video files of the popular treasure and the vehicle video files of the popular new treasure HS according to an every N-frame sampling mode so as to generate initial image sequences corresponding to the popular treasure HS and the popular new treasure HS, carrying out pixel segmentation by adopting a convolutional neural network Mask R-CNN method based on the initial image sequences so as to obtain standard component images of each vehicle component of the popular treasure, constructing a standard image sequence of the popular treasure based on the standard component images of the popular treasure HS, obtaining standard component images of each vehicle component of the popular new treasure HS, constructing the standard image sequence of the popular new HS based on the standard component images of the popular new treasure HS, and storing the standard vehicle image sequence from Volkswagen Baotai and the standard vehicle image sequence from Volkswagen Xinbaolai HS in a 2D vehicle image database, wherein the 2D vehicle image database stores vehicle orientation marks and component marks corresponding to the standard image sequence from Volkswagen Baotai and the standard image sequence from Volkswagen Xinbaolai HS.

After a camera is calibrated, or 3D point cloud scanning is carried out, 3D point cloud of a popular treasure standard image sequence and 3D point cloud of a popular new treasure HS standard image sequence are obtained, clustering is carried out on the obtained 3D point cloud by using CMVS, dense matching is completed by PMVS through matching, expansion and filtering, dense 3D point cloud is generated at the same time, the dense 3D point cloud is divided according to components, component 3D point cloud corresponding to each component image in the popular treasure standard image sequence and component 3D point cloud corresponding to each component image in the popular treasure HS standard image sequence are generated, triangulation is carried out on the 3D points in the component 3D point cloud by using Delaunay triangulation algorithm, a space topological structure corresponding to each space point cloud is obtained, texture mapping is carried out on each space topological structure, and each space topological structure is combined into a popular treasure standard 3D vehicle corresponding to the damaged vehicle The method comprises the steps of obtaining a model and a popular New Zealand HS standard 3D vehicle model, storing the standard 3D vehicle model in a standard 3D vehicle model database, wherein the popular Zealand standard 3D vehicle model in the standard 3D vehicle database and the popular New Zealand HS standard 3D vehicle model respectively correspond to vehicle part labels, and each part corresponds to an image index in a 2D vehicle database.

For example, a vehicle owner sends a vehicle damage assessment request to a processor through a vehicle damage assessment interface, uploads a vehicle damaged image where a damaged part from the public treasure is located, uploads a part identifier corresponding to each damaged part in a user 3D interaction manner, the processor receives the vehicle damaged image of the damaged vehicle and the part identifier corresponding to each damaged part, performs pixel segmentation on the damaged image from the public treasure by adopting a convolutional neural network Mask R-CNN method to obtain the damaged image of each damaged part from the public treasure, searches a standard part image corresponding to each part identifier from the public treasure in a 2D vehicle database according to the part identifier corresponding to each damaged part and the association relationship between standard 2D vehicle image data and standard 3D vehicle model data, and searches the searched standard part image corresponding to each part identifier from the damaged public treasure, and correspondingly replacing damaged images of all damaged parts by the damaged public treasure, after the replacement is completed, matching all standard part images except the damaged images of the damaged parts in the public treasure in a standard 2D vehicle database to obtain a damaged vehicle image sequence of the damaged public treasure, performing three-dimensional model reconstruction based on the damaged vehicle image sequence of the damaged public treasure to generate a damaged 3D vehicle model, comparing the damaged 3D vehicle model with the 3D vehicle model of the public treasure in the standard 3D vehicle model database, calculating a deformation coefficient and a damaged area of the damaged public treasure, inputting the deformation coefficient and the damaged area into a pre-established vehicle damage calculation model, and determining the damage grade and the public claim amount of the damaged public treasure after the vehicle damage calculation model is processed.

It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. For the device-like embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functions of the units may be implemented in the same software and/or hardware or in a plurality of software and/or hardware when implementing the invention.

From the above description of the embodiments, it is clear to those skilled in the art that the present invention can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which may be stored in a storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.

The method and the device for determining the damage of the vehicle provided by the invention are described in detail, the principle and the implementation mode of the invention are explained by applying specific examples, and the description of the examples is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims

1. A method of vehicle damage assessment, comprising:

inputting the deformation information into a pre-established vehicle damage calculation model, and determining the damage grade of the damaged vehicle after the deformation information is processed by the vehicle damage calculation model;

comparing the damaged vehicle three-dimensional model with a standard vehicle three-dimensional model corresponding to the vehicle type of the damaged vehicle, and determining deformation information of the damaged vehicle, wherein the method comprises the following steps:

calculating the similarity between the damaged vehicle three-dimensional model and the standard vehicle three-dimensional model according to the Euler distance, and taking the calculated similarity as deformation information of the damaged vehicle, wherein the deformation information comprises a deformation coefficient and a deformation area;

wherein, the establishing process of the vehicle damage calculation model comprises the following steps:

2. The method of claim 1, wherein said determining a sequence of damaged vehicle images corresponding to said impairment request comprises:

3. The method of claim 2, wherein the standard vehicle image sequence creation process comprises:

4. The method of claim 1, wherein constructing a damaged vehicle stereo model corresponding to a damaged vehicle from the damaged vehicle image sequence comprises:

5. The method of claim 1, further comprising:

6. A vehicle damage assessment device, comprising:

the deformation calculation unit is used for comparing the damaged vehicle three-dimensional model with a standard vehicle three-dimensional model corresponding to the vehicle type of the damaged vehicle, and determining the deformation information of the damaged vehicle, wherein the damaged vehicle three-dimensional model is compared with a standard vehicle three-dimensional model corresponding to the vehicle type of the damaged vehicle, and the deformation information of the damaged vehicle is determined, and specifically comprises the following steps: determining a damaged shape distribution matrix corresponding to the damaged vehicle three-dimensional model and a standard shape distribution matrix corresponding to the standard vehicle three-dimensional model; calculating an Euler distance between the damaged shape distribution matrix and the standard shape distribution matrix; calculating the similarity between the damaged vehicle three-dimensional model and the standard vehicle three-dimensional model according to the Euler distance, and taking the calculated similarity as deformation information of the damaged vehicle, wherein the deformation information comprises a deformation coefficient and a deformation area; wherein, the establishing process of the vehicle damage calculation model comprises the following steps: acquiring deformation information of a plurality of determined damaged vehicles and a vehicle damage assessment grade corresponding to the deformation information of each determined damaged vehicle; inputting the deformation information of the plurality of determined damaged vehicles and the vehicle damage assessment grade corresponding to the deformation information of each determined damaged vehicle into a neural network model, and training the neural network model to obtain the vehicle damage calculation model;

7. A storage medium comprising stored instructions, wherein the instructions, when executed, control a device on which the storage medium is located to perform a vehicle damage assessment method according to any one of claims 1 to 5.

8. An electronic device comprising a memory and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by the one or more processors to perform the vehicle damage assessment method according to any one of claims 1 to 5.