WO2020063099A1 - 优化损伤识别结果的方法及装置 - Google Patents

优化损伤识别结果的方法及装置 Download PDF

Info

Publication number
WO2020063099A1
WO2020063099A1 PCT/CN2019/098545 CN2019098545W WO2020063099A1 WO 2020063099 A1 WO2020063099 A1 WO 2020063099A1 CN 2019098545 W CN2019098545 W CN 2019098545W WO 2020063099 A1 WO2020063099 A1 WO 2020063099A1
Authority
WO
WIPO (PCT)
Prior art keywords
damage
recognition result
frame
picture
candidate
Prior art date
Application number
PCT/CN2019/098545
Other languages
English (en)
French (fr)
Inventor
徐娟
Original Assignee
阿里巴巴集团控股有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 阿里巴巴集团控股有限公司 filed Critical 阿里巴巴集团控股有限公司
Priority to SG11202100124XA priority Critical patent/SG11202100124XA/en
Priority to EP19864038.5A priority patent/EP3859592A4/en
Publication of WO2020063099A1 publication Critical patent/WO2020063099A1/zh
Priority to US17/156,198 priority patent/US11069052B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/20—Administration of product repair or maintenance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0278—Product appraisal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0283—Price estimation or determination
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • G06Q40/08—Insurance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/40—Business processes related to the transportation industry
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/0006—Industrial image inspection using a design-rule based approach
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30156—Vehicle coating
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08—Detecting or categorising vehicles
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C5/00—Registering or indicating the working of vehicles
    • G07C5/08—Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
    • G07C5/0808—Diagnosing performance data

Definitions

  • the embodiments of the present specification relate to the field of vehicle damage determination, and in particular, to a method and device for optimizing a damage identification result.
  • the previous damage recognition result and the interaction information with the user are used as inputs to achieve the optimization of the damage recognition result.
  • a method for optimizing a damage recognition result is provided, which is applied to the processing of a single picture.
  • the method includes: obtaining a vehicle damage picture input by a user; extracting feature information of a picture based on a pre-trained damage recognition model, and determining a damage recognition result corresponding to the vehicle damage picture, wherein the damage recognition result includes at least a damage frame;
  • the user displays the damage recognition result and receives changes made by the user based on the damage recognition result; based on a pre-trained long-term and short-term memory network LSTM, the damage recognition result, the picture feature information, and the change
  • an optimized damage recognition result is obtained.
  • the damage recognition model is pre-trained based on the following steps: obtaining multiple historical vehicle damage pictures labeled with the damage recognition result; and using the convolutional neural network CNN as the training sample , Training the damage recognition model.
  • the damage recognition result further includes a damage category corresponding to the damage box; the change includes changing the damage box, and / or, changing the damage category; wherein the changing the damage box includes deleting, adding, moving, At least one of reduction and enlargement.
  • the modification includes changing a damage frame; and obtaining the optimized damage recognition result includes: obtaining the damage recognition model according to the picture feature information to obtain The candidate damage box generated from the damage recognition result, which includes the damage box in the damage result; the candidate damage box is updated based on the change; the updated candidate damage box is based on the similarity between each candidate damage box Degree, determine the optimized damage frame, and use the optimized damage frame as part of the optimized damage recognition result.
  • determining the optimized damage frame includes: determining a plurality of any of the first candidate damage frame and other candidate damage frames of the updated candidate damage frame. Similarity; inputting the plurality of similarities into a predetermined prediction model, and determining whether a first candidate damage frame is an abnormal frame according to an output result of the prediction model, the prediction model is included in the LSTM; in the first When the candidate damage frame is not an abnormal frame, the first candidate damage frame is used as a part of the optimized damage frame.
  • determining the multiple similarities between any first candidate damage box and other candidate damage boxes in the updated candidate damage box includes calculating the first candidate damage box.
  • the corresponding first feature vector is a dot product of a plurality of other feature vectors respectively corresponding to a plurality of other candidate damage frames, and a plurality of dot product results are determined as the multiple similarities.
  • the prediction model is pre-trained by a positive sample and a negative sample
  • the positive sample includes a plurality of injury regions labeled as real injuries
  • the negative sample includes a plurality of injuries labeled as real injuries. Area and at least one area marked as non-damaging.
  • the prediction model is a linear regression model.
  • the damage recognition result includes multiple damage frames, the multiple damage frames include a first damage frame, and the modification includes deleting the first damage frame;
  • the damage recognition result includes: determining a plurality of similarities between the first damage frame and a plurality of other damage frames; and optimizing a damage frame corresponding to a degree of similarity lower than a preset threshold value among the plurality of similarities as an optimization Part of the damage identification result.
  • Another method for optimizing damage recognition results is provided, which is applied to the processing of multiple pictures.
  • the method includes: obtaining first picture characteristic information of a first vehicle damage picture and a first damage recognition result corresponding to the first vehicle damage picture; and obtaining second picture characteristic information of a second vehicle damage picture and the second vehicle damage picture.
  • a second damage recognition result corresponding to a second vehicle damage picture based on a pre-trained long-term and short-term memory network LSTM, combining the first damage recognition result, the first picture feature information, the second picture feature information, and the The second damage recognition result is used as an input to obtain the optimized second damage recognition result.
  • the first picture feature information and the second picture feature information are separately extracted based on a pre-trained damage recognition model.
  • the first damage recognition result and the second damage recognition result are determined separately based on a pre-trained damage recognition model, or are determined separately based on the method of claim 1.
  • the second damage recognition result includes a first damage frame; and the obtained optimized second damage recognition result includes: determining an area from the first vehicle picture based on an area matching and positioning algorithm. A matching region matched by the first damage frame; and optimizing a category of the first damage frame according to the matching region.
  • the method further includes: using the optimized second damage recognition result and the second picture feature information, and the first damage recognition result and the first picture feature information as inputs to obtain The optimized first damage recognition result.
  • the obtained optimized second damage recognition result includes: based on an area matching positioning algorithm, from the first A matching area that matches the first damage frame is determined in the vehicle picture; the category of the first damage frame is optimized according to the matching area; and the first damage recognition result obtained after optimization includes: The region optimizes at least one damage frame in the first damage recognition result.
  • an apparatus for optimizing a damage recognition result is provided, which is applied to the processing of a single picture.
  • the device includes: an acquisition unit for acquiring a vehicle damage picture input by a user; an extraction unit for extracting picture feature information based on a pre-trained damage recognition model; a determination unit for determining a damage corresponding to the vehicle damage picture A recognition result, the damage recognition result including at least a damage frame; a display unit for displaying the damage recognition result to a user; a receiving unit for receiving a change made by the user based on the damage recognition result; an optimization unit, It is used to obtain the optimized damage recognition result based on the pre-trained long-short-term memory network LSTM, taking the damage recognition result, the picture feature information, and the change as inputs.
  • an apparatus for optimizing a damage recognition result is provided, which is applied to the processing of multiple pictures.
  • the device includes: an obtaining unit, configured to obtain first picture feature information of a first vehicle damage picture and a first damage recognition result corresponding to the first vehicle damage picture, and obtain a second picture feature of a second vehicle damage picture Information and a second damage recognition result corresponding to the second vehicle damage picture; a first optimization unit, configured to combine the first damage recognition result, the first picture feature based on a pre-trained long-short-term memory network LSTM The information, the second picture feature information, and the second damage recognition result are used as inputs to obtain an optimized second damage recognition result.
  • a computer-readable storage medium having stored thereon a computer program, which when executed in a computer, causes the computer to execute the method described in the first aspect or the second aspect.
  • a computing device including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the first aspect or the first aspect is implemented. The method described in the second aspect.
  • the feature information of the picture is extracted based on the single fixed-loss picture taken by the user, and the damage recognition result is initially identified; and then the user is provided with the damage recognition result.
  • the extracted image feature information, the initially identified damage recognition results, and user modification data are used as inputs to update the damage recognition results. If the user is still dissatisfied with the updated damage identification result, he can modify it again, until the user is satisfied with the fixed damage result.
  • FIG. 1 shows a timing diagram of a neuron in an LSTM according to one embodiment
  • FIG. 2 is a schematic diagram of an implementation scenario of an embodiment disclosed in this specification.
  • FIG. 3 shows a flowchart of a method for optimizing a damage recognition result according to an embodiment
  • Figure 4a shows an example of a vehicle damage picture
  • FIG. 4b shows an example of the damage recognition result recognized by the damage recognition model
  • FIG. 4c shows the result of damage identification after the modification by the user
  • Figure 4d shows the optimized damage recognition results
  • FIG. 5 shows a timing diagram of an LSTM in use according to an embodiment
  • FIG. 6 shows a flowchart of a method for optimizing a damage recognition result according to an embodiment
  • Figure 7a shows an example of a picture of a vehicle damage
  • FIG. 7b shows an example of the damage recognition result recognized by the damage recognition model
  • Figure 7d shows the optimized damage recognition results
  • FIG. 8 is a structural diagram of an apparatus for optimizing a damage recognition result according to an embodiment
  • FIG. 9 illustrates a structure diagram of an apparatus for optimizing a damage recognition result according to an embodiment.
  • An embodiment of the present specification discloses a method for optimizing a damage recognition result. First, the inventive concept of the method is described below.
  • the method generally used in the industry is to obtain similar pictures by comparing with the massive historical database to determine the damaged parts and their degree on the picture.
  • the accuracy of damage recognition in this way is not ideal.
  • some damage identification models are also trained by using sample labeling to identify vehicle damage.
  • the accuracy of damage recognition is not high enough. For example, real damage is likely to appear in the recognition results and can be detected correctly. At the same time, a small amount of reflections or stains will also be detected as damage, which will cause false detection.
  • the damage identification model is usually used in the fixed loss client, so that ordinary users can take on-site loss pictures and upload the on-site loss pictures to the fixed loss client, thereby realizing automatic loss determination.
  • users may be dissatisfied with the results of damage recognition.
  • users usually take additional photos or replace some photos, and then upload them to the fixed loss client to re-determine the damage.
  • the client usually takes all the updated photos as input and re-determines the damage. After so many iterations, it may still be difficult to satisfy the user, at the same time, it will cause a large resource overhead and consume more time for the user.
  • the inventor thought that the user's feedback data on the damage recognition result could be obtained by interacting with the user, and the damage recognition result could be updated in combination with the feedback data.
  • a computing unit in the LSTM can memorize previous information and use it as a subsequent input.
  • x t-1 , x t , and x t + 1 respectively represent time t-1, time t, and time t + 1, respectively.
  • a 1 , a 2 , and a 3 represent the state of the neuron at time t-1 , time t , and t + 1 , respectively, and h t-1 , h t , h t + 1 respectively represent time t-1, Output at time t and time t + 1, where:
  • FIG. 2 is a schematic diagram of an implementation scenario of an embodiment disclosed in this specification.
  • the vehicle damage picture is first input to the damage recognition model for damage recognition.
  • the damage recognition model will extract the feature information of the picture.
  • the picture includes multiple damages, such as deformation, scratches, etc., it is usually recognized
  • the model will identify multiple candidate damage areas from the picture as the detection results.
  • the modification input by the user is received, and based on the modification data and the picture characteristic information of the vehicle damage picture extracted through the damage recognition model, the detection result is updated until the user is satisfied.
  • the specific implementation process of optimizing the damage recognition result is described below.
  • FIG. 3 shows a flowchart of a method for optimizing a damage recognition result according to an embodiment.
  • the method is used to process a single picture, and an execution subject thereof may be a device having a processing capability: a server or a system or a device. As shown in FIG.
  • the method flow includes the following steps: step S310, obtaining a vehicle damage picture input by a user; step S320, extracting picture feature information based on a pre-trained damage recognition model, and determining damage recognition corresponding to the vehicle damage picture
  • the damage recognition result includes at least a damage frame
  • step S330 displaying the damage recognition result to the user, and receiving changes made by the user based on the damage recognition result
  • step S340 based on the pre-trained LSTM, the damage recognition result and picture feature information
  • user changes as input to get optimized damage identification results The following describes how the above steps are performed.
  • step S310 a vehicle damage picture input by a user is acquired.
  • the picture can be a picture of a car damage scene taken by an ordinary user, and a picture to be identified for damage.
  • Figure 4a shows an example of a vehicle damage picture. This picture is an unprocessed live picture taken by ordinary users.
  • step S320 based on the pre-trained damage recognition model, picture feature information is extracted, and a damage recognition result corresponding to the vehicle damage picture is determined, where the damage recognition result includes at least a damage frame.
  • the damage recognition model is pre-trained based on the following steps: first obtaining multiple historical vehicle damage pictures marked with the damage recognition result; and then based on a Convolutional Neural Network (CNN) Vehicle injury pictures are used as training samples to train injury recognition models.
  • the labeled damage recognition result includes a damage frame, that is, including a bounding box of a damaged object and a damage category, that is, a category of the damaged object in the frame, and accordingly, is identified based on the model.
  • the damage identification results include damage frames and damage categories.
  • the labeled damage recognition result may further include a damage segmentation result, for example, damage contour information or mask information, and accordingly, the damage recognition result identified by the model may further include a damage segmentation result.
  • the model first extracts picture feature information, and then generates a damage recognition result based on the picture feature information.
  • the picture feature information may include a feature map (feature map) generated based on CNN. Further, based on this feature map, feature information in a region of interest (ROI) may be collected. Furthermore, classification, classification, bounding, regression, and segmentation of the damage are performed, and then the damage recognition result is determined.
  • the picture feature information may also include information of other layers in the CNN network, for example, feature information of ROI, or feature information of a candidate damage frame.
  • the damage recognition result corresponding to the vehicle damage picture can be preliminarily determined.
  • step S330 the user is shown the damage recognition result, and receives a change made by the user based on the damage recognition result.
  • the user can confirm the result if the damage recognition result is recognized.
  • the result can be directly used as the corresponding vehicle damage picture in The final result of the damage identification phase of a single picture and ends the current process.
  • the damage recognition result when the user is not satisfied with the damage recognition result, the damage recognition result can be changed.
  • the damage identification result may include a damage frame, and corresponding changes may include changing the damage frame, such as deleting, adding, moving, reducing, and enlarging.
  • the damage recognition result further includes a damage category corresponding to the damage frame, and the corresponding modification may include changing the damage category.
  • FIG. 4b it includes the damage recognition result of the vehicle damage picture in FIG. 4a determined based on the damage recognition model, that is, three damage frames and corresponding damage categories. Further, assuming that the user thinks that the damage frame on the right rear door is actually a light reflection, not a deformation damage, the damage frame can be deleted, and the corresponding damage category is deleted accordingly.
  • FIG. 4c the figure shows the interface of the damage recognition result modified by the user. It can be seen that the user's modification includes deleting the damage frame whose damage category is moderately deformed on the right rear door.
  • step S340 based on the pre-trained LSTM, the damage recognition result, picture feature information, and user changes are used as inputs to obtain an optimized damage recognition result.
  • H 0 represents the preliminary damage recognition result determined based on the damage recognition model
  • X 1 represents the picture feature information.
  • the user has not modified it, so the output H 1 is still the preliminary damage recognition result.
  • X 2 represents the feature information of the picture and the changes made by the user based on the preliminary damage recognition result.
  • H 1 and X 2 are input into the pre-trained LSTM, and the optimized damage recognition result H 2 can be obtained.
  • the user can continue to receive the user's changes to the damage recognition results, and use this change and picture feature information as X n , and enter X n and the damage recognition result H n-1 based on this modification into the LSTM. To obtain the corresponding optimized recognition result H n .
  • the process of pre-training the LSTM is similar to the process of using the trained LSTM, the difference is that in the process of training the LSTM, a large number of vehicle damage samples marked with the damage recognition result need to be used, combined with interaction with the staff Data to train the LSTM. Therefore, for an introduction to the training process of the LSTM, please refer to the following introduction to its use process, which will not be repeated here.
  • Attention mechanism is a concept often used in natural language processing.
  • natural language processing is needed to understand the meaning of a word or sentence, contextual information is very critical and can help understand the exact meaning of a word or sentence.
  • the influence of different contexts on the words and phrases to be processed is not the same, the "attention" to be invested is different, and the position of the context that has the most influence on the current words is not fixed, because it may appear in The distance is also uncertain before or after the current word. Therefore, attention mechanisms are needed to solve such problems.
  • Attention mechanism can also be applied to the field of image processing. For example, this mechanism can be applied to learn and determine which areas in a picture are critical to identifying the current object, that is, areas that require more attention.
  • the attention mechanism Based on the characteristics of the LSTM that can process the data sequence and the attention mechanism, in one or more embodiments of this specification, referring to the idea of the attention mechanism, for the multiple damage frames identified by the damage recognition model, attention is paid to a certain damage frame and other damage. The similarity correlation between the frames, thereby excluding outliers and optimizing the results of damage detection.
  • the damage recognition result includes multiple damage frames
  • the user's change includes deletion of one of the damage frames.
  • the damage object in the deleted damage frame is most likely not a damage, such as a stain or the like. Damage caused by an accident. Therefore, it can be inferred that, in other damage frames, if there is a damage frame that is similar to the deleted damage frame, it also needs to be deleted, and a less similar damage frame is retained. Therefore, in a specific embodiment, the user's modification includes deleting the first damage frame in the damage recognition result.
  • optimizing the damage recognition result may include: first determining a first damage frame and a plurality of other damage frames.
  • determining the similarity between the damage frames may include: first determining a feature vector corresponding to the damage frame, and then using a dot product between the corresponding feature vectors as the similarity.
  • the damage recognition result includes the three damage boxes shown in FIG. 4b, and the user deletes “moderate deformation of the right rear door”, and accordingly, the damage category can be calculated as “right rear door” “Medium Deformation” damage frame (denoted as damage frame 1) and the damage category of "moderate deformation of the fender” (denoted as damage frame 2), and the damage category is "mild scratch on the right rear door”
  • the similarity between the damage box (denoted as damage box 3). Assuming that the calculated similarities are 0.8 and 0.1, respectively, and the preset threshold is 0.7, the damage class can be "moderate deformation of the fender".
  • the damage box 2 is deleted, and the optimized damage recognition result is shown in FIG. 4d.
  • the candidate damage frame generated by the damage recognition model in the process of generating the damage recognition result can be used to optimize the damage recognition based on the user's change.
  • the damage recognition model will generate candidate damage frames based on the picture feature information, and then determine at least a part of the damage frames from the candidate damage frames according to preset judgment conditions.
  • optimizing the damage recognition result may include: first obtaining a candidate damage frame generated by the damage recognition model based on picture feature information, including the damage frame in the damage result; and then based on the user's damage to the damage frame. Recognize the changes in the damage box and update the candidate damage box; then, for the updated candidate damage box, based on the similarity between each candidate damage box, determine the optimized damage box, and use the optimized damage box as Part of the optimized damage identification results.
  • determining the optimized damage box from the updated candidate damage box includes: first, determining any first candidate damage box and other candidate damages in the updated candidate damage box Multiple similarities of the frames; then, input the multiple similarities into a predetermined prediction model, and determine whether the first candidate damage frame is an abnormal frame according to the output result of the prediction model; then, the first candidate damage frame is not an abnormal frame In the case, the first candidate damage frame is used as a part of the optimized damage frame.
  • determining multiple similarities between any first candidate damage box and other candidate damage boxes in the updated candidate damage box may include: calculating a first corresponding to the first candidate damage box.
  • the feature vector is a dot product of a plurality of other feature vectors respectively corresponding to a plurality of other candidate damage frames, and a plurality of dot product results are determined as the multiple similarities.
  • the similarity between the candidate damage frames may also be determined based on other mathematical operations between the feature vectors of the candidate damage frames, such as finding a difference vector, finding a distance, and the like.
  • determining the feature vector corresponding to each candidate damage frame may include: extracting a feature vector corresponding to each candidate damage frame based on the feature map included in the feature information of the picture.
  • determining a feature vector corresponding to each candidate damage frame may include: obtaining pixel features corresponding to each candidate damage frame from the pixel features of the original vehicle damage picture, such as RGB pixel values, and then based on these pixels Feature extraction feature vector of each candidate damage area.
  • the prediction model may be pre-trained by positive samples and negative samples, and then the trained prediction model is directly used as a part of the LSTM.
  • the positive sample includes a plurality of damaged areas marked as real damage
  • the negative sample includes a plurality of damaged areas marked as real damage and at least one area marked as non-damaged.
  • the damaged area and the non-damaged area can be understood as the area surrounded by the corresponding labeled damage frame.
  • the prediction model may be a linear regression model.
  • the prediction model may be jointly trained with other parts in the LSTM, that is, the training process of the LSTM includes the determination of parameters in the prediction model.
  • the candidate damage frames generated by the damage recognition model include damage frame A, damage frame B, damage frame C, damage frame D, and damage frame E, where damage frame A, damage frame B, and damage frame C are damage Damage box in results.
  • the user's changes to the damage box include the deletion of the damage box B and the reduction of the damage box C to the damage box C ', thereby obtaining updated candidate damage boxes including: the damage box A, the damage box C', and the damage. Box D and damage box E.
  • the similarity between damage frame A and damage frame C ', damage frame D and damage frame E can be determined respectively, and these three similarities are input into the prediction model to determine whether the damage frame A is an abnormal frame.
  • the damage frame C ′, the damage frame D, and the damage frame E are abnormal frames, respectively. Assume that it is determined that the damage frame A, damage frame C ', and damage frame D are not abnormal frames, and the damage frame E is an abnormal frame. Therefore, the damage frame A, damage frame C', and damage frame D can be used as the optimized damage results. portion.
  • the attention mechanism is introduced on the basis of LSTM, which can further optimize the damage recognition result.
  • the method for optimizing the damage recognition result provided by the embodiment of the present specification, firstly extract the feature information of the picture based on a single fixed damage picture taken by the user, and initially identify the damage recognition result; and then receive the user's damage recognition result. Modify the data; then based on the pre-trained LSTM, the extracted image feature information, the initially identified damage recognition results, and the user modification data are used as inputs to update the damage recognition results. If the user is still dissatisfied with the updated damage identification result, he can modify it again, until the user is satisfied with the fixed damage result.
  • FIG. 6 shows a flowchart of a method for optimizing a damage recognition result according to an embodiment.
  • the execution subject of the method may be a device with a processing capability: a server or a system or a device, for example, a fixed loss client. As shown in FIG.
  • the method flow includes the following steps: Step S610, acquiring first picture characteristic information of a first vehicle damage picture and a first damage recognition result corresponding to the first vehicle damage picture, and obtaining a second vehicle The second picture feature information of the damage picture and the second damage recognition result corresponding to the second vehicle damage picture; step S620, based on the pre-trained long-term and short-term memory network LSTM, the first damage recognition result and the first picture feature information The second picture feature information and the second damage recognition result are used as inputs to obtain an optimized second damage recognition result. The following describes how to perform the above steps.
  • step S610 the first picture characteristic information of the first vehicle damage picture and the first damage recognition result corresponding to the first vehicle damage picture are obtained, and the second picture characteristic information and the second vehicle damage picture are obtained. A second damage recognition result corresponding to the second vehicle damage picture.
  • the first picture feature information of the first vehicle damage picture may be extracted based on a pre-trained damage recognition model, and the first damage recognition result may be determined, and the second picture feature information of the second vehicle damage picture may be extracted. And determine the second damage recognition result.
  • the damage recognition model, picture feature information, and damage recognition results refer to the foregoing embodiments, and details are not described herein.
  • the obtained first damage recognition result or the second damage recognition result may also be a damage recognition result optimized based on the method shown in FIG. 3.
  • the first damage recognition result is a damage recognition result optimized based on user interaction data
  • the second damage recognition result is a damage recognition result initially determined based on a damage recognition model. Therefore, the first damage recognition result can be combined.
  • the second damage recognition result is optimized, and the optimized second damage recognition result is displayed to the user.
  • step S620 based on the pre-trained long-term and short-term memory network LSTM, the first damage recognition result, the first picture feature information, the second picture feature information, and the second damage recognition result are used as inputs to obtain an optimized second damage. Identify the results.
  • the pre-trained LSTM involved here is different from the pre-trained LSTM mentioned in the previous step S340, and the pre-trained LSTM in the aforementioned step S340 is used to optimize a single fixed-loss image based on user interaction data
  • the LSTM in this step is used to optimize the fixed-loss results of the current picture according to the fixed-loss results of other pictures. It can be understood that these are two models that need to be trained separately, however, these two models can be used in a nested manner.
  • the first picture feature information and the first damage recognition result may be used as initial inputs, and the second picture feature information and the second damage recognition result may be used as new inputs at the current moment, and the The inputs are jointly input into a pre-trained LSTM to obtain an optimized second injury recognition result.
  • an attention mechanism can be combined on the basis of LSTM to better optimize the second damage recognition result.
  • the second damage result includes the first damage frame.
  • optimizing the second damage recognition result may include: first, determining a match with the first damage frame from the first vehicle picture based on the area matching positioning algorithm. Matching region; then, the category of the first damage frame is optimized according to the matching region.
  • the user finds that the damage to the headlights in FIG. 4d has not been identified, so a close-up picture of the headlights is added, as shown in FIG. 7a.
  • the second damage recognition result shown in FIG. 7 b can be determined, that is, the first damage frame and the corresponding damage category “vehicle light is broken”.
  • the classification information of the broken headlights is identified from the picture, but specifically which headlights are broken, such as the vehicle headlights or vehicle taillights have not been identified, but based on Figure 4d, it can be known that the right taillight.
  • optimizing the second damage recognition result may include: first, determining an area matching the first damage frame from the first vehicle picture based on an area matching positioning algorithm; and then obtaining component information of the matching area identified based on the picture feature information That is, the right taillight of the vehicle, the category of the first damage frame is optimized to "the right taillight of the vehicle is broken", and the optimized damage recognition result is displayed to the user, as shown in FIG. 7c.
  • step S620 the user can continue to take pictures of the vehicle damage, or selectively delete the captured pictures of the vehicle damage. Based on the user's operation of adding or deleting pictures, FIG. 6 can be used.
  • the optimization method updates and optimizes the damage identification results.
  • step S330 and step S340 shown in FIG. 3 can also be used to further optimize the second damage recognition result by receiving the modification of the second damage recognition result by the user until the user is satisfied with the second damage recognition result. .
  • step S620 it may include taking as input the optimized second damage recognition result and the second picture feature information, and the first damage recognition result and the first picture feature information to optimize the First damage recognition result. That is, after optimizing the second damage recognition result according to the first damage recognition result, then the optimized second damage recognition result can be used to optimize the first damage recognition result.
  • optimizing the second damage recognition result may include: first, determining a matching area that matches the first damage frame from the first vehicle picture based on the area matching positioning algorithm; and then performing a classification of the first damage frame based on the matching area. optimization.
  • optimizing the second damage recognition result may include optimizing at least one damage frame in the first damage recognition result according to the matching region.
  • the optimized second damage recognition result includes a damage frame of the type "the right taillight of the vehicle is broken.” Based on this, it can be matched with the damage frame in the first vehicle picture Area, the corresponding damage frame is identified, and the optimized first damage recognition result is shown in FIG. 7d.
  • FIG. 8 shows a structural diagram of an apparatus for optimizing a damage recognition result according to an embodiment.
  • the apparatus 800 includes:
  • An obtaining unit 810 configured to obtain a vehicle damage picture input by a user
  • An extraction unit 820 configured to extract picture feature information based on a pre-trained damage recognition model
  • a determining unit 830 configured to determine a damage recognition result corresponding to the vehicle damage picture, where the damage recognition result includes at least a damage frame;
  • a display unit 840 configured to display the damage recognition result to a user
  • a receiving unit 850 configured to receive a change made by the user based on the damage recognition result
  • An optimization unit 860 is configured to obtain the optimized damage recognition result by using the damage recognition result, the picture feature information, and the change as inputs based on a pre-trained long-short-term memory network LSTM.
  • the damage recognition model in the extraction unit is pre-trained based on the following steps:
  • the multiple historical vehicle damage pictures are used as training samples to train the damage recognition model.
  • the damage recognition result further includes a damage category corresponding to the damage box; the change includes changing the damage box, and / or, changing the damage category; wherein the changing the damage box includes deleting, adding, moving, At least one of reduction and enlargement.
  • the modification includes changing a damage box;
  • the optimization unit 860 specifically includes:
  • An obtaining subunit 861 configured to obtain a candidate damage frame generated by the damage recognition model to obtain a damage recognition result according to picture feature information, including the damage frame in the damage result;
  • An update subunit 862 configured to update the candidate damage box based on the change
  • a determination subunit 863 is used to determine an optimized damage frame for the updated candidate damage frame based on the similarity between each candidate damage frame, and use the optimized damage frame as a part of the optimized damage recognition result. .
  • the determining subunit 863 is specifically configured to:
  • the first candidate damage frame is used as a part of the optimized damage frame.
  • the determining subunit 863 is specifically configured to determine multiple similarities between any first candidate damage box and other candidate damage boxes of the updated candidate damage box, including:
  • the prediction model is pre-trained by a positive sample and a negative sample
  • the positive sample includes a plurality of injury regions labeled as real injuries
  • the negative sample includes a plurality of injuries labeled as real injuries. Area and at least one area marked as non-damaging.
  • the prediction model is a linear regression model.
  • the damage recognition result includes multiple damage frames, the multiple damage frames include a first damage frame, and the modification includes deleting the first damage frame;
  • the optimization The unit 860 is specifically configured to:
  • the damage frame corresponding to the similarity of the plurality of similarities below a preset threshold is used as a part of the optimized damage recognition result.
  • the device for optimizing the damage recognition result first extracts the feature information of the picture based on a single fixed damage picture taken by the user, and initially recognizes the damage recognition result; and then receives the user's damage recognition result. Modify the data; then based on the pre-trained LSTM, the extracted image feature information, the initially identified damage recognition results, and the user modification data are used as inputs to update the damage recognition results. If the user is still dissatisfied with the updated damage identification result, he can modify it again, until the user is satisfied with the fixed damage result.
  • FIG. 9 shows a structural diagram of an apparatus for optimizing a damage recognition result according to an embodiment.
  • the apparatus 900 includes:
  • An obtaining unit 910 is configured to obtain first picture characteristic information of a first vehicle damage picture and a first damage recognition result corresponding to the first vehicle damage picture, and obtain second picture characteristic information and a second vehicle damage picture. A second damage recognition result corresponding to the second vehicle damage picture;
  • a first optimization unit 920 configured to: based on a pre-trained long-short-term memory network LSTM, compare the first damage recognition result, the first picture feature information, the second picture feature information, and the second damage recognition result As an input, an optimized second damage recognition result is obtained.
  • the first picture feature information and the second picture feature information are separately extracted based on a pre-trained damage recognition model.
  • the first damage recognition result and the second damage recognition result are respectively determined based on a pre-trained damage recognition model, or are determined separately based on the method shown in FIG. 3.
  • the second damage identification result includes a first damage frame; the first optimization unit 920 is specifically configured to:
  • the category of the first damage frame is optimized according to the matching region.
  • the apparatus further includes:
  • a second optimization unit 930 is configured to take as input the optimized second damage recognition result and the second picture feature information, and the first damage recognition result and the first picture feature information to obtain an optimized first damage recognition result.
  • a damage identification result is configured to take as input the optimized second damage recognition result and the second picture feature information, and the first damage recognition result and the first picture feature information to obtain an optimized first damage recognition result.
  • the first optimization unit 920 is specifically configured to:
  • the second optimization unit 930 is specifically configured to optimize at least one damage frame in the first damage recognition result according to the matching region.
  • the device for optimizing the damage recognition result provided by the embodiment of the present specification can optimize the fixed result of the current picture in combination with the fixed results of other pictures.
  • a computer-readable storage medium is further provided, on which a computer program is stored, and when the computer program is executed in the computer, the computer is caused to execute the description described in conjunction with FIG. 3 or FIG. 6. method.
  • a computing device which includes a memory and a processor, where the executable code is stored in the memory, and when the processor executes the executable code, the combination with FIG. 3 or FIG. 6 is implemented. The method described.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Strategic Management (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • Development Economics (AREA)
  • General Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Economics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Quality & Reliability (AREA)
  • Human Resources & Organizations (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Technology Law (AREA)
  • Tourism & Hospitality (AREA)
  • Game Theory and Decision Science (AREA)
  • Computational Mathematics (AREA)
  • Pure & Applied Mathematics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Algebra (AREA)
  • Mathematical Optimization (AREA)
  • Operations Research (AREA)
  • Mathematical Analysis (AREA)

Abstract

本说明书实施例提供优化损伤识别结果的方法，一方面，该方法包括根据与用户进行交互的数据，对单张图片的损伤识别结果进行优化的方法，具体包括：首先，基于CNN算法识别出单张图片的初步损伤识别结果，展示给用户，并接收用户对初步损伤识别结果的修改，然后结合此修改，通过LSTM和Attention机制影响重新输出损伤识别结果，并再次展示给用户，直到用户满意；另一方面，该方法还包括基于其他图片的损伤识别结果，对当前图片的损伤识别结果进行优化的方法，具体包括：基于CNN算法识别出当前图片的初步损伤识别结果，然后结合其他图片的损伤识别结果，通过LSTM和Attention机制对当前图片的初步损伤识别结果进行优化。

Description

优化损伤识别结果的方法及装置 技术领域
本说明书实施例涉及车辆定损领域，具体地，涉及一种优化损伤识别结果的方法及装置。
背景技术
在传统车险理赔场景中，保险公司需要派出专业的查勘定损人员到事故现场进行现场查勘定损，给出车辆的维修方案和赔偿金额，并拍摄现场照片，定损照片留档以供后台核查人员核损核价。由于需要人工查勘定损，保险公司需要投入大量的人力成本，和专业知识的培训成本。从普通用户的体验来说，理赔流程由于等待人工查勘员现场拍照、定损员在维修地点定损、核损人员在后台核损，理赔周期长达1-3天，用户的等待时间较长，体验较差。
针对需求背景中提到的这一人工成本巨大的行业痛点，开始设想将人工智能和机器学习应用到车辆定损的场景中，希望能够利用人工智能领域计算机视觉图像识别技术，根据普通用户拍摄的现场损失图片，自动识别图片中反映的车损状况，并自动给出维修方案。如此，无需人工查勘定损核损，大大减少了保险公司的成本，提升了普通用户的车险理赔体验。
不过，目前的智能定损方案，损伤识别的准确度还有待进一步提高。因此，希望能有改进的方案，能够对车辆的损伤识别结果进行进一步优化，提高识别准确度。
发明内容
在说明书提供的优化损伤识别结果的方法中，基于循环神经网络，将前次损伤识别结果以及与用户的交互信息作为输入，从而实现损伤识别结果的优化。
根据第一方面，提供一种优化损伤识别结果的方法，应用于单张图片的处理。该方法包括：获取用户输入的车辆损伤图片；基于预先训练的损伤识别模型，提取图片特征信息，并确定与所述车辆损伤图片对应的损伤识别结果，所述损伤识别结果至少包括损伤框；向用户展示所述损伤识别结果，并接收所述用户基于所述损伤识别结果做出的更改；基于预先训练的长短期记忆网络LSTM，将所述损伤识别结果、所述图片特征信息 和所述更改作为输入，得到优化后的损伤识别结果。
在一个实施例中，所述损伤识别模型基于以下步骤预先训练：获取标注有损伤识别结果的多张历史车辆损伤图片；基于卷积神经网络CNN，将所述多张历史车辆损伤图片作为训练样本，训练所述损伤识别模型。
在一个实施例中，所述损伤识别结果还包括与损伤框对应的损伤类别；所述更改包括更改损伤框，和/或，更改损伤类别；其中所述更改损伤框包括删除、添加、移动、缩小和放大中的至少一种。
进一步地，一方面，在一个具体的实施例中，其中所述更改包括更改损伤框；所述得到优化后的损伤识别结果，包括：根据所述图片特征信息，获取所述损伤识别模型为获得损伤识别结果而生成的候选损伤框，其中包括所述损伤结果中的损伤框；基于所述更改，更新所述候选损伤框；对于更新后的候选损伤框，基于各个候选损伤框之间的相似度，确定出优化后的损伤框，并将优化后的损伤框作为优化后的损伤识别结果的一部分。
更进一步地，在一个具体的实施例中，其中所述确定出优化后的损伤框，包括：确定所述更新后的候选损伤框中任意的第一候选损伤框与其他候选损伤框的多个相似度；将所述多个相似度输入预定的预测模型，根据预测模型的输出结果，确定第一候选损伤框是否为异常框，所述预测模型包含在所述LSTM中；在所述第一候选损伤框不是异常框的情况下，将所述第一候选损伤框作为所述优化后的损伤框中的一部分。
更进一步地，在一个例子中，中所述确定所述更新后的候选损伤框中任意的第一候选损伤框与其他候选损伤框的多个相似度，包括：计算所述第一候选损伤框对应的第一特征向量，分别与多个其他候选损伤框对应的多个其他特征向量的点积，将多个点积结果确定为所述多个相似度。
在另一个例子中，其中所述预测模型通过正样本和负样本预先训练，所述正样本包括，多个标注为真实损伤的损伤区域，所述负样本包括，多个标注为真实损伤的损伤区域和至少一个标注为非损伤的区域。
在又一个例子中，其中所述预测模型为线性回归模型。
另一方面，在一个具体的实施中，所述损伤识别结果包括多个损伤框，所述多个损伤框包括第一损伤框，所述更改包括删除所述第一损伤框；所述优化所述损伤识别结果，包括：确定所述第一损伤框与多个其他损伤框的多个相似度；将所述多个相似度中低于 预设阈值的相似度所对应的损伤框，作为优化后的损伤识别结果的一部分。
根据第二方面，提供另一种优化损伤识别结果的方法，应用于多张图片的处理。该方法包括：获取第一车辆损伤图片的第一图片特征信息和与所述第一车辆损伤图片对应的第一损伤识别结果，以及获取第二车辆损伤图片的第二图片特征信息和与所述第二车辆损伤图片对应的第二损伤识别结果；基于预先训练的长短期记忆网络LSTM，将所述第一损伤识别结果、所述第一图片特征信息、所述第二图片特征信息和所述第二损伤识别结果作为输入，得到优化后的所述第二损伤识别结果。
在一个实施例中，其中所述第一图片特征信息和第二图片特征信息基于预先训练的损伤识别模型而分别提取。
在一个实施例中，其中所述第一损伤识别结果和第二损伤识别结果，基于预先训练的损伤识别模型而分别确定，或者，基于权利要求1所述的方法而分别确定。
在一个实施例中，其中所述第二损伤识别结果包括第一损伤框；所述得到优化后的第二损伤识别结果，包括：基于区域匹配定位算法，从所述第一车辆图片中确定与所述第一损伤框匹配的匹配区域；根据所述匹配区域对所述第一损伤框的类别进行优化。
在一个实施例中，所述方法还包括：将优化后的第二损伤识别结果和所述第二图片特征信息，以及所述第一损伤识别结果和所述第一图片特征信息作为输入，得到优化后的第一损伤识别结果。
进一步地，在一个具体的实施例中，其中所述第二损伤识别结果包括第一损伤框；所述得到优化后的第二损伤识别结果，包括：基于区域匹配定位算法，从所述第一车辆图片中确定与所述第一损伤框匹配的匹配区域；根据所述匹配区域对所述第一损伤框的类别进行优化；其中得到优化后的第一损伤识别结果，包括：根据所述匹配区域优化所述第一损伤识别结果中的至少一个损伤框。
根据第三方面，提供一种优化损伤识别结果的装置，应用于单张图片的处理。该装置包括：获取单元，用于获取用户输入的车辆损伤图片；提取单元，用于基于预先训练的损伤识别模型，提取图片特征信息；确定单元，用于确定与所述车辆损伤图片对应的损伤识别结果，所述损伤识别结果至少包括损伤框；展示单元，用于向用户展示所述损伤识别结果；接收单元，用于接收所述用户基于所述损伤识别结果做出的更改；优化单元，用于基于预先训练的长短期记忆网络LSTM，将所述损伤识别结果、所述图片特征信息和所述更改作为输入，得到优化后的所述损伤识别结果。
根据第四方面，提供一种优化损伤识别结果的装置，应用于多张图片的处理。该装置包括：获取单元，用于获取第一车辆损伤图片的第一图片特征信息和与所述第一车辆损伤图片对应的第一损伤识别结果，以及获取第二车辆损伤图片的第二图片特征信息和与所述第二车辆损伤图片对应的第二损伤识别结果；第一优化单元，用于基于预先训练的长短期记忆网络LSTM，将所述第一损伤识别结果、所述第一图片特征信息、所述第二图片特征信息和所述第二损伤识别结果作为输入，得到优化后的第二损伤识别结果。
根据第五方面，提供了一种计算机可读存储介质，其上存储有计算机程序，当所述计算机程序在计算机中执行时，令计算机执行第一方面或第二方面所描述的方法。
根据第六方面，提供了一种计算设备，包括存储器和处理器，其特征在于，所述存储器中存储有可执行代码，所述处理器执行所述可执行代码时，实现第一方面或第二方面所描述的方法。
在本说明书实施例披露的优化单张图片的损伤识别结果的方法中，首先基于用户拍摄的单张定损图片提取出图片特征信息，并初步识别出损伤识别结果；然后接收用户对损伤识别结果的修改数据；再基于预先训练的LSTM，将提取出的图片特征信息、初步识别出的损伤识别结果和用户修改数据作为输入，更新损伤识别结果。如果用户对更新后的损伤识别结果仍不满意，还可以再次修改，如此直到用户对定损结果满意为止。
附图说明
为了更清楚地说明本说明书披露的多个实施例的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本说明书披露的多个实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1示出根据一个实施例的LSTM中神经元的时序图；
图2为本说明书批露的一个实施例的实施场景示意图；
图3示出根据一个实施例的优化损伤识别结果的方法流程图；
图4a示出车辆损伤图片的一个示例；
图4b示出损伤识别模型识别出的损伤识别结果的例子；
图4c示出用户修改后的损伤识别结果；
图4d示出优化后的损伤识别结果；
图5示出根据一个实施例的LSTM在使用过程中的时序示意图；
图6示出根据一个实施例的优化损伤识别结果的方法流程图；
图7a示出车辆损伤图片的一个示例；
图7b示出损伤识别模型识别出的损伤识别结果的例子；
图7c示出优化后的损伤识别结果；
图7d示出优化后的损伤识别结果；
图8示出根据一个实施例的优化损伤识别结果的装置结构图；
图9示出根据一个实施例的优化损伤识别结果的装置结构图。
具体实施方式
下面结合附图，对本说明书披露的多个实施例进行描述。
本说明书实施例披露一种优化损伤识别结果的方法,下面首先介绍所述方法的发明构思。
为了对车辆定损图片中的损伤状况进行识别，业界普遍采用的方法是，通过与海量历史数据库进行比对得到相似的图片,来决定图片上的损伤部件及其程度。然而，这样的方式损伤识别准确率不够理想。此外，还采用样本标注的方式训练一些损伤识别模型，进行车辆损伤的识别。在进行损伤识别过程中，由于受到光照、污渍、拍摄角度、距离、车型等各种因素的影响，使得损伤识别准确率不够高，比如，在识别结果中很可能会出现真正的损伤能够正确检测，而同时少量反光或污渍也会被检测成损伤的情况，从而出现错误检测。
而且，损伤识别模型通常被用于定损客户端中，以使普通用户可以拍摄现场损失图片，并将现场损失图片上传至定损客户端中，从而实现自动定损。鉴于现有的损伤识别模型的准确率不够高，用户可能对损伤识别结果并不满意，在这种情况下，用户通常会补拍或更换部分照片，然后上传至定损客户端重新定损，客户端通常会将更新后的所有照片作为输入，重新定损，如此多次反复可能还是很难让用户满意，同时，还会造成较大的资源开销，以及消耗用户较多的时间。
为了快速得到令用户满意的损伤识别结果，发明人想到，可以通过与用户的交互的 方式，得到用户对损伤识别结果的反馈数据，并结合反馈数据更新损伤识别结果。
进一步地，考虑到长短期记忆网络LSTM可以用于处理时间序列信息，具体地，LSTM中的计算单元，例如神经元，可以记忆以前的信息，并将其作为之后的输入。具体地，参见图1中示出的LSTM中神经元的时序图，对于某个神经元，x t-1、x t、x t+1分别表示t-1时刻、t时刻和t+1时刻的输入，a 1、a 2、a 3分别表示t-1时刻、t时刻和t+1时刻神经元的状态，而h t-1、h t、h t+1分别表示t-1时刻、t时刻和t+1时刻的输出，其中：
a t＝g(U*x t-1+W*h t-1+b a)   (1)
h t＝f(V*a t-1+b h)   (2)
a t+1＝g(U*x t+W*h t+b a)   (3)
h t+1＝f(V*a t+b h)   (4)
从图1中可以看出，t时刻的输出h t是由上一时刻的输出h t-1和当前的输入x t共同决定的。因此，可以利用LSTM能够处理时序信息的特点，结合用户对损伤识别结果的修改反馈，实现对损伤识别结果的更新。如此，既利用了用户的反馈数据，还可以利用之前已确定的损伤数据，从而可以更加快速、准确地更新损伤识别结果，以使用户满意。
图2为本说明书批露的一个实施例的实施场景示意图。如图2所示，车辆损伤图片首先被输入到损伤识别模型进行损伤识别，损伤识别模型将提取图片的特征信息，同时，如果图片中包括多处损伤，例如，变形、刮擦等，通常识别模型会从图片中识别出多个候选损伤区域作为检测结果。根据本说明书的实施例，接收用户输入的修改，并基于修改数据，以及经过损伤识别模型提取的车辆损伤图片的图片特征信息，更新检测结果，直到用户满意。下面描述优化损伤识别结果的具体实现过程。
图3示出根据一个实施例的优化损伤识别结果的方法流程图，所述方法用于对单张图片进行处理，且其执行主体可以为具有处理能力的设备：服务器或者系统或者装置。如图3所示，该方法流程包括以下步骤：步骤S310，获取用户输入的车辆损伤图片；步骤S320，基于预先训练的损伤识别模型，提取图片特征信息，并确定与车辆损伤图片对应的损伤识别结果，其中损伤识别结果至少包括损伤框；步骤S330，向用户展示损伤识别结果，并接收用户基于损伤识别结果做出的更改；步骤S340，基于预先训练的LSTM，将损伤识别结果、图片特征信息和用户更改作为输入，得到优化后的损伤识别结果。下面描述以上各个步骤的执行方式。
首先，在步骤S310，获取用户输入的车辆损伤图片。可以理解，该图片可以是普通 用户拍摄的车损现场的图片，是有待进行损伤识别的图片。
图4a示出车辆损伤图片的一个示例。该图片是普通用户拍摄的未经处理的现场图片。
接着在步骤S320，基于预先训练的损伤识别模型，提取图片特征信息，并确定与车辆损伤图片对应的损伤识别结果，其中损伤识别结果至少包括损伤框。
在一个实施例中，其中损伤识别模型基于以下步骤预先训练：首先获取标注有损伤识别结果的多张历史车辆损伤图片；然后基于卷积神经网络(Convolutional Neural Network，简称CNN)，将多张历史车辆损伤图片作为训练样本，训练损伤识别模型。在一个具体的实施例中，其中标注的损伤识别结果包括损伤框，即，包括损伤对象的边框(bounding box)和损伤类别，即，边框中损伤对象的类别，相应地，基于该模型识别出来的损伤识别结果包括损伤框和损伤类别。进一步地，其标注的损伤识别结果还可以包括损伤分割结果，例如，损伤的轮廓信息或蒙层(mask)信息，相应地，该模型识别出来的损伤识别结果还可以包括损伤分割结果。
在一个实施例中，将车辆损伤图片输入损伤识别模型之后，该模型会先提取图片特征信息，然后再基于图片特征信息生成损伤识别结果。在一个具体的实施例中，其中图片特征信息可以包括基于CNN生成的特征图(feature map)，进一步地，基于此特征图可以采集感兴趣区域(region of interest，简称ROI)中的特征信息，进而进行损伤的分类(classification)、边框回归(bounding box regression)和分割(segmentation)，再确定出损伤识别结果。在另一个具体的实施例中，其中图片特征信息除了包括特征图，还可以包括CNN网络中其他层的信息，例如，ROI的特征信息，或者候选损伤框的特征信息。
以上，可以初步确定出与车辆损伤图片对应的损伤识别结果。接着，在步骤S330，向用户展示损伤识别结果，并接收用户基于损伤识别结果做出的更改。
需要说明的是，用户在查看初步确定的损伤识别结果之后，在认可该损伤识别结果的情况下，可以对此结果进行确认操作，响应于该确认操作可以将此结果直接作为对应车辆损伤图片在单张图片的损伤识别阶段的最终结果，并结束当前流程。
另一方面，当用户对该损伤识别结果不满意的情况下，可以对损伤识别结果进行更改。在一个实施例中，损伤识别结果中可以包括损伤框，相应的更改可以包括更改损伤框，如删除、添加、移动、缩小和放大等。进一步地，在一个具体的实施例中，损伤识别结果中还包括与损伤框对应的损伤类别，相应的更改可以包括更改损伤类别。
根据一个具体的例子，如图4b所示，其中包括基于损伤识别模型，确定出的图4a中车辆损伤图片的损伤识别结果，即三个损伤框和对应的损伤类别。进一步地，假定用户认为右后车门上的损伤框实际上是光反，并非是变形损伤，则可以将该损伤框进行删除，与其对应的损伤类别随之被删除。如图4c所示，图中示出用户修改后的损伤识别结果界面。由此可知，用户更改包括删除损伤类别为右后车门中度变形的损伤框。
以上，可以得到基于损伤识别模型提取的图片特征信息和基于图片特征信息确定的损伤识别结果，以及用户对损伤识别结果的更改数据。然后，在步骤S340，基于预先训练的LSTM，将损伤识别结果、图片特征信息和用户更改作为输入，得到优化后的损伤识别结果。
在一个实施例中，如图5所示，H 0表示基于损伤识别模型确定的初步损伤识别结果，X 1表示图片特征信息，此时用户没有修改，所以输出的H 1仍为初步损伤识别结果；进一步地，X 2表示图片特征信息和用户基于初步损伤识别结果进行的更改，将H 1和X 2输入预先训练的LSTM中，可以得到的优化后的损伤识别结果H 2。
更进一步地，后续可以继续接收用户对损伤识别结果的更改，并将此次更改和图片特征信息作为X n，以及将X n和此次修改所基于的损伤识别结果H n-1输入LSTM中，以得到对应的优化识别结果H n。
需要说明的是，预先训练LSTM的过程与使用训练好的LSTM的过程类似，区别在于在训练LSTM的过程中，需要使用大量的标注有损伤识别结果的车辆损伤样本，并结合与工作人员的交互数据，以对LSTM进行训练。因此，对LSTM的训练过程的介绍可以参见以下对其的使用过程的介绍，在此不作赘述。
此外，考虑到一次碰撞或者刮擦事故造成的车体表面的多处损伤往往具有相似的视觉特征，比如损伤的高度基本一致，痕迹趋于连贯，或者由于刮擦而附着上的颜色相同。同时，对于容易被误判为损伤的反光、污渍等，通常具有类似的视觉效果。根据这样的特点，提出在使用LSTM优化损伤识别结果的过程中引入Attention机制，即注意力机制，来进一步优化损伤检测的结果。
Attention机制是自然语言处理中常常用到的一个概念。在进行自然语言处理，需要理解一个词或一句话的意思的时候，上下文的信息非常关键，可以帮助理解一个词或一句话的准确意思。然而，不同位置的上下文对当前要处理的词句的影响作用并不相同，要投入的“注意力”也就不同，并且对当前词句最有影响的上下文的位置并不固定，因 为它可能出现在当前词句之前或之后，距离也不确定。因此就需要attention机制解决这样的问题。
Attention机制也可以被应用到图像处理领域中。例如，可以应用该机制去学习和确定，在一张图片中，哪些区域是对识别当前对象比较关键的区域，也就是需要投入更多注意力的区域。
基于LSTM可以处理数据序列和attention机制的特点，在本说明书的一个或多个实施例中，借鉴Attention机制的思路，对于损伤识别模型识别出的多个损伤框，关注某个损伤框与其他损伤框之间的相似度关联，由此排除异常区域(outlier)，优化损伤检测的结果。
在一个实施例中，损伤识别结果中包括多个损伤框，用户更改包括对其中某个损伤框的删除，被删除的损伤框中的损伤对象很可能不是损伤，如实为污渍等，或者不是此次事故造成的损伤。由此可以推想，在其他损伤框中，如果存在与被删除的损伤框较为相似的损伤框，也需要对其进行删除，并保留不太相似的损伤框。由此，在一个具体的实施例中，用户更改包括删除损伤识别结果中的第一损伤框，相应地，优化损伤识别结果可以包括：首先确定第一损伤框与多个其他损伤框的多个相似度，然后将多个相似度中低于预设阈值的相似度所对应的损伤框，作为优化后的损伤识别结果的一部分。进一步地，在一个具体的实施例中，确定损伤框之间的相似度可以包括，首先确定损伤框对应的特征向量，然后将对应的特征向量之间的点积作为相似度。
根据一个具体的例子，损伤识别结果包括图4b中示出的三个损伤框，用户对其中的“右后车门中度变形”进行了删除，据此，可以分别计算损伤类别为“右后车门中度变形”损伤框(记为损伤框1)与损伤类别为“翼子板中度变形”的损伤框(记为损伤框2)、以及与损伤类别为“右后车门轻度刮擦”的损伤框(记为损伤框3)之间的相似度，假定计算出的相似度分别为0.8和0.1，而预设阈值为0.7，则可以对损伤类别为“翼子板中度变形”的损伤框2进行删除，优化后的损伤识别结果如图4d所示。可以看到，损伤框2与用户删除的损伤框1之间存在较高的相似度(实际上两者都是光反)，在用户确认损伤框1不是真实损伤因而进行删除的情况下，经过损伤优化过程，与损伤框1相似的损伤框2也得到删除。
另一方面，在用户通过更多方式更改损伤框的情况下，可以利用损伤识别模型在生成损伤识别结果过程中产生的候选损伤框，基于用户的更改进行损伤识别优化。
一般来说，损伤识别模型为了识别损伤结果，在提取图片特征信息后，会基于图片特征信息生成候选损伤框，然后再根据预设的判别条件，从候选损伤框中确定出至少一部分损伤框，作为损伤结果中的损伤框。基于此，在用户更改损伤框的情况下，优化损伤识别结果可以包括：首先获取损伤识别模型基于图片特征信息生成的候选损伤框，其中包括所述损伤结果中的损伤框；接着基于用户对损伤识别结果中损伤框的更改，更新候选损伤框；然后，对于更新后的候选损伤框，基于各个候选损伤框之间的相似度，确定出优化后的损伤框，并将优化后的损伤框作为优化后的损伤识别结果的一部分。
进一步地，根据一个具体的实施例，其中从更新后的候选损伤框中确定出优化后的损伤框，包括：首先，确定更新后的候选损伤框中任意的第一候选损伤框与其他候选损伤框的多个相似度；接着，将多个相似度输入预定的预测模型，并根据预测模型的输出结果，确定第一候选损伤框是否为异常框；然后，在第一候选损伤框不是异常框的情况下，将第一候选损伤框作为优化后的损伤框中的一部分。
更进一步地，在一个具体的实施例中，确定更新后的候选损伤框中任意的第一候选损伤框与其他候选损伤框的多个相似度可以包括：计算第一候选损伤框对应的第一特征向量，分别与多个其他候选损伤框对应的多个其他特征向量的点积，将多个点积结果确定为所述多个相似度。在其他实施例中，还可以基于候选损伤框的特征向量之间的其他数学运算，例如求差值向量，求距离等，确定候选损伤框之间的相似度。
关于候选损伤框的特征向量的提取，在一个例子中，确定与各个候选损伤框对应的特征向量，可以包括：基于图片特征信息中包括的特征图，提取与各个候选损伤框对应的特征向量。在另一个例子中，其中确定与各个候选损伤框对应的特征向量，可以包括：从原始的车辆损伤图片的像素特征中获取各个候选损伤框对应的像素特征，例如RGB像素值，然后基于这些像素特征提取各个候选损伤区域的特征向量。
在一个具体的实施例中，其中预测模型可以通过正样本和负样本预先训练，然后直接将训练好的预测模型作为LSTM中的一部分。进一步地，其中正样本包括，多个标注为真实损伤的损伤区域，所述负样本包括，多个标注为真实损伤的损伤区域和至少一个标注为非损伤的区域。需要说明的是，其中的损伤区域和非损伤区域可以理解为对应的标注损伤框所围成的区域。在一个例子中，预测模型可以为线性回归模型。在另一个具体的实施例中，预测模型可以与LSTM中的其他部分联合协同训练，也就是说，LSTM的训练过程包括对预测模型中参数的确定。
根据一个具体的例子，由损伤识别模型生成的候选损伤框包括损伤框A、损伤框B、 损伤框C、损伤框D和损伤框E，其中损伤框A、损伤框B和损伤框C为损伤结果中的损伤框。同时，用户对损伤框的更改包括对损伤框B的删除，以及将损伤框C缩小为损伤框C’，由此可以得到更新后的候选损伤框包括：损伤框A、损伤框C’、损伤框D和损伤框E。然后，可以分别确定损伤框A与损伤框C’、损伤框D和损伤框E之间的相似度，并将这三个相似度输入预测模型中，以确定损伤框A是否为异常框，同理，还可以分别确定出损伤框C’、损伤框D和损伤框E是否为异常框。假定确定出损伤框A、损伤框C’和损伤框D不是异常框，而损伤框E为异常框，由此可以将损伤框A、损伤框C’和损伤框D作为优化后的损伤结果的一部分。
以上，在LSTM的基础上引入attention机制，可以实现对损伤识别结果的进一步优化。
综上可知，采用本说明书实施例提供的优化损伤识别结果的方法，首先基于用户拍摄的单张定损图片提取出图片特征信息，并初步识别出损伤识别结果；然后接收用户对损伤识别结果的修改数据；再基于预先训练的LSTM，将提取出的图片特征信息、初步识别出的损伤识别结果和用户修改数据作为输入，更新损伤识别结果。如果用户对更新后的损伤识别结果仍不满意，还可以再次修改，如此直到用户对定损结果满意为止。
以上，主要讨论的是针对单张定损图片的优化损伤识别结果的方法。考虑到定损过程中通常涉及到多张定损图片，除了采用前述方法对其中各张定损图片分别进行损伤识别及优化损伤识别结果之外，发明人认为，还可以考虑各张定损图片之间的关联信息，以实现对多张定损图片的损伤识别结果的共同优化。
同样地，考虑到LSTM可以处理时间序列信息，本说明书实施例还提供一种用于多张图片的优化损伤识别结果的方法。图6示出根据一个实施例的优化损伤识别结果的方法流程图，所述方法的执行主体可以为具有处理能力的设备：服务器或者系统或者装置，例如，定损客户端。如图6所示，该方法流程包括以下步骤：步骤S610，获取第一车辆损伤图片的第一图片特征信息和与所述第一车辆损伤图片对应的第一损伤识别结果，以及获取第二车辆损伤图片的第二图片特征信息和与所述第二车辆损伤图片对应的第二损伤识别结果；步骤S620，基于预先训练的长短期记忆网络LSTM，将第一损伤识别结果、第一图片特征信息、第二图片特征信息和第二损伤识别结果作为输入，得到优化后的第二损伤识别结果。下面介绍以上各步骤的执行方式。
首先，在步骤S610，获取第一车辆损伤图片的第一图片特征信息和与所述第一车辆损伤图片对应的第一损伤识别结果，以及获取第二车辆损伤图片的第二图片特征信息和 与所述第二车辆损伤图片对应的第二损伤识别结果。
在一个实施例中，可以基于预先训练的损伤识别模型，提取第一车辆损伤图片的第一图片特征信息，并确定第一损伤识别结果，以及提取第二车辆损伤图片的第二图片特征信息，并确定第二损伤识别结果。其中对于损伤识别模型、图片特征信息和损伤识别结果的介绍可以参见前述实施例，在此不作赘述。
进一步地，在一个具体的实施例中，其中获取的第一损伤识别结果或第二损伤识别结果还可以为基于图3示出的方法优化后的损伤识别结果。在一个例子中，第一损伤识别结果为基于用户交互数据优化后的损伤识别结果，而第二损伤识别结果为基于损伤识别模型初步确定的损伤识别结果，由此，可以结合第一损伤识别结果对第二损伤识别结果进行优化，再将优化后的第二损伤识别结果展示给用户。
接着，在步骤S620，基于预先训练的长短期记忆网络LSTM，将第一损伤识别结果、第一图片特征信息、第二图片特征信息和第二损伤识别结果作为输入，得到优化后的第二损伤识别结果。
需要说明的是，此处涉及的预先训练的LSTM与前述步骤S340中提及的预先训练的LSTM不相同，前述步骤S340中的预先训练的LSTM用于根据用户的交互数据优化单张定损图片的定损结果，而此步骤中的LSTM用于根据其他图片的定损结果来优化当前图片的定损结果。可以理解的是，这是两个需要分别训练的模型，但是，可以将这两个模型进行嵌套使用。
在一个实施例中，可以将第一图片特征信息和第一损伤识别结果作为初始输入，以及将第二图片特征信息和第二损伤识别结果作为当前时刻的新增输入，并将这两部分的输入共同输入到预先训练的LSTM中，以得到优化后的第二损伤识别结果。同样地，可以在LSTM的基础上结合attention机制，以使第二损伤识别结果得到更好的优化。
在一个实施例中，第二损伤结果中包括第一损伤框，相应地，优化第二损伤识别结果可以包括：首先，基于区域匹配定位算法，从第一车辆图片中确定与第一损伤框匹配的匹配区域；然后，根据匹配区域对第一损伤框的类别进行优化。
在一个具体的例子中，用户发现图4d中车灯的损伤没有被识别出来，因此增加了一张对车灯拍摄的近景图片，具体如图7a所示。此时，基于损伤识别模型可以确定出图7b中示出的第二损伤识别结果，即，第一损伤框和对应的损伤类别“车灯碎裂”。据此，从图片中识别出车灯碎裂的类别信息，但具体是哪个车灯发生碎裂，比如车辆前灯或车 辆尾灯没有被识别出来，但基于图4d可以得知，为车辆右侧尾灯。具体地，优化第二损伤识别结果可以包括：首先，基于区域匹配定位算法，从第一车辆图片中确定与第一损伤框匹配区域；然后，获取基于图片特征信息识别出的匹配区域的部件信息，即车辆右侧尾灯，将第一损伤框的类别优化为“车辆右侧尾灯碎裂”，并将优化后的损伤识别结果展示给用户，如图7c所示。
需要说明的是，在步骤S620之后，一方面，用户可以继续拍摄车辆损伤图片，或者对已拍摄的车辆损伤图片进行选择性删除，基于用户对图片的添加或删除操作，可以采用图6示出的优化方法对损伤识别结果进行更新优化。另一方面，还可以采用图3示出的步骤S330和步骤S340，通过接收用户对第二损伤识别结果的修改，对第二损伤识别结果进行进一步优化，直到用户对第二损伤识别结果满意为止。
此外，在步骤S620之后，可以包括将优化后的第二损伤识别结果和所述第二图片特征信息，以及所述第一损伤识别结果和所述第一图片特征信息作为输入，以优化所述第一损伤识别结果。也就是说，在根据第一损伤识别结果对第二损伤识别结果进行优化以后，接着可以利用优化后的第二损伤识别结果对第一损伤识别结果进行优化。
如前述实施例，优化第二损伤识别结果可以包括：首先基于区域匹配定位算法，从第一车辆图片中确定与第一损伤框匹配的匹配区域；然后根据匹配区域对第一损伤框的类别进行优化。
进一步地，优化第二损伤识别结果可以包括：根据所述匹配区域优化第一损伤识别结果中的至少一个损伤框。根据一个具体的例子，如图7c所示，优化后的第二损伤识别结果包括类别为“车辆右侧尾灯碎裂”的损伤框，据此可以在第一车辆图片中与该损伤框的匹配区域，标识出相应的损伤框，优化后的第一损伤识别结果如图7d所示。
综上可知，采用本说明书实施例提供的优化损伤识别结果的方法，可以结合其他图片的定损结果，对当前图片的定损结果进行优化。
根据再一方面的实施例，还提供一种优化装置，应用于单张图片的处理。图8示出根据一个实施例的优化损伤识别结果的装置结构图，该装置800包括：
获取单元810，用于获取用户输入的车辆损伤图片；
提取单元820，用于基于预先训练的损伤识别模型，提取图片特征信息；
确定单元830，用于确定与所述车辆损伤图片对应的损伤识别结果，所述损伤识别结果至少包括损伤框；
展示单元840，用于向用户展示所述损伤识别结果；
接收单元850，用于接收所述用户基于所述损伤识别结果做出的更改；
优化单元860，用于基于预先训练的长短期记忆网络LSTM，将所述损伤识别结果、所述图片特征信息和所述更改作为输入，得到优化后的损伤识别结果。
在一个实施例中，所述提取单元中的损伤识别模型基于以下步骤预先训练：
获取标注有损伤识别结果的多张历史车辆损伤图片；
基于卷积神经网络CNN，将所述多张历史车辆损伤图片作为训练样本，训练所述损伤识别模型。
在一个实施例中，所述损伤识别结果还包括与损伤框对应的损伤类别；所述更改包括更改损伤框，和/或，更改损伤类别；其中所述更改损伤框包括删除、添加、移动、缩小和放大中的至少一种。
进一步地，一方面，在一个具体的实施例中，其中所述更改包括更改损伤框；所述优化单元860具体包括：
获取子单元861，用于根据图片特征信息，获取所述损伤识别模型为获得损伤识别结果而生成的候选损伤框，其中包括所述损伤结果中的损伤框；
更新子单元862，用于基于所述更改，更新所述候选损伤框；
确定子单元863，用于对于更新后的候选损伤框，基于各个候选损伤框之间的相似度，确定出优化后的损伤框，并将优化后的损伤框作为优化后的损伤识别结果的一部分。
更进一步地，在一个例子中，其中所述确定子单元863具体用于：
确定所述更新后的候选损伤框中任意的第一候选损伤框与其他候选损伤框的多个相似度；
将所述多个相似度输入预定的预测模型，根据预测模型的输出结果，确定第一候选损伤框是否为异常框，所述预测模型包含在所述LSTM中；
在所述第一候选损伤框不是异常框的情况下，将所述第一候选损伤框作为所述优化后的损伤框中的一部分。
更进一步地，在一个例子中，其中所述确定子单元863具体用于确定所述更新 后的候选损伤框中任意的第一候选损伤框与其他候选损伤框的多个相似度，包括：
计算所述第一候选损伤框对应的第一特征向量，分别与多个其他候选损伤框对应的多个其他特征向量的点积，将多个点积结果确定为所述多个相似度。
在另一个例子中，其中所述预测模型通过正样本和负样本预先训练，所述正样本包括，多个标注为真实损伤的损伤区域，所述负样本包括，多个标注为真实损伤的损伤区域和至少一个标注为非损伤的区域。
在又一个例子中，其中所述预测模型为线性回归模型。
另一方面，在一个具体的实施例中，所述损伤识别结果包括多个损伤框，所述多个损伤框包括第一损伤框，所述更改包括删除所述第一损伤框；所述优化单元860具体用于：
确定所述第一损伤框与多个其他损伤框的多个相似度；
将所述多个相似度中低于预设阈值的相似度所对应的损伤框，作为优化后的损伤识别结果的一部分。
综上可知，采用本说明书实施例提供的优化损伤识别结果的装置，首先基于用户拍摄的单张定损图片提取出图片特征信息，并初步识别出损伤识别结果；然后接收用户对损伤识别结果的修改数据；再基于预先训练的LSTM，将提取出的图片特征信息、初步识别出的损伤识别结果和用户修改数据作为输入，更新损伤识别结果。如果用户对更新后的损伤识别结果仍不满意，还可以再次修改，如此直到用户对定损结果满意为止。
根据再一方面的实施例，还提供一种用于优化装置，应用于多张图片的处理。图9示出根据一个实施例的优化损伤识别结果的装置结构图，如图9所示，该装置900包括：
获取单元910，用于获取第一车辆损伤图片的第一图片特征信息和与所述第一车辆损伤图片对应的第一损伤识别结果，以及获取第二车辆损伤图片的第二图片特征信息和与所述第二车辆损伤图片对应的第二损伤识别结果；
第一优化单元920，用于基于预先训练的长短期记忆网络LSTM，将所述第一损伤识别结果、所述第一图片特征信息、所述第二图片特征信息和所述第二损伤识别结果作为输入，得到优化后的第二损伤识别结果。
在一个实施例中，其中所述第一图片特征信息和第二图片特征信息基于预先训 练的损伤识别模型而分别提取。
在一个实施例中，其中所述第一损伤识别结果和第二损伤识别结果，基于预先训练的损伤识别模型而分别确定，或者，基于图3中示出的方法而分别确定。
在一个实施例中，其中所述第二损伤识别结果包括第一损伤框；所述第一优化单元920具体用于：
基于区域匹配定位算法，从所述第一车辆图片中确定与所述第一损伤框匹配的匹配区域；
根据所述匹配区域对所述第一损伤框的类别进行优化。
在一个实施例中，该装置还包括：
第二优化单元930，用于将优化后的第二损伤识别结果和所述第二图片特征信息，以及所述第一损伤识别结果和所述第一图片特征信息作为输入，得到优化后的第一损伤识别结果。
进一步地，在一个具体的实施例中，其中所述第二损伤识别结果包括第一损伤框；所述第一优化单元920具体用于：
基于区域匹配定位算法，从所述第一车辆图片中确定与所述第一损伤框匹配的匹配区域；
根据所述匹配区域对所述第一损伤框的类别进行优化；
其中所述第二优化单元930具体用于：根据所述匹配区域优化所述第一损伤识别结果中的至少一个损伤框。
综上可知，采用本说明书实施例提供的优化损伤识别结果的装置，可以结合其他图片的定损结果，对当前图片的定损结果进行优化。
如上，根据又一方面的实施例，还提供一种计算机可读存储介质，其上存储有计算机程序，当所述计算机程序在计算机中执行时，令计算机执行结合图3或图6所描述的方法。
根据又一方面的实施例，还提供一种计算设备，包括存储器和处理器，所述存储器中存储有可执行代码，所述处理器执行所述可执行代码时，实现结合图3或图6所描述的方法。
本领域技术人员应该可以意识到，在上述一个或多个示例中，本说明书披露的多个实施例所描述的功能可以用硬件、软件、固件或它们的任意组合来实现。当使用软件实现时，可以将这些功能存储在计算机可读介质中或者作为计算机可读介质上的一个或多个指令或代码进行传输。
以上所述的具体实施方式，对本说明书披露的多个实施例的目的、技术方案和有益效果进行了进一步详细说明，所应理解的是，以上所述仅为本说明书披露的多个实施例的具体实施方式而已，并不用于限定本说明书披露的多个实施例的保护范围，凡在本说明书披露的多个实施例的技术方案的基础之上，所做的任何修改、等同替换、改进等，均应包括在本说明书披露的多个实施例的保护范围之内。

Claims (32)

  1. 一种优化损伤识别结果的方法，所述方法应用于单张图片的处理，所述方法包括：
    获取用户输入的车辆损伤图片；
    基于预先训练的损伤识别模型，提取图片特征信息，并确定与所述车辆损伤图片对应的损伤识别结果，所述损伤识别结果至少包括损伤框；
    向用户展示所述损伤识别结果，并接收所述用户基于所述损伤识别结果做出的更改；
    基于预先训练的长短期记忆网络LSTM，将所述损伤识别结果、所述图片特征信息和所述更改作为输入，得到优化后的损伤识别结果。
  2. 根据权利要求1所述的方法，其中，所述损伤识别模型基于以下步骤预先训练：
    获取标注有损伤识别结果的多张历史车辆损伤图片；
    基于卷积神经网络CNN，将所述多张历史车辆损伤图片作为训练样本，训练所述损伤识别模型。
  3. 根据权利要求1所述的方法，其中，所述损伤识别结果还包括与损伤框对应的损伤类别；所述更改包括更改损伤框，和/或，更改损伤类别；其中所述更改损伤框包括删除、添加、移动、缩小和放大中的至少一种。
  4. 根据权利要求3所述的方法，其中所述更改包括更改损伤框；所述得到优化后的损伤识别结果，包括：
    根据所述图片特征信息，获取所述损伤识别模型为获得损伤识别结果而生成的候选损伤框，其中包括所述损伤结果中的损伤框；
    基于所述更改，更新所述候选损伤框；
    对于更新后的候选损伤框，基于各个候选损伤框之间的相似度，确定出优化后的损伤框，并将优化后的损伤框作为优化后的损伤识别结果的一部分。
  5. 根据权利要求4所述的方法，其中所述确定出优化后的损伤框，包括：
    确定所述更新后的候选损伤框中任意的第一候选损伤框与其他候选损伤框的多个相似度；
    将所述多个相似度输入预定的预测模型，根据预测模型的输出结果，确定第一候选损伤框是否为异常框，所述预测模型包含在所述LSTM中；
    在所述第一候选损伤框不是异常框的情况下，将所述第一候选损伤框作为所述优化后的损伤框中的一部分。
  6. 根据权利要求5所述的方法，其中所述确定所述更新后的候选损伤框中任意的 第一候选损伤框与其他候选损伤框的多个相似度，包括：
    计算所述第一候选损伤框对应的第一特征向量，分别与多个其他候选损伤框对应的多个其他特征向量的点积，将多个点积结果确定为所述多个相似度。
  7. 根据权利要求5所述的方法，其中所述预测模型通过正样本和负样本预先训练，所述正样本包括，多个标注为真实损伤的损伤区域，所述负样本包括，多个标注为真实损伤的损伤区域和至少一个标注为非损伤的区域。
  8. 根据权利要求5所述的方法，其中所述预测模型为线性回归模型。
  9. 根据权利要求3所述的方法，其中，所述损伤识别结果包括多个损伤框，所述多个损伤框包括第一损伤框，所述更改包括删除所述第一损伤框；所述优化所述损伤识别结果，包括：
    确定所述第一损伤框与多个其他损伤框的多个相似度；
    将所述多个相似度中低于预设阈值的相似度所对应的损伤框，作为优化后的损伤识别结果的一部分。
  10. 一种优化损伤识别结果的方法，所述方法应用于多张图片的处理，所述方法包括：
    获取第一车辆损伤图片的第一图片特征信息和与所述第一车辆损伤图片对应的第一损伤识别结果，以及获取第二车辆损伤图片的第二图片特征信息和与所述第二车辆损伤图片对应的第二损伤识别结果；
    基于预先训练的长短期记忆网络LSTM，将所述第一损伤识别结果、所述第一图片特征信息、所述第二图片特征信息和所述第二损伤识别结果作为输入，得到优化后的所述第二损伤识别结果。
  11. 根据权利要求10所述的方法，其中所述第一图片特征信息和第二图片特征信息基于预先训练的损伤识别模型而分别提取。
  12. 根据权利要求10所述的方法，其中所述第一损伤识别结果和第二损伤识别结果，基于预先训练的损伤识别模型而分别确定，或者，基于权利要求1所述的方法而分别确定。
  13. 根据权利要求10所述的方法，其中所述第二损伤识别结果包括第一损伤框；所述得到优化后的第二损伤识别结果，包括：
    基于区域匹配定位算法，从所述第一车辆图片中确定与所述第一损伤框匹配的匹配区域；
    根据所述匹配区域对所述第一损伤框的类别进行优化。
  14. 根据权利要求10所述的方法，还包括：
    将优化后的第二损伤识别结果和所述第二图片特征信息，以及所述第一损伤识别结果和所述第一图片特征信息作为输入，得到优化后的第一损伤识别结果。
  15. 根据权利要求14所述的方法，其中所述第二损伤识别结果包括第一损伤框；所述得到优化后的第二损伤识别结果，包括：
    基于区域匹配定位算法，从所述第一车辆图片中确定与所述第一损伤框匹配的匹配区域；
    根据所述匹配区域对所述第一损伤框的类别进行优化；
    其中所述得到优化后的第一损伤识别结果，包括：根据所述匹配区域优化所述第一损伤识别结果中的至少一个损伤框。
  16. 一种优化损伤识别结果的装置，所述装置应用于单张图片的处理，所述装置包括：
    获取单元，用于获取用户输入的车辆损伤图片；
    提取单元，用于基于预先训练的损伤识别模型，提取图片特征信息；
    确定单元，用于确定与所述车辆损伤图片对应的损伤识别结果，所述损伤识别结果至少包括损伤框；
    展示单元，用于向用户展示所述损伤识别结果；
    接收单元，用于接收所述用户基于所述损伤识别结果做出的更改；
    优化单元，用于基于预先训练的长短期记忆网络LSTM，将所述损伤识别结果、所述图片特征信息和所述更改作为输入，得到优化后的损伤识别结果。
  17. 根据权利要求16所述的装置，其中，所述提取单元中的损伤识别模型基于以下步骤预先训练：
    获取标注有损伤识别结果的多张历史车辆损伤图片；
    基于卷积神经网络CNN，将所述多张历史车辆损伤图片作为训练样本，训练所述损伤识别模型。
  18. 根据权利要求16所述的装置，其中，所述损伤识别结果还包括与损伤框对应的损伤类别；所述更改包括更改损伤框，和/或，更改损伤类别；其中所述更改损伤框包括删除、添加、移动、缩小和放大中的至少一种。
  19. 根据权利要求18所述的装置，其中所述更改包括更改损伤框；所述优化单元具体包括：
    获取子单元，用于根据所述图片特征信息，获取所述损伤识别模型为获得损伤识别 结果而生成的候选损伤框，其中包括所述损伤结果中的损伤框；
    更新子单元，用于基于所述更改，更新所述候选损伤框；
    确定子单元，用于对于更新后的候选损伤框，基于各个候选损伤框之间的相似度，确定出优化后的损伤框，并将优化后的损伤框作为优化后的损伤识别结果的一部分。
  20. 根据权利要求19所述的装置，所述确定子单元具体用于：
    确定所述更新后的候选损伤框中任意的第一候选损伤框与其他候选损伤框的多个相似度；
    将所述多个相似度输入预定的预测模型，根据预测模型的输出结果，确定第一候选损伤框是否为异常框，所述预测模型包含在所述LSTM中；
    在所述第一候选损伤框不是异常框的情况下，将所述第一候选损伤框作为所述优化后的损伤框中的一部分。
  21. 根据权利要求20所述的装置，其中所述确定子单元具体用于确定所述更新后的候选损伤框中任意的第一候选损伤框与其他候选损伤框的多个相似度，包括：
    计算所述第一候选损伤框对应的第一特征向量，分别与多个其他候选损伤框对应的多个其他特征向量的点积，将多个点积结果确定为所述多个相似度。
  22. 根据权利要求21所述的装置，其中所述预测模型通过正样本和负样本预先训练，所述正样本包括，多个标注为真实损伤的损伤区域，所述负样本包括，多个标注为真实损伤的损伤区域和至少一个标注为非损伤的区域。
  23. 根据权利要求22所述的装置，其中所述预测模型为线性回归模型。
  24. 根据权利要求18所述的装置，所述损伤识别结果包括多个损伤框，所述多个损伤框包括第一损伤框，所述更改包括删除所述第一损伤框；所述优化单元具体用于：
    确定所述第一损伤框与多个其他损伤框的多个相似度；
    将所述多个相似度中低于预设阈值的相似度所对应的损伤框，作为优化后的损伤识别结果的一部分。
  25. 一种优化损伤识别结果的装置，所述装置应用于多张图片的处理，所述装置包括：
    获取单元，用于获取第一车辆损伤图片的第一图片特征信息和与所述第一车辆损伤图片对应的第一损伤识别结果，以及获取第二车辆损伤图片的第二图片特征信息和与所述第二车辆损伤图片对应的第二损伤识别结果；
    第一优化单元，用于基于预先训练的长短期记忆网络LSTM，将所述第一损伤识别结果、所述第一图片特征信息、所述第二图片特征信息和所述第二损伤识别结果作为输 入，得到优化后的所述第二损伤识别结果。
  26. 根据权利要求25所述的装置，其中所述第一图片特征信息和第二图片特征信息基于预先训练的损伤识别模型而分别提取。
  27. 根据权利要求25所述的装置，其中所述第一损伤识别结果和第二损伤识别结果，基于预先训练的损伤识别模型而分别确定，或者，基于权利要求1所述的方法而分别确定。
  28. 根据权利要求25所述的装置，其中所述第二损伤识别结果包括第一损伤框；所述第一优化单元具体用于：
    基于区域匹配定位算法，从所述第一车辆图片中确定与所述第一损伤框匹配的匹配区域；
    根据所述匹配区域对所述第一损伤框的类别进行优化。
  29. 根据权利要求25所述的装置，还包括：
    第二优化单元，用于将优化后的第二损伤识别结果和所述第二图片特征信息，以及所述第一损伤识别结果和所述第一图片特征信息作为输入，得到优化后的第一损伤识别结果。
  30. 根据权利要求29所述的装置，其中所述第二损伤识别结果包括第一损伤框；所述第一优化单元具体用于：
    基于区域匹配定位算法，从所述第一车辆图片中确定与所述第一损伤框匹配的匹配区域；
    根据所述匹配区域对所述第一损伤框的类别进行优化；
    其中所述第二优化单元具体用于：根据所述匹配区域优化所述第一损伤识别结果中的至少一个损伤框。
  31. 一种计算机可读存储介质，其上存储有计算机程序，当所述计算机程序在计算机中执行时，令计算机执行权利要求1-15中任一项的所述的方法。
  32. 一种计算设备，包括存储器和处理器，其特征在于，所述存储器中存储有可执行代码，所述处理器执行所述可执行代码时，实现权利要求1-15中任一项所述的方法。
PCT/CN2019/098545 2018-09-26 2019-07-31 优化损伤识别结果的方法及装置 WO2020063099A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
SG11202100124XA SG11202100124XA (en) 2018-09-26 2019-07-31 Damage identification result optimization method and apparatus
EP19864038.5A EP3859592A4 (en) 2018-09-26 2019-07-31 OPTIMIZATION METHOD AND DEVICE FOR DETECTION OF DAMAGE
US17/156,198 US11069052B2 (en) 2018-09-26 2021-01-22 Damage identification result optimization method and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811122031.9A CN110569700B (zh) 2018-09-26 2018-09-26 优化损伤识别结果的方法及装置
CN201811122031.9 2018-09-26

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/156,198 Continuation US11069052B2 (en) 2018-09-26 2021-01-22 Damage identification result optimization method and apparatus

Publications (1)

Publication Number Publication Date
WO2020063099A1 true WO2020063099A1 (zh) 2020-04-02

Family

ID=68772389

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/098545 WO2020063099A1 (zh) 2018-09-26 2019-07-31 优化损伤识别结果的方法及装置

Country Status (6)

Country Link
US (1) US11069052B2 (zh)
EP (1) EP3859592A4 (zh)
CN (1) CN110569700B (zh)
SG (1) SG11202100124XA (zh)
TW (1) TWI698804B (zh)
WO (1) WO2020063099A1 (zh)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110569864A (zh) * 2018-09-04 2019-12-13 阿里巴巴集团控股有限公司 基于gan网络的车损图像生成方法和装置
US11657373B2 (en) * 2020-08-21 2023-05-23 Accenture Global Solutions Limited System and method for identifying structural asset features and damage
CN113325011B (zh) * 2021-05-24 2024-03-29 华能秦煤瑞金发电有限责任公司 一种基于深度学习的混凝土结构损伤检测方法
US20240013364A1 (en) * 2021-06-29 2024-01-11 Beijing Baidu Netcom Science Technology Co., Ltd. Image-based vehicle damage assessment method, apparatus and storage medium
CN113361457A (zh) * 2021-06-29 2021-09-07 北京百度网讯科技有限公司 基于图像的车辆定损方法、装置及系统
CN113537167B (zh) * 2021-09-15 2021-12-03 成都数联云算科技有限公司 一种车辆外观识别方法及系统及装置及介质
US12002192B2 (en) * 2021-11-16 2024-06-04 Solera Holdings, Llc Transfer of damage markers from images to 3D vehicle models for damage assessment
CN115546767B (zh) * 2022-11-30 2023-04-07 广汽埃安新能源汽车股份有限公司 数据发送方法、装置、设备和计算机可读介质
CN116091457B (zh) * 2023-01-16 2023-10-27 北京航科星云科技有限公司 一种卫星太阳电池阵异常预测方法、装置及计算机设备
CN116168356B (zh) * 2023-04-26 2023-07-21 威海海洋职业学院 一种基于计算机视觉的车辆损伤判别方法
CN117197538B (zh) * 2023-08-16 2024-05-14 哈尔滨工业大学 一种基于高斯分布权重采样的贝叶斯卷积神经网络结构表观损伤识别方法
CN117437496B (zh) * 2023-12-21 2024-05-28 惠联云技术（深圳）有限公司 基于机器视觉的电脑一体机良率检测方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105550756A (zh) * 2015-12-08 2016-05-04 优易商业管理成都有限公司 一种基于模拟车辆受损的汽车快速定损方法
US20170293894A1 (en) * 2016-04-06 2017-10-12 American International Group, Inc. Automatic assessment of damage and repair costs in vehicles
CN107944831A (zh) * 2017-12-07 2018-04-20 平安科技（深圳）有限公司 汽车报损方法、应用服务器及计算机可读存储介质
CN107977895A (zh) * 2017-12-08 2018-05-01 北京小米移动软件有限公司 车辆损毁信息的确定方法、装置及用户设备

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7809587B2 (en) * 2004-05-07 2010-10-05 International Business Machines Corporation Rapid business support of insured property using image analysis
US20110218825A1 (en) * 2010-03-03 2011-09-08 International Business Machines Corporation Three-dimensional interactive vehicle damage claim interface
US20140309872A1 (en) * 2013-04-15 2014-10-16 Flextronics Ap, Llc Customization of vehicle user interfaces based on user intelligence
US10387960B2 (en) * 2012-05-24 2019-08-20 State Farm Mutual Automobile Insurance Company System and method for real-time accident documentation and claim submission
TWI536326B (zh) * 2012-07-20 2016-06-01 緯創資通股份有限公司 車輛碰撞事故通報系統及方法
US20170147990A1 (en) * 2015-11-23 2017-05-25 CSI Holdings I LLC Vehicle transactions using objective vehicle data
US10268671B2 (en) * 2015-12-31 2019-04-23 Google Llc Generating parse trees of text segments using neural networks
US20170262996A1 (en) * 2016-03-11 2017-09-14 Qualcomm Incorporated Action localization in sequential data with attention proposals from a recurrent network
CN107871134A (zh) * 2016-09-23 2018-04-03 北京眼神科技有限公司 一种人脸检测方法及装置
US10963782B2 (en) * 2016-11-04 2021-03-30 Salesforce.Com, Inc. Dynamic coattention network for question answering
US10255521B2 (en) * 2016-12-12 2019-04-09 Jack Cooper Logistics, LLC System, method, and apparatus for detection of damages on surfaces
CN108461080A (zh) * 2017-02-21 2018-08-28 中兴通讯股份有限公司 一种基于hlstm模型的声学建模方法和装置
CN107092922B (zh) * 2017-03-13 2018-08-31 平安科技（深圳）有限公司 车损识别方法及服务器
CN107403424B (zh) * 2017-04-11 2020-09-18 阿里巴巴集团控股有限公司 一种基于图像的车辆定损方法、装置及电子设备
CN107220648B (zh) * 2017-04-11 2018-06-22 平安科技（深圳）有限公司 理赔单据的字符识别方法及服务器
CN107358596B (zh) * 2017-04-11 2020-09-18 阿里巴巴集团控股有限公司 一种基于图像的车辆定损方法、装置、电子设备及系统
CN111914692B (zh) * 2017-04-28 2023-07-14 创新先进技术有限公司 车辆定损图像获取方法及装置
CN107665353A (zh) * 2017-09-15 2018-02-06 平安科技（深圳）有限公司 基于卷积神经网络的车型识别方法、装置、设备及计算机可读存储介质
CN107748893A (zh) * 2017-09-29 2018-03-02 阿里巴巴集团控股有限公司 提升车辆定损图像识别结果的方法、装置及服务器
CN108090838B (zh) * 2017-11-21 2020-09-29 阿里巴巴集团控股有限公司 识别车辆受损部件的方法、装置、服务器、客户端及系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105550756A (zh) * 2015-12-08 2016-05-04 优易商业管理成都有限公司 一种基于模拟车辆受损的汽车快速定损方法
US20170293894A1 (en) * 2016-04-06 2017-10-12 American International Group, Inc. Automatic assessment of damage and repair costs in vehicles
CN107944831A (zh) * 2017-12-07 2018-04-20 平安科技（深圳）有限公司 汽车报损方法、应用服务器及计算机可读存储介质
CN107977895A (zh) * 2017-12-08 2018-05-01 北京小米移动软件有限公司 车辆损毁信息的确定方法、装置及用户设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3859592A4 *

Also Published As

Publication number Publication date
EP3859592A1 (en) 2021-08-04
CN110569700B (zh) 2020-11-03
SG11202100124XA (en) 2021-02-25
EP3859592A4 (en) 2022-07-06
TW202013249A (zh) 2020-04-01
TWI698804B (zh) 2020-07-11
US11069052B2 (en) 2021-07-20
US20210174493A1 (en) 2021-06-10
CN110569700A (zh) 2019-12-13

Similar Documents

Publication Publication Date Title
WO2020063099A1 (zh) 优化损伤识别结果的方法及装置
US11176381B2 (en) Video object segmentation by reference-guided mask propagation
TWI742382B (zh) 透過電腦執行的、用於車輛零件識別的神經網路系統、透過神經網路系統進行車輛零件識別的方法、進行車輛零件識別的裝置和計算設備
US11144889B2 (en) Automatic assessment of damage and repair costs in vehicles
US10691982B2 (en) Method and apparatus for vehicle damage identification
TWI729405B (zh) 優化損傷檢測結果的方法及裝置
WO2020000390A1 (en) Systems and methods for depth estimation via affinity learned with convolutional spatial propagation networks
CN110264444B (zh) 基于弱分割的损伤检测方法及装置
TW202027007A (zh) 電腦執行的車輛定損方法及裝置
US20170039723A1 (en) Image Object Segmentation Using Examples
EP3852061B1 (en) Method and device for damage segmentation of vehicle damage image
WO2021057069A1 (zh) 计算机执行的车辆定损方法及装置
CN110751646A (zh) 利用车辆视频中的多个图像帧进行损伤识别的方法及装置
CN110738576B (zh) 为受损车辆生成定损文件的方法及装置
CN115008454A (zh) 一种基于多帧伪标签数据增强的机器人在线手眼标定方法
CN116433936A (zh) 图像检测方法、装置、电子设备及存储介质
CN113792569B (zh) 对象识别方法、装置、电子设备及可读介质
Dahlberg et al. Tag recognition from panoramic scans of industrial facilities
CN114495231A (zh) 人脸活体检测方法、人脸活体检测模型的训练方法及设备
CN117935128A (zh) 一种物品检测识别方法、装置、电子设备及存储介质
KR20230152961A (ko) 무인점포에 비치된 상품의 도난을 방지하는 전자장치 및 이의 동작방법
CN114998953A (zh) 人脸关键点检测方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19864038

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019864038

Country of ref document: EP

Effective date: 20210426