WO2020228370A1 - 计算机执行的从图片中识别损伤的方法及装置 - Google Patents

计算机执行的从图片中识别损伤的方法及装置 Download PDF

Info

Publication number
WO2020228370A1
WO2020228370A1 PCT/CN2020/071675 CN2020071675W WO2020228370A1 WO 2020228370 A1 WO2020228370 A1 WO 2020228370A1 CN 2020071675 W CN2020071675 W CN 2020071675W WO 2020228370 A1 WO2020228370 A1 WO 2020228370A1
Authority
WO
WIPO (PCT)
Prior art keywords
damage
frame
loss
height
width
Prior art date
Application number
PCT/CN2020/071675
Other languages
English (en)
French (fr)
Inventor
王萌
Original Assignee
创新先进技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 创新先进技术有限公司 filed Critical 创新先进技术有限公司
Priority to US16/813,355 priority Critical patent/US10885625B2/en
Publication of WO2020228370A1 publication Critical patent/WO2020228370A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8854—Grading and classifying of flaws
    • G01N2021/8861—Determining coordinates of flaws
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8854—Grading and classifying of flaws
    • G01N2021/8874—Taking dimensions of defect into account
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8854—Grading and classifying of flaws
    • G01N2021/888—Marking defects
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8883—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges involving the calculation of gauges, generating models
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08—Detecting or categorising vehicles

Definitions

  • One or more embodiments of this specification relate to the field of machine learning, and more particularly to methods and devices for identifying damage from pictures using machine learning.
  • the insurance company needs to send professional damage assessment personnel to the accident site to conduct on-site investigation and assessment of the damage, give the vehicle's maintenance plan and compensation amount, and take photos of the scene, and keep the damage assessment photos on file for use Backstage inspectors verify damage and price.
  • Due to the need for manual damage inspection insurance companies need to invest a lot of labor costs and professional knowledge training costs.
  • the claim settlement process waits for the manual surveyor to take photos on site, the damage assessor assesses the damage at the repair site, and the damage inspector checks the damage in the background.
  • the claim settlement cycle is as long as 1-3 days, and users have a long waiting time. , The experience is poor.
  • One or more embodiments of this specification describe a method and device for training a damage recognition model to identify damage from a picture.
  • the training of the damage recognition model is optimized for the scene of damage recognition, which is more conducive to the application of damage recognition in whole image analysis. .
  • a computer-executed method for training a damage detection model including:
  • the marked picture includes at least one damage marking frame for selecting damaged objects
  • Determining the position loss item related to the position deviation in the loss function of this prediction includes determining whether there is a damage labeling frame in the at least one damage labeling frame, which completely contains the first damage prediction area; if there is a damage labeling frame that completely contains For the first damage prediction area, the location loss item is determined to be zero; if there is no damage labeling frame that completely contains the first damage prediction area, the target damage labeling frame is determined from the at least one damage labeling frame, at least based on The distance between the center of the first damage prediction area and the center of the target damage marking frame determines the position loss item;
  • the damage detection model is updated so that the loss function decreases after the update.
  • the first damage prediction area is specifically a pixel corresponding to the first damage prediction point.
  • judging whether there is a damage labeling frame that completely includes the first damage prediction area may specifically be to determine whether the coordinates of the first damage prediction point fall within the coordinate range of each damage labeling frame;
  • the target damage marking frame can be determined by the following method: determining the distance between the first damage prediction point and the center of each damage marking frame, and using the damage marking frame corresponding to the shortest distance as the target damage marking frame; then, The position loss term is determined based on the shortest distance.
  • the first damage prediction area is specifically a first damage prediction frame, which has a first center, a first width, and a first height.
  • the target damage marking frame can be determined as follows:
  • For each damage marking frame determine the intersection area of the first damage prediction frame and the damage marking frame;
  • the target damage marking frame is determined from the at least one damage marking frame.
  • the above-mentioned first width is a preset width
  • the first height is a preset height; in this case, the center of the marking frame may be based on the first center and the target damage The distance between determines the position loss term.
  • the above-mentioned first width is the predicted width
  • the first height is the predicted height
  • the position loss term can be determined as follows:
  • the location loss term is determined based on the sum of the first loss term and the second loss term.
  • the second loss term may be determined as the arithmetic sum of the predicted width and predicted height.
  • the target damage labeling frame has a labeling width and a labeling height; accordingly, the second loss item can be determined as follows:
  • the width loss item based on the length of the predicted width exceeding the label width
  • determining the height loss item based on the length of the predicted height exceeding the marked height
  • the sum of the width loss term and the height loss term is used as the second loss term.
  • a device for training a damage detection model including:
  • the acquiring unit is configured to acquire a marked picture, the marked picture including at least one damage marking frame for selecting a damaged object;
  • a prediction unit configured to use a damage detection model to predict at least one damage prediction area in the labeled picture, including a first damage prediction area
  • the determining unit is configured to determine the position loss item related to the position deviation in the loss function of this prediction, and the determining unit includes:
  • a judging subunit configured to judge whether there is a damage labeling frame in the at least one damage labeling frame, which completely includes the first damage prediction area
  • the first determining subunit is configured to determine the location loss item as zero if there is a damage marking frame that completely contains the first damage prediction area;
  • the second determining subunit is configured to, if there is no damage labeling frame that completely contains the first damage prediction area, determine the target damage labeling frame from the at least one damage labeling frame, at least based on the first damage prediction area The distance between the center and the center of the target damage marking frame determines the position loss item;
  • the update unit is configured to update the damage detection model according to the loss function, so that the loss function decreases after the update.
  • a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method of the first aspect.
  • a computing device including a memory and a processor, characterized in that executable code is stored in the memory, and when the processor executes the executable code, the method of the first aspect is implemented .
  • an improved method of training damage detection model is proposed.
  • the goal of training is to make the damage prediction box fit the center position of the labeling box, but the size of the labeling box is not accurate. Fitting, but to make the prediction box not exceed the label box as much as possible, even the smaller the better.
  • the damage detection model obtained in this way is more suitable for the unique characteristics of damage recognition, and the damage detection results obtained thereby are more conducive to subsequent fusion with other detection results for overall image analysis.
  • Figure 1 shows a sample of annotated pictures for annotating a picture of a vehicle damage in an example
  • Figure 2 shows a flowchart of training a damage detection model according to an embodiment
  • FIG. 3 shows a schematic diagram of determining a target damage marking frame in an embodiment
  • Figure 4 shows the damage recognition result according to an embodiment
  • Fig. 5 shows a schematic block diagram of an apparatus for training a damage detection model according to an embodiment.
  • a basic and typical task in image recognition is target detection, that is, to identify specific target objects, such as human faces, animals, etc., from pictures, and to classify the target objects.
  • the labeled picture samples can be used for model training, and the trained target detection model can detect and recognize specific target objects in unknown pictures.
  • the target detection model outputs a target detection frame and a prediction category, where the target detection frame is the smallest rectangular frame for selecting the target object, and the prediction category is the predicted category for the target object selected by the target detection frame.
  • the damaged object can be labeled as a specific target object, and the damage detection model can be obtained by training based on the labeled picture samples thus obtained.
  • the damage detection model is a specific application of the target detection model, in which the damage is the object of target detection.
  • the training process of the damage detection model is similar to the conventional target detection model.
  • the annotator uses the form of a labeling frame to label the damaged object in the picture, and then a training picture containing several damage labeling frames is obtained.
  • a preliminary damage detection model is used to perform damage recognition on the training picture, and several damage prediction frames are obtained.
  • the loss function is used to measure the deviation between the predicted damage prediction box and the damage labeling box marked by the staff.
  • the deviation includes the deviation of the prediction category, the deviation of the prediction box and the center of the labeling box, and the deviation of the size.
  • the damage detection model is updated based on such a loss function, so that the damage detection model is adjusted in the direction where the deviation is reduced.
  • the direction and goal of training is to make the damage prediction box fully fit the damage labeling box.
  • the loss function is small enough and the damage prediction box is enough to fit the damage labeling box, it is considered that the model training is completed and the unknown picture can be predicted.
  • the inventor found through research and analysis that in the scene of damage recognition, the damage as the object to be recognized has its own unique characteristics. Accordingly, the above model training process can be further optimized to be adapted to the target. Scene characteristics of damage identification. The following describes with examples of vehicle damage.
  • Fig. 1 shows a sample of annotated pictures that annotated a picture of a vehicle damage in an example.
  • vehicle damage has a certain degree of ambiguity, continuity and self-containment, which is not the same as the conventional well-defined goals.
  • a continuous scratch can be regarded as a damage object, and a part of the scratch can also be regarded as a damage object (so-called self-containment).
  • This makes the labelers have a certain degree of arbitrariness when labeling.
  • damage recognition is often a part of image analysis in the corresponding scene, and needs to be fused with other detection results for subsequent analysis.
  • the final concern is what type of damage the component receives. Therefore, the result of damage identification needs to be combined with the result of component identification to obtain the damage status of the component.
  • a type of damage often corresponds to a replacement plan, so the damage status of the component only needs to be expressed as (component name, damage type), and it is not necessary to accurately know the size of the damaged area. For example, for (left front door, scratching), regardless of the size of the scratched area, the entire left front door needs to be painted, so it is not sensitive to the size of the damaged area.
  • Medical image analysis also has similar characteristics. More attention is paid to which organs and types of lesions/anomalies are reflected in the image map, and in many cases it is not sensitive to the precise size of the anomaly area.
  • the damage labeling frame itself is not very accurate, and on the other hand, it is not sensitive to the predicted damage size in most cases. Therefore, the inventor proposes that in the training process of the damage recognition model It is not necessary to accurately fit the size of the damage labeling frame. Moreover, the damage detection model obtained by accurately fitting the damage labeling frame will often identify a damage prediction frame similar in size to the damage labeling frame. Since such a damage prediction frame is not limited in size, it often cross-components, similar to the figure The label box on the far right in 1 spans the two parts of the front door and the rear door. This is not conducive to subsequent integration with the component recognition results.
  • an improved method of training damage detection model is proposed.
  • the goal of training is to make the damage prediction box fit the center position of the labeling box, but the size of the labeling box It does not perform accurate fitting, but makes the prediction box not exceed the label box as much as possible, and even the smaller the better.
  • Figure 2 shows a flowchart of training a damage detection model according to one embodiment.
  • the method can be executed by any device, device, platform or device cluster with computing and processing capabilities.
  • the method includes at least the following steps: Step 210: Obtain annotated pictures, where the annotated pictures include at least one damage labeling frame for selecting the damaged object; Step 220, use the damage detection model to mark the image At least one damage prediction area is obtained in the middle prediction, including the first damage prediction area; step 230, according to whether the first damage prediction area completely falls into at least one damage label frame, determine the loss function of this prediction related to the position deviation Position loss term; step 240, update the damage detection model according to the loss function, so that the loss function decreases after the update.
  • Step 210 Obtain annotated pictures, where the annotated pictures include at least one damage labeling frame for selecting the damaged object
  • Step 220 use the damage detection model to mark the image
  • At least one damage prediction area is obtained in the middle prediction, including the first damage prediction area
  • step 230 according
  • an annotated picture is obtained.
  • annotated pictures are images that are manually annotated by annotators and used for model training.
  • labeling the picture includes selecting the damage labeling frame of the damaged object, and there may be one or more damage labeling frames.
  • Figure 1 is the damage labeling picture with the damage object marked, which contains 6 damage labeling boxes.
  • each damage marking frame can be expressed in the form of (X, Y, W, H), where X and Y are the abscissa and ordinate of the center position of the damage marking frame, and W is the damage marking frame H is the height of the damage marking frame.
  • the damage marking frame can also be represented by the coordinates of its four vertices. And, it can be understood that the vertex representation and the (X, Y, W, H) representation can be easily converted to each other.
  • the damage detection model is used to predict the above-mentioned annotated pictures to obtain at least one damage prediction area.
  • the damage detection model in step 220 is a model in the training process, and it may be an initial model including initial parameters, or an intermediate model that has undergone several parameter adjustments.
  • the damage detection model is an embodiment of the target detection model.
  • various target detection models have been proposed based on various network structures and various detection algorithms.
  • a one-stage (one-stage) detection model can directly determine the category probability and position coordinates of the target object from the picture, that is, directly identify the target object.
  • Typical examples of single-stage inspection models include SSD model, Yolo model, etc.
  • the two-stage (two-stage) detection model first generates a candidate region, or ROI, in the picture, and then performs target recognition and border regression in the candidate region.
  • Typical examples of two-stage detection models include R-CNN model, Fast R-CNN model, Faster R-CNN model, etc.
  • Other target detection models are also proposed. These specific model structures and algorithms can all be used as damage detection models.
  • the prediction result can be obtained, that is, a number of damage prediction areas and corresponding predicted damage categories.
  • the loss function will include category loss items related to category deviations and location loss items related to location deviations.
  • the category loss item can be determined in a conventional manner, for example, comparing the similarities and differences between the predicted category of the damage prediction area and the label category of the corresponding damage labeling frame, and determining the category loss item according to the comparison result.
  • the loss is determined according to whether the damage prediction area completely falls into the damage labeling frame.
  • the position loss term in the function is determined according to whether the damage prediction area completely falls into the damage labeling frame.
  • step 230 the position loss item related to the position deviation in the loss function of this prediction is determined.
  • the description will be made in conjunction with any damage detection area, which is hereinafter referred to as the first damage prediction area.
  • the step 230 of determining the location loss item includes, step 231, judging whether there is a damage labeling frame that completely contains the first damage prediction area; if there is any damage labeling frame that completely contains the first damage prediction
  • the position loss item corresponding to the first damage prediction area is determined to be zero; if each damage labeling frame does not completely contain the first damage prediction area, then in step 233, from at least one damage labeling frame
  • the target damage marking frame is determined, and in step 234, the position loss item is determined based on at least the distance between the center of the first damage prediction area and the center of the target damage marking frame.
  • the obtained damage prediction area can be expressed as (x, y), which degenerates into a damage prediction point, or it is considered that the damage prediction area is the pixel corresponding to the damage prediction point.
  • any of the aforementioned first damage prediction areas are specifically pixels corresponding to the first damage prediction points, which can be expressed as (x 1 , y 1 ).
  • step 231 it is only necessary to determine whether the coordinates (x 1 , y 1 ) of the first damage prediction point fall within the coordinate range of each damage labeling frame.
  • each damage labeling box can be expressed in the form of four vertices, or the coordinates of the four vertices can be obtained by transforming (X, Y, W, H).
  • the coordinates of the four vertices can define the coordinate range of the damage marking frame. Therefore, in this step 231, (x 1 , y 1 ) may be sequentially compared with the coordinate ranges of each damage labeling frame to determine whether the first damage prediction point falls into a certain damage labeling frame.
  • step 232 the position loss item corresponding to the first damage prediction point is determined to be zero.
  • a target damage labeling frame is determined from at least one damage labeling frame.
  • the damage may be calculated first predicted point (x 1, y 1) and the distance between the center of each box marked damage (X i, Y i), the shortest distance corresponding to the target block denoted injury injury callout box. Assume that the center of the target damage marking frame is (X 1 , Y 1 ).
  • step 234 the position loss term is determined based on the distance between the first damage prediction point (x 1 , y 1 ) and the center (X 1 , Y 1 ) of the target damage marking frame, that is, the aforementioned shortest distance.
  • the position loss term L can be determined as the shortest distance mentioned above, namely:
  • the position loss term L can be determined as the square of the above shortest distance, namely:
  • the width w 0 and height h 0 of the damage prediction area are fixed, and the center coordinates (x, y) of the damage area are predicted.
  • the obtained damage prediction area appears as a fixed-size rectangular damage
  • the prediction frame, the damage prediction frame can be expressed as (x, y, w 0 , h 0 ). Since it is ultimately desired to obtain a smaller damage prediction frame, the fixed width and height can be set to smaller values, for example, w 0 and h 0 are both 4 pixels, and so on.
  • any of the above-mentioned first damage prediction areas is specifically the first damage prediction frame, expressed as (x 1 , y 1 , w 0 , h 0 ), which has a first center (x 1 , y 1 ), and is fixed Width w 0 and fixed height h 0 .
  • step 231 it is necessary to determine whether the first damage prediction frame (x 1 , y 1 , w 0 , h 0 ) completely falls into each damage labeling frame.
  • the coordinates of the four vertices of the first damage prediction frame can be determined according to its center coordinates (x 1 , y 1 ), width w 0 and height h 0 , for example:
  • the coordinates of the four vertices can be obtained in advance, and then the coordinate range can be obtained.
  • step 232 the position loss item corresponding to the first damage prediction box is determined to be zero.
  • step 233 from each damage labeling frame Determine the target damage marking frame as the comparison target of the first damage prediction frame to calculate the deviation loss. This step can be performed in multiple ways.
  • Fig. 3 shows a schematic diagram of determining a target damage marking frame in an embodiment.
  • two damage labeling boxes 31 and 32 are schematically shown with solid lines; a damage prediction box 33 is also shown with a dotted box.
  • the damage marking frame 31 may be expressed as (X 1 , Y 1 , W 1 , H 1 ), and the damage marking frame 32 may be expressed as (X 2 , Y 2 , W 2 , H 2 ).
  • the damage prediction frame 33 as the first damage prediction frame to be analyzed as an example, the process of determining the target damage marking frame is described.
  • the distance between the center (x 1 , y 1 ) of the first damage prediction frame and the center of each damage labeling frame is calculated, and the damage labeling frame corresponding to the shortest distance is used as the target damage labeling frame.
  • the distance between the center (x 1 , y 1 ) and (X 1 , Y 1 ) of the damage prediction frame 33 and the distance from (X 2 , Y 2 ) can be calculated separately. Assuming that the distance to (X 1 , Y 1 ) is smaller, the damage labeling frame 31 can be selected as the target damage labeling frame.
  • the intersection area of the first damage prediction frame and the damage labeling frame can be determined.
  • the intersection area S1 (shown in gray) between the damage prediction frame 33 and the damage labeling frame 31 and the intersection area S2 (shown by the diagonal line) between the damage prediction frame 33 and the damage labeling frame 32 can be calculated separately.
  • a target damage marking frame is determined.
  • the damage marking frame corresponding to the largest intersection area may be used as the target damage marking frame.
  • the damage marking frame 32 may be used as the target damage marking frame.
  • the intersection ratio (IoU) of the damage prediction frame and the damage labeling frame is obtained according to the intersection area, that is, the ratio of the intersection area to the combined area, and the damage labeling frame with the largest intersection IoU is selected as the target damage labeling frame.
  • the damage labeling frame 31 may be used as the target damage labeling frame.
  • the position loss item is determined based on the distance between the center of the first damage prediction frame and the center of the target damage labeling frame.
  • the width and height of the first damage prediction frame are preset fixed values, it is not necessary to consider the loss in the size of the prediction frame, but only the loss in the center position.
  • the aforementioned formula (1) or formula (2) can still be used to calculate the position loss term, where (x 1 , y 1 ) is the center coordinate of the first damage prediction frame, (X 1 , Y 1 ) is the center coordinate of the target damage marking frame.
  • the center position and size of the damage area are predicted to obtain a rectangular damage prediction frame, which can be expressed as (x, y, w, h), where w is the predicted width, h Is the predicted height.
  • the resulting damage prediction frame varies in size, and will continue to adjust and change as the model training process.
  • any of the first damage prediction areas is specifically the first damage prediction frame, expressed as (x 1 , y 1 , w 1 , h 1 ), which has a first center (x 1 , y 1 ), A width w 1 and a first height h 1 .
  • step 231 it is determined whether the first damage prediction frame (x 1 , y 1 , w 1 , h 1 ) completely falls into each damage labeling frame. Specifically, according to the first center, first width, and first height of the first damage prediction frame, determine its four vertices coordinates; then for each damage labeling frame, determine in turn whether the four vertices coordinates fall into the damage label Within the coordinate range of the box.
  • the specific calculation and execution process are similar to the foregoing second embodiment, and will not be repeated here.
  • step 232 the position loss item corresponding to the first damage prediction box is determined to be zero.
  • a target damage labeling frame is determined from each damage labeling frame as a comparison target for calculating the deviation loss of the first damage prediction frame.
  • the target damage labeling frame can be determined according to the center distance, the intersection area, and the intersection ratio between the first damage prediction frame and each damage labeling frame. The specific calculation and execution process are similar to the foregoing second embodiment, and will not be repeated here. Mark the target damage marking box as (X 1 , Y 1 , W 1 , H 1 ).
  • the first damage prediction frame (x 1 , y 1 , w 1 , h 1 ) is compared with the target damage marking frame (X 1 , Y 1 , W 1 , H 1 ) , To determine the location loss item.
  • the position loss item should consider not only the loss caused by the deviation of the prediction frame center, but also the loss in terms of frame size. Therefore, the first loss item, that is, the loss item in terms of center deviation, can be determined based on the distance between the first center and the center of the target damage labeling frame. In an example, the first loss term can be calculated using the above formula (1) or (2).
  • determine the second loss item that is, the loss item in terms of the frame size; then, determine the location loss item based on the sum of the first loss item and the second loss item.
  • the above-mentioned second loss term may be determined as the arithmetic sum of the width w 1 and the predicted height h 1 of the first damage prediction frame, such as the sum of squares.
  • the entire position loss term can be expressed as:
  • the second loss term in terms of the frame size is further decomposed into a width loss term and a height loss term.
  • For the width loss term compare the predicted width w 1 of the first damage prediction frame with the label width W 1 of the target damage label frame. If the predicted width is not greater than the label width, the width loss term is determined to be zero; if the predicted width is greater than For label width, the width loss item is determined based on the length of the predicted width exceeding the label width. Similar processing is performed on the height: if the predicted height h 1 is not greater than the label height H 1 , the height loss term is determined to be zero; if the predicted height is greater than the label height, the height loss term is determined based on the length of the predicted height exceeding the label height. Finally, the sum of the width loss term and the height loss term is used as the second loss term in terms of the frame size.
  • the location loss term can be expressed as:
  • loss items in the loss function can be determined in a conventional manner.
  • the loss item is determined for each damage prediction area, and based on the sum of the loss items of each damage prediction area, the loss function corresponding to this prediction can be obtained.
  • step 240 the damage detection model is updated according to the aforementioned loss function, so that the loss function decreases after the update.
  • the model can be updated by back propagation, gradient descent, etc.
  • the direction and goal of the damage detection model update is to make the loss function as small as possible. Therefore, the definition of the loss function, such as the different determination methods of the position loss term, determines the training direction of the damage detection model.
  • the method for determining the location loss item in step 230 of FIG. 2 will make the predicted damage area be included in the damage labeling frame as much as possible, instead of completely fitting the size of the labeling frame.
  • the resulting damage detection model will predict different damage prediction areas.
  • the resulting damage detection model can only predict the damage center point, and make the predicted damage center point fall into the damage labeling frame, or as close as possible to the damage label The center of the box.
  • the resulting damage detection model will predict the damage frame.
  • the damage detection model will be adjusted and optimized to make the loss prediction frame width w And the height h is as small as possible.
  • model parameters are continuously adjusted and optimized in the direction of loss function reduction, and finally an optimized and updated damage detection model can be obtained.
  • the updated damage detection model can be used to identify damage to the image to be tested.
  • Figure 4 shows the damage recognition result according to one embodiment.
  • the picture to be tested is a picture of vehicle damage.
  • the damage detection model trained by the above method is used to identify the damage of the vehicle damage picture, a series of damage prediction frames shown in the figure can be obtained. It can be seen that, unlike the prediction frame that fully fits the size of the label frame in the conventional technology, the damage prediction frame in Figure 4 is very small, but it still accurately indicates the location of the damage. Such a damage prediction frame has a very low probability of crossing components, which is very beneficial for subsequent fusion with component detection results, which is more conducive to the comprehensive damage analysis of the entire map.
  • an apparatus for training a damage detection model is provided.
  • the apparatus can be deployed in any device, platform or device cluster with computing and processing capabilities.
  • Fig. 5 shows a schematic block diagram of a model training device according to an embodiment. As shown in FIG. 5, the device 500 includes:
  • the acquiring unit 51 is configured to acquire a marked picture, the marked picture including at least one damage marking frame for selecting a damaged object;
  • the prediction unit 52 is configured to use a damage detection model to predict at least one damage prediction area in the labeled picture, including the first damage prediction area;
  • the determining unit 53 is configured to determine the position loss item related to the position deviation in the loss function predicted this time, and the determining unit 53 further includes:
  • the judging subunit 531 is configured to judge whether the first damage prediction area is completely contained in each damage labeling frame;
  • the first determining subunit 532 is configured to determine the position loss item as zero in the case that any damage marking frame completely includes the first damage prediction area;
  • the second determining subunit 533 is configured to determine a target damage labeling frame from the at least one damage labeling frame in the case that each damage labeling frame cannot completely include the first damage prediction area, based at least on the first damage labeling frame.
  • the distance between the center of a damage prediction area and the center of the target damage marking frame determines the position loss item;
  • the updating unit 54 is configured to update the damage detection model according to the loss function, so that the loss function decreases after the update.
  • the first damage prediction area is specifically a pixel corresponding to the first damage prediction point.
  • the determining subunit 531 is configured to: determine whether the coordinates of the first damage prediction point fall within the coordinate range of each damage marking frame; the second determining subunit 533 is configured to: determine For the distance between the first damage prediction point and the center of each damage marking frame, the damage marking frame corresponding to the shortest distance is used as the target damage marking frame; and the location loss item is determined based on the shortest distance.
  • the first damage prediction area is specifically a first damage prediction frame, which has a first center, a first width, and a first height.
  • the judging subunit 531 is configured to determine the coordinates of the four vertices of the first damage prediction frame according to the first center, the first width and the first height; Damage marking frame, and sequentially determine whether the coordinates of the four vertices all fall within the coordinate range of the damage marking frame.
  • the second determining subunit 533 is configured to: for each damage marking frame, determine the intersection area between the first damage prediction frame and the damage marking frame; according to the intersection area, The target damage marking frame is determined from the at least one damage marking frame.
  • the first width is a preset width
  • the first height is a preset height
  • the second determining subunit 533 may be configured to: based on the first center and the The distance between the centers of the target damage marking frame determines the position loss item.
  • the foregoing first width is a predicted width
  • the first height is a predicted height
  • the second determining subunit 533 may further include (not shown):
  • a first damage determination module configured to determine a first loss item based on the distance between the first center and the center of the target damage marking frame
  • a second loss determining module configured to determine a second loss item based on the predicted width and predicted height
  • the third determining module is configured to determine the location loss item based on the sum of the first loss item and the second loss item.
  • the second loss determining module is configured to determine the second loss item as the arithmetic sum of the predicted width and the predicted height.
  • the target damage label frame has a label width and a label height; in this case, the second loss determination module is configured as:
  • the width loss item based on the length of the predicted width exceeding the label width
  • determining the height loss item based on the length of the predicted height exceeding the marked height
  • the sum of the width loss term and the height loss term is used as the second loss term.
  • a damage detection model more suitable for identification of damaged objects can be obtained through training.
  • the embodiment of this specification also provides a device for identifying damage from the picture.
  • the device may include an acquisition unit for acquiring the damage detection model trained as described above, and an identification unit configured to use the above damage detection model to perform damage recognition on the picture to be tested.
  • a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method described in conjunction with FIG. 2.
  • a computing device including a memory and a processor, the memory is stored with executable code, and when the processor executes the executable code, the implementation described in conjunction with FIG. 2 method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Image Analysis (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

本说明书实施例提供一种计算机执行的损伤识别方法和装置，方法包括，获取标注图片，其中包括若干损伤标注框；然后利用损伤检测模型，在标注图片中预测得到损伤预测区；接着，确定本次预测的损失函数中与位置偏差相关的位置损失项，这具体包括，如果存在损伤标注框完全包含该损伤预测区，则将位置损失项确定为零；如果不存在，则确定出目标损伤标注框，并基于损伤预测区的中心和目标损伤标注框的中心之间的距离确定位置损失项。之后，可以根据损失函数，更新损伤检测模型，并利用更新后的损伤检测模型进行损伤识别。

Description

计算机执行的从图片中识别损伤的方法及装置 技术领域
本说明书一个或多个实施例涉及机器学习领域，尤其涉及利用机器学习从图片中识别损伤的方法和装置。
背景技术
随着机器学习的快速发展，各种人工智能技术已经应用于多种场景，帮助人们解决相应场景下的技术问题。其中，计算机视觉图像识别技术在多种领域多种场景下，都有强烈的应用需求，例如，应用于医疗影像分析，车损智能识别，等等。
例如，在传统车险理赔场景中，保险公司需要派出专业的查勘定损人员到事故现场进行现场查勘定损，给出车辆的维修方案和赔偿金额，并拍摄现场照片，定损照片留档以供后台核查人员核损核价。由于需要人工查勘定损，保险公司需要投入大量的人力成本，和专业知识的培训成本。从普通用户的体验来说，理赔流程由于等待人工查勘员现场拍照、定损员在维修地点定损、核损人员在后台核损，理赔周期长达1-3天，用户的等待时间较长，体验较差。针对这样的行业痛点，希望能够利用图像识别技术，根据普通用户拍摄的现场损失图片，自动识别图片中反映的车损状况，并自动给出维修方案。如此，无需人工查勘定损核损，大大减少保险公司的成本，提升普通用户的车险理赔体验。
在医疗影像分析中，也希望能够借助于图像识别技术，基于医疗影像智能地给出图像特点的分析，帮助医师进行诊断。
以上场景中，都需要从图片(车损图片或医疗图像)中识别出损伤对象。然而，目前的图像识别方案中，对于损伤对象识别的准确度以及可用性还有待进一步提高。因此，希望能有改进的方案，能够更精准地从图片中识别出损伤，从而优化整图分析结果。
发明内容
本说明书一个或多个实施例描述了训练损伤识别模型从而从图片中识别损伤的方法和装置，其中针对损伤识别的场景优化损伤识别模型的训练，更有利于损伤识别在整图分析中的应用。
根据第一方面，提供了一种计算机执行的训练损伤检测模型的方法，包括：
获取标注图片，所述标注图片包括框选出损伤对象的至少一个损伤标注框；
利用损伤检测模型，在所述标注图片中预测得到至少一个损伤预测区，其中包括第一损伤预测区；
确定本次预测的损失函数中与位置偏差相关的位置损失项，包括，判断所述至少一个损伤标注框中是否存在损伤标注框完全包含所述第一损伤预测区；如果存在损伤标注框完全包含该第一损伤预测区，将所述位置损失项确定为零；如果不存在损伤标注框完全包含该第一损伤预测区，从所述至少一个损伤标注框中确定出目标损伤标注框，至少基于所述第一损伤预测区的中心和所述目标损伤标注框的中心之间的距离确定所述位置损失项；
根据所述损失函数，更新所述损伤检测模型，以使得更新后所述损失函数下降。
在一种实施方式中，第一损伤预测区具体为第一损伤预测点对应的像素。
在这样的情况下，判断是否存在损伤标注框完全包含所述第一损伤预测区具体可以为，判断所述第一损伤预测点的坐标是否落入各个损伤标注框的坐标范围之内；
并且可以通过以下方式确定出目标损伤标注框：确定所述第一损伤预测点与各个损伤标注框的中心之间的距离，将最短距离对应的损伤标注框作为所述目标损伤标注框；然后，基于所述最短距离确定所述位置损失项。
在另一种实施方式中，第一损伤预测区具体为第一损伤预测框，具有第一中心，第一宽度和第一高度。
在这样的情况下，可以如下判断第一损伤预测区是否完全包含在各个损伤标注框中：
根据所述第一中心，第一宽度和第一高度，确定所述第一损伤预测框的四个顶点坐标；
对于各个损伤标注框，依次判断所述四个顶点坐标是否均落入该损伤标注框的坐标范围之内。
在该实施方式的一个示例中，可以如下确定出目标损伤标注框：
对于各个损伤标注框，确定所述第一损伤预测框与该损伤标注框的相交面积；
根据所述相交面积，从所述至少一个损伤标注框中确定出所述目标损伤标注框。
在该实施方式下的一个实施例中，上述第一宽度为预设宽度，第一高度为预设高度；在这样的情况下，可以基于所述第一中心和所述目标损伤标注框的中心之间的距离确定所述位置损失项。
在该实施方式下的另一实施例中，上述第一宽度为预测宽度，第一高度为预测高度；在这样的情况下，可以如下确定位置损失项：
基于所述第一中心和所述目标损伤标注框的中心之间的距离确定第一损失项；
基于所述预测宽度和预测高度，确定第二损失项；
基于所述第一损失项和第二损失项的加和，确定所述位置损失项。
在一个更具体的示例中，可以将所述第二损失项确定为，所述预测宽度和预测高度的运算和。
在另一具体示例中，目标损伤标注框具有标注宽度和标注高度；相应的，可以如下确定第二损失项：
在所述预测宽度不大于所述标注宽度的情况下，将宽度损失项确定为零；
在所述预测宽度大于所述标注宽度的情况下，基于所述预测宽度超出所述标注宽度的长度，确定所述宽度损失项；
在所述预测高度不大于所述标注高度的情况下，将高度损失项确定为零；
在所述预测高度大于所述标注高度的情况下，基于所述预测高度超出所述标注高度的长度，确定所述高度损失项；
将所述宽度损失项和高度损失项的加和，作为所述第二损失项。
根据第二方面，提供一种训练损伤检测模型的装置，包括：
获取单元，配置为获取标注图片，所述标注图片包括框选出损伤对象的至少一个损伤标注框；
预测单元，配置为利用损伤检测模型，在所述标注图片中预测得到至少一个损伤预测区，其中包括第一损伤预测区；
确定单元，配置为确定本次预测的损失函数中与位置偏差相关的位置损失项，所述确定单元包括，
判断子单元，配置为判断所述至少一个损伤标注框中是否存在损伤标注框完全包含 所述第一损伤预测区；
第一确定子单元，配置为，如果存在损伤标注框完全包含该第一损伤预测区，将所述位置损失项确定为零；
第二确定子单元，配置为，如果不存在损伤标注框完全包含该第一损伤预测区，从所述至少一个损伤标注框中确定出目标损伤标注框，至少基于所述第一损伤预测区的中心和所述目标损伤标注框的中心之间的距离确定所述位置损失项；
更新单元，配置为根据所述损失函数，更新所述损伤检测模型，以使得更新后所述损失函数下降。
根据第三方面，提供了一种计算机可读存储介质，其上存储有计算机程序，当所述计算机程序在计算机中执行时，令计算机执行第一方面的方法。
根据第四方面，提供了一种计算设备，包括存储器和处理器，其特征在于，所述存储器中存储有可执行代码，所述处理器执行所述可执行代码时，实现第一方面的方法。
根据本说明书实施例提供的方法和装置，提出一种改进的训练损伤检测模型的方式，训练的目标是使得，损伤预测框拟合标注框的中心位置，但对标注框的大小并不进行精确拟合，而是使得预测框尽量不超过标注框，甚至越小越好。如此得到的损伤检测模型更适合于损伤识别独有的特点，由此得到的损伤检测结果更有利于后续与其他检测结果进行融合进行整图分析。
附图说明
为了更清楚地说明本发明实施例的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1示出在一个例子中对车辆损伤图片进行标注的标注图片样本；
图2示出根据一个实施例的训练损伤检测模型的流程图；
图3示出在一个实施例中确定目标损伤标注框的示意图；
图4示出根据一个实施例的损伤识别结果；
图5示出根据一个实施例的训练损伤检测模型的装置的示意性框图。
具体实施方式
下面结合附图，对本说明书提供的方案进行描述。
如前所述，在多种场景中，需要利用计算机视觉技术进行图像识别。图像识别中一项基本而典型的任务是目标检测，也就是，从图片中识别出特定的目标对象，例如人脸，动物等，并对目标对象进行分类。为此，可以利用标注的图片样本进行模型训练，训练得到的目标检测模型可以对未知图片中的特定目标对象进行检测和识别。具体而言，目标检测模型会输出目标检测框和预测类别，其中目标检测框是框选出目标对象的最小矩形框，预测类别为针对目标检测框所框选的目标对象预测的类别。
在车辆智能定损、医疗影像分析等多种场景中，需要从图片中识别出损伤对象，例如车辆损伤，器官病变损伤。为此，可以将损伤对象作为特定的目标对象进行标注，基于如此得到的标注图片样本进行训练，即可得到损伤检测模型。换而言之，损伤检测模型是目标检测模型的一种具体化应用，其中将损伤作为目标检测的对象。
相应的，一般来说，损伤检测模型的训练过程与常规的目标检测模型相类似。具体地，由标注人员采用标注框的形式对图片中的损伤对象进行标注，于是得到包含若干损伤标注框的训练图片。然后，利用初步的损伤检测模型对该训练图片进行损伤识别，得到若干损伤预测框。通过损失函数衡量预测得到的损伤预测框与工作人员标注的损伤标注框之间的偏差。一般地，该偏差包含预测类别的偏差，预测框与标注框中心的偏差以及大小的偏差。基于这样的损失函数更新损伤检测模型，使得损伤检测模型向上述偏差减小的方向调整。换而言之，训练的方向和目标是，使得损伤预测框充分地拟合损伤标注框。当损失函数足够小，损伤预测框足够拟合损伤标注框时，认为模型训练完成，可以进行未知图片的预测。
然而，发明人经过研究和分析发现，在损伤识别这一场景中，作为待识别对象的损伤有其独有的特点，相应的，以上的模型训练过程可以进行进一步优化，以针对性的适应于损伤识别的场景特点。下面结合车辆损伤的例子进行描述。
图1示出在一个例子中对车辆损伤图片进行标注的标注图片样本。从图1中可以看到，车辆损伤具有一定的模糊性、连续性和自包含性，与常规的界限分明的目标并不相同。例如，对于车辆上的刮擦损伤来说，一条连续的刮擦划痕可以认为是一个损伤对象，同时该划痕的一部分也可以认为是一个损伤对象(即所谓的自包含性)。这使得标注人员在进行标注时也具有一定的随意性。
此外，损伤识别往往是对应场景中图片分析的一环，需要与其他检测结果进行融合用于后续分析。例如，对于车辆定损来说，最终关注的是什么部件受到什么类型的损伤，因此，损伤识别的结果需要与部件识别的结果进行结合，以得到部件的受损状况。并且，对于特定部件来说，一种损伤类型往往对应一种换修方案，因此部件受损状况只需要表达为(部件名，损伤类型)即可，并不必须精确获知损伤区域的大小。例如，对于(左前门，刮擦)来说，无论刮擦区域大小，都需要对整个左前门进行喷漆处理，因此对于损伤区域大小并不敏感。医疗影像分析中也具有类似的特点，其中更加关注的是，影像图中反映了什么器官具有什么类型的病变/异样，在许多情况下对于异样区域的精确大小并不敏感。
也就是说，由于损伤对象的特点，一方面损伤标注框本身并不十分精确，另一方面在多数情况下对于预测的损伤大小并不敏感，因此发明人提出，在损伤识别模型的训练过程中，精确地拟合损伤标注框的大小并不十分必要。并且，通过精确拟合损伤标注框得到的损伤检测模型，往往会识别出与损伤标注框大小类似的损伤预测框，这样的损伤预测框由于尺寸没有限制，常常出现跨部件的情况，类似于图1中最右侧的标注框，跨越了车前门和车后门两个部件。这反而不利于后续与部件识别结果进行融合。
基于以上的考虑和分析，在本说明书的实施例中，提出一种改进的训练损伤检测模型的方式，训练的目标是使得，损伤预测框拟合标注框的中心位置，但对标注框的大小并不进行精确拟合，而是使得预测框尽量不超过标注框，甚至越小越好。下面描述以上构思的具体实现方式和执行步骤。
图2示出根据一个实施例的训练损伤检测模型的流程图。该方法可以通过任何具有计算、处理能力的装置、设备、平台或设备集群来执行。如图2所示，该方法至少包括以下步骤：步骤210，获取标注图片，所述标注图片包括框选出损伤对象的至少一个损伤标注框；步骤220，利用损伤检测模型，在所述标注图片中预测得到至少一个损伤预测区，其中包括第一损伤预测区；步骤230，根据第一损伤预测区是否完全落入至少一个损伤标注框中，确定本次预测的损失函数中与位置偏差相关的位置损失项；步骤240，根据所述损失函数，更新所述损伤检测模型，以使得更新后所述损失函数下降。下面描述以上各个步骤的执行方式。
首先，在步骤210，获取标注图片。一般地，标注图片是由标注人员经过人工标注产生的用于模型训练的图片。在训练损伤检测模型的情况下，标注图片包括框选出损伤对象的损伤标注框，损伤标注框可以有一个或多个。如前所述，图1即为标注出损伤对 象的损伤标注图片，其中包含6个损伤标注框。一般而言，每个损伤标注框可以表示为(X，Y，W，H)的形式，其中X和Y分别为该损伤标注框的中心位置的横坐标和纵坐标，W为该损伤标注框的宽度，H为该损伤标注框的高度。在其他例子中，损伤标注框也可以用其四个顶点的坐标来表示。并且，可以理解，顶点表示方式和(X，Y，W，H)表示方式可以容易地互相转换。
接着，在步骤220，利用损伤检测模型，针对上述标注图片进行预测，得到至少一个损伤预测区。可以理解，步骤220中的损伤检测模型是训练过程中的模型，既可以是包括初始参数的初始模型，也可以是经过若干次参数调优的中间模型。
如前所述，损伤检测模型是目标检测模型的一种具体化。在本领域中，已经基于各种网络结构和各种检测算法提出了各种各样的目标检测模型。例如，单阶段(one stage)检测模型可以从图片中直接确定出目标对象的类别概率和位置坐标，也就是直接识别出目标对象。单阶段检测模型的典型例子包括，SSD模型，Yolo模型等。两阶段(two stage)的检测模型首先在图片中生成候选区域，或称为兴趣区域ROI，然后在候选区域中进行目标识别和边框回归。两阶段的检测模型的典型例子包括，R-CNN模型，Fast R-CNN模型，Faster R-CNN模型等。还提出有其他目标检测模型。以上这些具体的模型结构和算法，均可以用作损伤检测模型。
利用这样的损伤检测模型对标注图片进行预测，可以得到预测结果，即若干损伤预测区，以及对应的预测损伤类别。为了进行模型的训练和优化，接下来，将预测结果与标注数据进行比对，来确定损失函数。上述比对一般包含类别的比对和位置的比对，相应的，损失函数会包含与类别偏差相关的类别损失项，和与位置偏差相关的位置损失项。根据本说明书的实施例，可以采用常规方式确定类别损失项，例如，比对损伤预测区的预测类别和对应损伤标注框的标注类别之间的异同，根据对比结果确定该类别损失项。而对于位置损失项，不同于常规方案中既考虑中心偏差又考虑大小偏差的拟合目标，在本说明书一个或多个实施例中，根据损伤预测区是否完全落入损伤标注框中，确定损失函数中的位置损失项。
于是，在步骤230，根据上述构思，确定本次预测的损失函数中与位置偏差相关的位置损失项。为了描述的简单，结合任意的一个损伤检测区，下文中称为第一损伤预测区，进行描述。
具体的，如图2所示，确定位置损失项的步骤230包括，步骤231，判断是否存在损伤标注框完全包含该第一损伤预测区；如果存在任一损伤标注框完全包含该第一损伤 预测区，在步骤232，将该第一损伤预测区对应的位置损失项确定为零；如果各个损伤标注框均未完全包含该第一损伤预测区，那么在步骤233，从至少一个损伤标注框中确定出作为目标损伤标注框，并在步骤234，至少基于第一损伤预测区的中心和目标损伤标注框的中心之间的距离确定位置损失项。
下面结合损伤预测区的不同形式，描述在不同实施例中确定位置损失项的具体执行过程。
在一个实施例，在前述步骤220中，仅预测损伤区中心坐标。此时，得到的损伤预测区可以表示为(x，y)，退化为损伤预测点，或者认为损伤预测区即为该损伤预测点对应的像素。相应的，上述任意的第一损伤预测区具体为第一损伤预测点对应的像素，可以表示为(x 1,y 1)。
在这样的情况下，在步骤231，只需要判断该第一损伤预测点的坐标(x 1,y 1)是否落入各个损伤标注框的坐标范围之内。如前所述，各个损伤标注框可以表示为四个顶点的形式，或者根据(X，Y，W，H)转换得到四个顶点的坐标。四个顶点的坐标可以限定损伤标注框的坐标范围。于是，在该步骤231，可以将(x 1，y 1)依次与各个损伤标注框的坐标范围进行比对，以判断该第一损伤预测点是否落入某个损伤标注框中。
如果第一损伤预测点落入任意的某个损伤标注框中，那么在步骤232，将该第一损伤预测点对应的位置损失项确定为零。
如果经过步骤231的比对发现，第一损伤预测点在所有损伤标注框之外，那么，在步骤233，从至少一个损伤标注框中确定出目标损伤标注框。具体的，可以计算第一损伤预测点(x 1，y 1)与各个损伤标注框的中心(X i，Y i)之间的距离，将最短距离对应的损伤标注框作为目标损伤标注框。假定目标损伤标注框的中心为(X 1，Y 1)。
接着，在步骤234，基于第一损伤预测点(x 1，y 1)与目标损伤标注框的中心(X 1，Y 1)之间的距离，即上述最短距离，确定位置损失项。
例如，在一个例子中，可以将位置损失项L确定为上述最短距离，即：
Figure PCTCN2020071675-appb-000001
在另一例子中，可以将位置损失项L确定为上述最短距离的平方，即：
L＝(x 1-X 1) 2+(y 1-Y 1) 2    (2)
如此，确定损伤预测点的位置损失项。
根据第二实施例，在步骤220中，固定损伤预测区的宽度w 0和高度h 0，预测损伤区中心坐标(x，y)，此时，得到的损伤预测区表现为固定大小的矩形损伤预测框，损伤预测框可以表示为(x，y，w 0，h 0)。由于最终希望得到较小的损伤预测框，因此可以将固定宽度和高度设定为较小的值，例如w 0和h 0均为4个像素，等等。相应的，上述任意的第一损伤预测区具体即为第一损伤预测框，表示为(x 1，y 1，w 0，h 0)，其具有第一中心(x 1，y 1)，固定宽度w 0和固定高度h 0。
在这样的情况下，在步骤231，需要判断该第一损伤预测框(x 1，y 1，w 0，h 0)是否完全落入各个损伤标注框中。为此，可以根据其中心坐标(x 1，y 1)，宽度w 0和高度h 0，确定该第一损伤预测框的四个顶点坐标，例如包括：
Figure PCTCN2020071675-appb-000002
Figure PCTCN2020071675-appb-000003
如前所述，对于各个损伤标注框，可以预先获得其四个顶点坐标，进而获取其坐标范围。对于每个损伤标注框，可以依次判断上述第一损伤预测框的四个顶点坐标是否均落入该损伤标注框的坐标范围之内。仅在该四个顶点坐标均落入某个损伤标注框的坐标范围的情况下，才认为该第一损伤预测框完全落入或包含在该损伤标注框中。如此，判断第一损伤预测框是否完全落入各个损伤标注框。
如果第一损伤预测框完全落入任意的某个损伤标注框中，那么在步骤232，将该第一损伤预测框对应的位置损失项确定为零。
如果对于各个损伤标注框，该第一损伤预测框均未完全落入，换而言之，各个损伤标注框均不能完全包含该第一损伤预测框，那么在步骤233，从各个损伤标注框中确定出目标损伤标注框，作为第一损伤预测框计算偏差损失的对比目标。该步骤可以有多种执行方式。
图3示出在一个实施例中确定目标损伤标注框的示意图。在图3中示意性用实线示出了2个损伤标注框31和32；还用虚线框示出一个损伤预测框33。损伤标注框31可表示为(X 1，Y 1，W 1，H 1)，损伤标注框32可表示为(X 2，Y 2，W 2，H 2)。以该损伤预测框33作为当前要分析的第一损伤预测框为例，描述确定目标损伤标注框的过程。
在一个例子中，计算第一损伤预测框的中心(x 1，y 1)与各个损伤标注框的中心的距离，将最短距离对应的损伤标注框作为目标损伤标注框。
例如，可以分别计算损伤预测框33的中心(x 1，y 1)与(X 1，Y 1)的距离以及与(X 2，Y 2)的距离。假定与(X 1，Y 1)的距离更小，那么可以选择损伤标注框31作为目标损 伤标注框。
在另一例子中，对于各个损伤标注框，可以确定第一损伤预测框与损伤标注框的相交面积。例如，可以分别计算得到损伤预测框33与损伤标注框31的相交面积S1(灰色所示)，以及与损伤标注框32的相交面积S2(斜线区所示)。
然后，根据所述相交面积，确定出目标损伤标注框。例如，可以将最大相交面积对应的损伤标注框作为目标损伤标注框。例如，在这样的情况下，可以将损伤标注框32作为目标损伤标注框。在另一例子中，根据相交面积得到损伤预测框和损伤标注框的交并比(IoU)，即相交面积与合并面积的比例，选择交并比IoU最大的损伤标注框作为目标损伤标注框。在这样的情况下，在图3的例子中，可以将损伤标注框31作为目标损伤标注框。
在确定出目标损伤标注框后，在步骤234，基于第一损伤预测框的中心和目标损伤标注框的中心之间的距离确定位置损失项。在该实施例中，由于第一损伤预测框的宽度和高度都为预先设定的固定值，因此可以不必考虑预测框大小方面的损失，仅需考虑中心位置方面的损失。对于中心位置方面的损失，仍然可以采用前述的公式(1)或公式(2)来计算位置损失项，其中(x 1，y 1)为第一损伤预测框的中心坐标，(X 1，Y 1)为目标损伤标注框的中心坐标。
如此，对于大小固定的第一损伤预测框，确定出其位置损失项。
根据第三实施例，在步骤220中，对损伤区的中心位置和大小均进行预测，得到矩形的损伤预测框，可以表示为(x，y，w，h)，其中w为预测宽度，h为预测高度。这与常规的预测过程类似。由此生成的损伤预测框大小不一，并且随着模型训练过程也会不断调整变化。相应的，其中任意的第一损伤预测区具体即为第一损伤预测框，表示为(x 1，y 1，w 1，h 1)，其具有第一中心(x 1，y 1)，第一宽度w 1和第一高度h 1。
在这样的情况下，在步骤231，判断该第一损伤预测框(x 1，y 1，w 1，h 1)是否完全落入各个损伤标注框中。具体的，根据第一损伤预测框的第一中心，第一宽度和第一高度，确定其四个顶点坐标；然后对于各个损伤标注框，依次判断该四个顶点坐标是否均落入该损伤标注框的坐标范围之内。具体计算和执行过程与前述的第二实施例类似，不再赘述。
如果第一损伤预测框完全落入任意的某个损伤标注框中，那么在步骤232，将该第一损伤预测框对应的位置损失项确定为零。
如果各个损伤标注框均不能完全包含该第一损伤预测框，那么在步骤233，从各个损伤标注框中确定出目标损伤标注框，作为第一损伤预测框计算偏差损失的对比目标。在不同例子中，可以根据第一损伤预测框与各个损伤标注框之间的中心距离、相交面积、交并比等，确定出目标损伤标注框。具体计算和执行过程与前述的第二实施例类似，不再赘述。将目标损伤标注框记为(X 1，Y 1，W 1，H 1)。
在确定出目标损伤标注框后，在步骤234，对比第一损伤预测框(x 1，y 1，w 1，h 1)和目标损伤标注框(X 1，Y 1，W 1，H 1)，来确定位置损失项。在该实施例中，位置损失项既要考虑预测框中心偏差带来的损失，又要考虑框大小方面的损失。因此，可以基于第一中心和目标损伤标注框的中心之间的距离确定第一损失项，即中心偏差方面的损失项。在一个例子中，第一损失项可以采用以上公式(1)或(2)来计算。此外，还基于第一损伤预测框的宽度和高度，确定第二损失项，即框大小方面的损失项；然后基于第一损失项和第二损失项的加和，确定位置损失项。
更具体的，在一个例子中，可以将上述第二损失项确定为，第一损伤预测框的宽度w 1和预测高度h 1的运算和，例如平方和。相应的，在一个例子中，整个位置损失项可以表示为：
Figure PCTCN2020071675-appb-000004
其中，(x 1-X 1) 2+(y 1-Y 1) 2作为中心偏差的损失项，
Figure PCTCN2020071675-appb-000005
作为框大小方面的损失项。
在另一例子中，将框大小方面的第二损失项进一步分解为宽度损失项和高度损失项。对于宽度损失项，将第一损伤预测框的预测宽度w 1与目标损伤标注框的标注宽度W 1进行比对，如果预测宽度不大于标注宽度，将宽度损失项确定为零；如果预测宽度大于标注宽度，则基于预测宽度超出标注宽度的长度，确定宽度损失项。对于高度进行类似的处理：如果预测高度h 1不大于标注高度H 1，将高度损失项确定为零；如果预测高度大于标注高度，则基于预测高度超出标注高度的长度，确定高度损失项。最终将宽度损失项和高度损失项的加和，作为上述框大小方面的第二损失项。
更具体的，在一个例子中，位置损失项可以表示为：
L＝(x 1-X 1) 2+(y 1-Y 1) 2+‖max(w 1-W 1，0)‖ 2+‖max(h 1-H 1，0)‖ 2
                                     (4)
以上式(4)中，(x 1-X 1) 2+(y 1-Y 1) 2作为中心偏差的第一损失项，‖max(w 1- W 1，0)‖ 2+‖max(h 1-H 1，0)‖ 2作为框大小方面的第二损失项；其中，max(w 1-W 1，0)表示w 1-W 1和0之间取较大者。
如此，对于预测得到的任意的第一损伤预测框，确定出其位置损失项。
如前所述，对于损失函数中的其他损失项，例如预测类别损失项，可以按照常规方式确定。如此，针对各个损伤预测区确定出其损失项，基于各个损伤预测区的损失项的加和，可以得到本次预测对应的损失函数。
回到图2，在步骤240，根据上述损失函数，更新损伤检测模型，以使得更新后损失函数下降。模型的更新可以采用反向传播、梯度下降等方式。
可以理解，损伤检测模型更新的方向和目标是，使得损失函数尽量的小。因此，损失函数的定义方式，例如其中位置损失项的不同确定方式，决定了损伤检测模型的训练方向。总体而言，图2步骤230中位置损失项的确定方式，会使得预测的损伤区尽量地包含在损伤标注框中，而不是完全拟合标注框的大小。在此基础上，采用不同形式的位置损失项，得到的损伤检测模型会预测出不同的损伤预测区。
例如，当采用以上公式(1)或(2)的位置损失项时，得到的损伤检测模型可以仅预测损伤中心点，且使得预测的损伤中心点落入损伤标注框，或者尽量的接近损伤标注框的中心。
当采用公式(3)确定位置损失项进而确定损失函数时，得到的损伤检测模型会预测出损伤框，为了使得损失函数尽量小，损伤检测模型会被调整优化为，使得损失预测框的宽度w和高度h尽量小。
当采用公式(4)确定位置损失项进而确定损失函数时，得到的损伤检测模型会预测出损伤框，并且损伤检测模型会被调整为，使得损伤预测框的宽度w和高度h不超过标注框的宽度和高度。
如此，在损失函数减小的方向上不断调整和优化模型参数，最终可以得到优化更新后的损伤检测模型。
之后，在使用阶段，就可以利用更新后的损伤检测模型，对待测图片进行损伤识别。
图4示出根据一个实施例的损伤识别结果。在图4中，待测图片为车辆损伤图片。当采用以上方法训练得到的损伤检测模型对车辆损伤图片进行损伤识别时，可以得 到图中所示的一系列损伤预测框。可以看到，不同于常规技术中充分拟合标注框大小的预测框，图4中的损伤预测框都非常小，但是仍然准确指示了损伤的位置。这样的损伤预测框出现跨部件的概率非常低，对于后续与部件检测结果进行融合非常有利，从而更加有利于整图的综合定损分析。
根据另一方面的实施例，提供了一种训练损伤检测模型的装置，该装置可以部署在任何具有计算、处理能力的设备、平台或设备集群中。图5示出根据一个实施例的模型训练装置的示意性框图。如图5所示，该装置500包括：
获取单元51，配置为获取标注图片，所述标注图片包括框选出损伤对象的至少一个损伤标注框；
预测单元52，配置为利用损伤检测模型，在所述标注图片中预测得到至少一个损伤预测区，其中包括第一损伤预测区；
确定单元53，配置为确定本次预测的损失函数中与位置偏差相关的位置损失项，所述确定单元53进一步包括，
判断子单元531，配置为判断所述第一损伤预测区是否完全包含在各个损伤标注框中；
第一确定子单元532，配置为，在任一损伤标注框完全包含该第一损伤预测区的情况下，将所述位置损失项确定为零；
第二确定子单元533，配置为，在各个损伤标注框均不能完全包含该第一损伤预测区的情况下，从所述至少一个损伤标注框中确定出目标损伤标注框，至少基于所述第一损伤预测区的中心和所述目标损伤标注框的中心之间的距离确定所述位置损失项；
更新单元54，配置为根据所述损失函数，更新所述损伤检测模型，以使得更新后所述损失函数下降。
在一种实施方式中，第一损伤预测区具体为第一损伤预测点对应的像素。
在该实施方式下的一个示例中，判断子单元531配置为：判断所述第一损伤预测点的坐标是否落入各个损伤标注框的坐标范围之内；第二确定子单元533配置为：确定所述第一损伤预测点与各个损伤标注框的中心之间的距离，将最短距离对应的损伤标注框作为所述目标损伤标注框；以及，基于所述最短距离确定所述位置损失项。
在另一种实施方式中，第一损伤预测区具体为第一损伤预测框，具有第一中心， 第一宽度和第一高度。
在该实施方式下的一个示例中，所述判断子单元531配置为：根据所述第一中心，第一宽度和第一高度，确定所述第一损伤预测框的四个顶点坐标；对于各个损伤标注框，依次判断所述四个顶点坐标是否均落入该损伤标注框的坐标范围之内。
在该实施方式下的一个示例中，所述第二确定子单元533配置为：对于各个损伤标注框，确定所述第一损伤预测框与该损伤标注框的相交面积；根据所述相交面积，从所述至少一个损伤标注框中确定出所述目标损伤标注框。
在一个实施例中，上述第一宽度为预设宽度，第一高度为预设高度；在这样的情况下，所述第二确定子单元533可以配置为：基于所述第一中心和所述目标损伤标注框的中心之间的距离确定所述位置损失项。
在另一实施例中，上述第一宽度为预测宽度，第一高度为预测高度；在这样的情况下，所述第二确定子单元533可以进一步包括(未示出)：
第一损伤确定模块，配置为基于所述第一中心和所述目标损伤标注框的中心之间的距离确定第一损失项；
第二损失确定模块，配置为基于所述预测宽度和预测高度，确定第二损失项；
第三确定模块，配置为基于所述第一损失项和第二损失项的加和，确定所述位置损失项。
在一个更具体的例子中，第二损失确定模块配置为：将所述第二损失项确定为，所述预测宽度和预测高度的运算和。
在另一个更具体的例子中，目标损伤标注框具有标注宽度和标注高度；在这样的情况下，第二损失确定模块配置为：
在所述预测宽度不大于所述标注宽度的情况下，将宽度损失项确定为零；
在所述预测宽度大于所述标注宽度的情况下，基于所述预测宽度超出所述标注宽度的长度，确定所述宽度损失项；
在所述预测高度不大于所述标注高度的情况下，将高度损失项确定为零；
在所述预测高度大于所述标注高度的情况下，基于所述预测高度超出所述标注高度的长度，确定所述高度损失项；
将所述宽度损失项和高度损失项的加和，作为所述第二损失项。
通过以上的方法和装置，训练得到更适合于进行损伤对象识别的损伤检测模型。
本说明书实施例还提供从图片中识别损伤的装置。该装置可以包括获取单元，用于获取如上所述训练得到的损伤检测模型，以及包括识别单元，配置为利用上述损伤检测模型，对待测图片进行损伤识别。
根据另一方面的实施例，还提供一种计算机可读存储介质，其上存储有计算机程序，当所述计算机程序在计算机中执行时，令计算机执行结合图2所描述的方法。
根据再一方面的实施例，还提供一种计算设备，包括存储器和处理器，所述存储器中存储有可执行代码，所述处理器执行所述可执行代码时，实现结合图2所述的方法。
本领域技术人员应该可以意识到，在上述一个或多个示例中，本发明所描述的功能可以用硬件、软件、固件或它们的任意组合来实现。当使用软件实现时，可以将这些功能存储在计算机可读介质中或者作为计算机可读介质上的一个或多个指令或代码进行传输。
以上所述的具体实施方式，对本发明的目的、技术方案和有益效果进行了进一步详细说明，所应理解的是，以上所述仅为本发明的具体实施方式而已，并不用于限定本发明的保护范围，凡在本发明的技术方案的基础之上，所做的任何修改、等同替换、改进等，均应包括在本发明的保护范围之内。

Claims (23)

  1. 一种计算机执行的训练损伤检测模型的方法，包括：
    获取标注图片，所述标注图片包括框选出损伤对象的至少一个损伤标注框；
    利用损伤检测模型，在所述标注图片中预测得到至少一个损伤预测区，其中包括第一损伤预测区；
    确定本次预测的损失函数中与位置偏差相关的位置损失项，包括，
    判断所述至少一个损伤标注框中是否存在损伤标注框完全包含所述第一损伤预测区；
    如果存在损伤标注框完全包含该第一损伤预测区，将所述位置损失项确定为零；
    如果不存在损伤标注框完全包含该第一损伤预测区，从所述至少一个损伤标注框中确定出目标损伤标注框，至少基于所述第一损伤预测区的中心和所述目标损伤标注框的中心之间的距离确定所述位置损失项；
    根据所述损失函数，更新所述损伤检测模型，以使得更新后所述损失函数下降。
  2. 根据权利要求1所述的方法，其中，所述第一损伤预测区具体为第一损伤预测点对应的像素。
  3. 根据权利要求2所述的方法，其中，判断所述至少一个损伤标注框中是否存在损伤标注框完全包含所述第一损伤预测区，包括：判断所述第一损伤预测点的坐标是否落入各个损伤标注框的坐标范围之内；
    从所述至少一个损伤标注框中确定出目标损伤标注框，包括：确定所述第一损伤预测点与各个损伤标注框的中心之间的距离，将最短距离对应的损伤标注框作为所述目标损伤标注框；
    至少基于所述第一损伤预测区的中心和所述目标损伤标注框的中心之间的距离确定所述位置损失项，包括：基于所述最短距离确定所述位置损失项。
  4. 根据权利要求1所述的方法，其中，所述第一损伤预测区具体为第一损伤预测框，所述第一损伤预测框具有第一中心，第一宽度和第一高度。
  5. 根据权利要求4所述的方法，其中，判断所述至少一个损伤标注框中是否存在损伤标注框完全包含所述第一损伤预测区，包括：
    根据所述第一中心，第一宽度和第一高度，确定所述第一损伤预测框的四个顶点坐标；
    对于各个损伤标注框，依次判断所述四个顶点坐标是否均落入该损伤标注框的坐标 范围之内。
  6. 根据权利要求4所述的方法，其中，从所述至少一个损伤标注框中确定出目标损伤标注框，包括：
    对于各个损伤标注框，确定所述第一损伤预测框与该损伤标注框的相交面积；
    根据所述相交面积，从所述至少一个损伤标注框中确定出所述目标损伤标注框。
  7. 根据权利要求4所述的方法，其中，所述第一宽度为预设宽度，所述第一高度为预设高度；
    至少基于所述第一损伤预测区的中心和所述目标损伤标注框的中心之间的距离确定所述位置损失项，包括：基于所述第一中心和所述目标损伤标注框的中心之间的距离确定所述位置损失项。
  8. 根据权利要求4所述的方法，其中，所述第一宽度为预测宽度，所述第一高度为预测高度；
    至少基于所述第一损伤预测区的中心和所述目标损伤标注框的中心之间的距离确定所述位置损失项，包括：
    基于所述第一中心和所述目标损伤标注框的中心之间的距离确定第一损失项；
    基于所述预测宽度和预测高度，确定第二损失项；
    基于所述第一损失项和第二损失项的加和，确定所述位置损失项。
  9. 根据权利要求8所述的方法，其中，基于所述预测宽度和预测高度，确定第二损失项包括：
    将所述第二损失项确定为，所述预测宽度和预测高度的运算和。
  10. 根据权利要求8所述的方法，其中，所述目标损伤标注框具有标注宽度和标注高度；
    基于所述预测宽度和预测高度，确定第二损失项包括：
    在所述预测宽度不大于所述标注宽度的情况下，将宽度损失项确定为零；
    在所述预测宽度大于所述标注宽度的情况下，基于所述预测宽度超出所述标注宽度的长度，确定所述宽度损失项；
    在所述预测高度不大于所述标注高度的情况下，将高度损失项确定为零；
    在所述预测高度大于所述标注高度的情况下，基于所述预测高度超出所述标注高度的长度，确定所述高度损失项；
    将所述宽度损失项和高度损失项的加和，作为所述第二损失项。
  11. 一种训练损伤检测模型的装置，包括：
    获取单元，配置为获取标注图片，所述标注图片包括框选出损伤对象的至少一个损伤标注框；
    预测单元，配置为利用损伤检测模型，在所述标注图片中预测得到至少一个损伤预测区，其中包括第一损伤预测区；
    确定单元，配置为确定本次预测的损失函数中与位置偏差相关的位置损失项，所述确定单元包括，
    判断子单元，配置为判断所述至少一个损伤标注框中是否存在损伤标注框完全包含所述第一损伤预测区；
    第一确定子单元，配置为，如果存在损伤标注框完全包含该第一损伤预测区，将所述位置损失项确定为零；
    第二确定子单元，配置为，如果不存在损伤标注框完全包含该第一损伤预测区，从所述至少一个损伤标注框中确定出目标损伤标注框，至少基于所述第一损伤预测区的中心和所述目标损伤标注框的中心之间的距离确定所述位置损失项；
    更新单元，配置为根据所述损失函数，更新所述损伤检测模型，以使得更新后所述损失函数下降。
  12. 根据权利要求11所述的装置，其中，所述第一损伤预测区具体为第一损伤预测点对应的像素。
  13. 根据权利要求12所述的装置，其中，所述判断子单元配置为：判断所述第一损伤预测点的坐标是否落入各个损伤标注框的坐标范围之内；
    所述第二确定子单元配置为：确定所述第一损伤预测点与各个损伤标注框的中心之间的距离，将最短距离对应的损伤标注框作为所述目标损伤标注框；以及，基于所述最短距离确定所述位置损失项。
  14. 根据权利要求11所述的装置，其中，所述第一损伤预测区具体为第一损伤预测框，所述第一损伤预测框具有第一中心，第一宽度和第一高度。
  15. 根据权利要求14所述的装置，其中，所述判断子单元配置为：
    根据所述第一中心，第一宽度和第一高度，确定所述第一损伤预测框的四个顶点坐标；
    对于各个损伤标注框，依次判断所述四个顶点坐标是否均落入该损伤标注框的坐标范围之内。
  16. 根据权利要求14所述的装置，其中，所述第二确定子单元配置为：
    对于各个损伤标注框，确定所述第一损伤预测框与该损伤标注框的相交面积；
    根据所述相交面积，从所述至少一个损伤标注框中确定出所述目标损伤标注框。
  17. 根据权利要求14所述的装置，其中，所述第一宽度为预设宽度，所述第一高度为预设高度；
    所述第二确定子单元配置为：基于所述第一中心和所述目标损伤标注框的中心之间的距离确定所述位置损失项。
  18. 根据权利要求14所述的装置，其中，所述第一宽度为预测宽度，所述第一高度为预测高度；
    所述第二确定子单元包括：
    第一损伤确定模块，配置为基于所述第一中心和所述目标损伤标注框的中心之间的距离确定第一损失项；
    第二损失确定模块，配置为基于所述预测宽度和预测高度，确定第二损失项；
    第三确定模块，配置为基于所述第一损失项和第二损失项的加和，确定所述位置损失项。
  19. 根据权利要求18所述的装置，其中，所述第二损失确定模块配置为：将所述第二损失项确定为，所述预测宽度和预测高度的运算和。
  20. 根据权利要求18所述的装置，其中，所述目标损伤标注框具有标注宽度和标注高度；
    所述第二损失确定模块配置为：
    在所述预测宽度不大于所述标注宽度的情况下，将宽度损失项确定为零；
    在所述预测宽度大于所述标注宽度的情况下，基于所述预测宽度超出所述标注宽度的长度，确定所述宽度损失项；
    在所述预测高度不大于所述标注高度的情况下，将高度损失项确定为零；
    在所述预测高度大于所述标注高度的情况下，基于所述预测高度超出所述标注高度的长度，确定所述高度损失项；
    将所述宽度损失项和高度损失项的加和，作为所述第二损失项。
  21. 一种计算机执行的从图片中识别损伤的方法，包括：
    获取根据权利要求1-10中任一项的方法训练得到的损伤检测模型；
    利用所述损伤检测模型，对待测图片进行损伤识别。
  22. 一种从图片中识别损伤的装置，包括：
    获取单元，配置为获取通过权利要求11-20中任一项的装置训练得到的损伤检测模 型；
    识别单元，配置为利用所述损伤检测模型，对待测图片进行损伤识别。
  23. 一种计算设备，包括存储器和处理器，其特征在于，所述存储器中存储有可执行代码，所述处理器执行所述可执行代码时，实现权利要求1-10中任一项所述的方法。
PCT/CN2020/071675 2019-05-10 2020-01-13 计算机执行的从图片中识别损伤的方法及装置 WO2020228370A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/813,355 US10885625B2 (en) 2019-05-10 2020-03-09 Recognizing damage through image analysis

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910388548.0A CN110569703B (zh) 2019-05-10 2019-05-10 计算机执行的从图片中识别损伤的方法及装置
CN201910388548.0 2019-05-10

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/813,355 Continuation US10885625B2 (en) 2019-05-10 2020-03-09 Recognizing damage through image analysis

Publications (1)

Publication Number Publication Date
WO2020228370A1 true WO2020228370A1 (zh) 2020-11-19

Family

ID=68773458

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/071675 WO2020228370A1 (zh) 2019-05-10 2020-01-13 计算机执行的从图片中识别损伤的方法及装置

Country Status (2)

Country Link
CN (1) CN110569703B (zh)
WO (1) WO2020228370A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112884054A (zh) * 2021-03-03 2021-06-01 歌尔股份有限公司 一种目标标注方法和一种目标标注装置
CN113128553A (zh) * 2021-03-08 2021-07-16 北京航空航天大学 基于目标架构的目标检测方法、装置、设备及存储介质
CN113780356A (zh) * 2021-08-12 2021-12-10 北京金水永利科技有限公司 基于集成学习模型的水质预测方法及系统
CN115375987A (zh) * 2022-08-05 2022-11-22 北京百度网讯科技有限公司 一种数据标注方法、装置、电子设备及存储介质
CN117078901A (zh) * 2023-07-12 2023-11-17 长江勘测规划设计研究有限责任公司 一种钢筋视图单点筋自动标注方法

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10885625B2 (en) 2019-05-10 2021-01-05 Advanced New Technologies Co., Ltd. Recognizing damage through image analysis
CN110569703B (zh) * 2019-05-10 2020-09-01 阿里巴巴集团控股有限公司 计算机执行的从图片中识别损伤的方法及装置
CN116368537A (zh) * 2021-10-28 2023-06-30 京东方科技集团股份有限公司 目标检测模型的训练方法及装置、目标检测方法及装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109087294A (zh) * 2018-07-31 2018-12-25 深圳辰视智能科技有限公司 一种产品缺陷检测方法、系统及计算机可读存储介质
CN109117831A (zh) * 2018-09-30 2019-01-01 北京字节跳动网络技术有限公司 物体检测网络的训练方法和装置
CN109308681A (zh) * 2018-09-29 2019-02-05 北京字节跳动网络技术有限公司 图像处理方法和装置
WO2019028725A1 (en) * 2017-08-10 2019-02-14 Intel Corporation CONVOLUTIVE NEURAL NETWORK STRUCTURE USING INVERTED CONNECTIONS AND OBJECTIVITY ANTERIORITIES TO DETECT AN OBJECT
CN109377508A (zh) * 2018-09-26 2019-02-22 北京字节跳动网络技术有限公司 图像处理方法和装置
CN110569703A (zh) * 2019-05-10 2019-12-13 阿里巴巴集团控股有限公司 计算机执行的从图片中识别损伤的方法及装置

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8488181B2 (en) * 2010-04-07 2013-07-16 Xerox Corporation Preserving user applied markings made to a hardcopy original document
WO2012115594A1 (en) * 2011-02-21 2012-08-30 Stratech Systems Limited A surveillance system and a method for detecting a foreign object, debris, or damage in an airfield
CN106096670B (zh) * 2016-06-17 2019-07-30 深圳市商汤科技有限公司 级联卷积神经网络训练和图像检测方法、装置及系统
CN106897742B (zh) * 2017-02-21 2020-10-27 北京市商汤科技开发有限公司 用于检测视频中物体的方法、装置和电子设备
CN107657224B (zh) * 2017-09-19 2019-10-11 武汉大学 一种基于部件的多层并行网络sar图像飞机目标检测方法
CN108197658B (zh) * 2018-01-11 2020-08-14 阿里巴巴集团控股有限公司 图像标注信息处理方法、装置、服务器及系统
CN108537215B (zh) * 2018-03-23 2020-02-21 清华大学 一种基于图像目标检测的火焰检测方法
CN108921811B (zh) * 2018-04-03 2020-06-30 阿里巴巴集团控股有限公司 检测物品损伤的方法和装置、物品损伤检测器
CN108711148B (zh) * 2018-05-11 2022-05-27 沈阳理工大学 一种基于深度学习的轮胎缺陷智能检测方法
CN109190631A (zh) * 2018-08-31 2019-01-11 阿里巴巴集团控股有限公司 图片的目标对象标注方法及装置
CN109389640A (zh) * 2018-09-29 2019-02-26 北京字节跳动网络技术有限公司 图像处理方法和装置
CN109409365A (zh) * 2018-10-25 2019-03-01 江苏德劭信息科技有限公司 一种基于深度目标检测的待采摘水果识别和定位方法
CN109657716B (zh) * 2018-12-12 2020-12-29 中汽数据(天津)有限公司 一种基于深度学习的车辆外观损伤识别方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019028725A1 (en) * 2017-08-10 2019-02-14 Intel Corporation CONVOLUTIVE NEURAL NETWORK STRUCTURE USING INVERTED CONNECTIONS AND OBJECTIVITY ANTERIORITIES TO DETECT AN OBJECT
CN109087294A (zh) * 2018-07-31 2018-12-25 深圳辰视智能科技有限公司 一种产品缺陷检测方法、系统及计算机可读存储介质
CN109377508A (zh) * 2018-09-26 2019-02-22 北京字节跳动网络技术有限公司 图像处理方法和装置
CN109308681A (zh) * 2018-09-29 2019-02-05 北京字节跳动网络技术有限公司 图像处理方法和装置
CN109117831A (zh) * 2018-09-30 2019-01-01 北京字节跳动网络技术有限公司 物体检测网络的训练方法和装置
CN110569703A (zh) * 2019-05-10 2019-12-13 阿里巴巴集团控股有限公司 计算机执行的从图片中识别损伤的方法及装置

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112884054A (zh) * 2021-03-03 2021-06-01 歌尔股份有限公司 一种目标标注方法和一种目标标注装置
CN112884054B (zh) * 2021-03-03 2022-12-09 歌尔股份有限公司 一种目标标注方法和一种目标标注装置
CN113128553A (zh) * 2021-03-08 2021-07-16 北京航空航天大学 基于目标架构的目标检测方法、装置、设备及存储介质
CN113780356A (zh) * 2021-08-12 2021-12-10 北京金水永利科技有限公司 基于集成学习模型的水质预测方法及系统
CN113780356B (zh) * 2021-08-12 2023-08-08 北京金水永利科技有限公司 基于集成学习模型的水质预测方法及系统
CN115375987A (zh) * 2022-08-05 2022-11-22 北京百度网讯科技有限公司 一种数据标注方法、装置、电子设备及存储介质
CN115375987B (zh) * 2022-08-05 2023-09-05 北京百度网讯科技有限公司 一种数据标注方法、装置、电子设备及存储介质
CN117078901A (zh) * 2023-07-12 2023-11-17 长江勘测规划设计研究有限责任公司 一种钢筋视图单点筋自动标注方法
CN117078901B (zh) * 2023-07-12 2024-04-16 长江勘测规划设计研究有限责任公司 一种钢筋视图单点筋自动标注方法

Also Published As

Publication number Publication date
CN110569703A (zh) 2019-12-13
CN110569703B (zh) 2020-09-01

Similar Documents

Publication Publication Date Title
WO2020228370A1 (zh) 计算机执行的从图片中识别损伤的方法及装置
TWI742382B (zh) 透過電腦執行的、用於車輛零件識別的神經網路系統、透過神經網路系統進行車輛零件識別的方法、進行車輛零件識別的裝置和計算設備
WO2018108129A1 (zh) 用于识别物体类别的方法及装置、电子设备
TWI726364B (zh) 電腦執行的車輛定損方法及裝置
WO2020238256A1 (zh) 基于弱分割的损伤检测方法及装置
Wei et al. A vision and learning-based indoor localization and semantic mapping framework for facility operations and management
US10853699B2 (en) Method and apparatus for vehicle damage identification
US11080839B2 (en) System and method for training a damage identification model
US10885625B2 (en) Recognizing damage through image analysis
TWI716012B (zh) 樣本標註方法、裝置、儲存媒體和計算設備、損傷類別的識別方法及裝置
US20200294222A1 (en) Method and apparatus for outputting information
US20150302240A1 (en) Method and device for locating feature points on human face and storage medium
WO2022012110A1 (zh) 胚胎光镜图像中细胞的识别方法及系统、设备及存储介质
CN110569700B (zh) 优化损伤识别结果的方法及装置
CN110580723A (zh) 一种利用深度学习和计算机视觉进行精准定位的方法
US11004204B2 (en) Segmentation-based damage detection
CN111656358A (zh) 分析捕获的图像以确定测试成果
CN108428248B (zh) 车窗定位方法、系统、设备及存储介质
WO2020052352A1 (zh) 用于对车辆损伤图像进行损伤分割的方法及装置
CN110751179A (zh) 病灶信息获取方法、病灶预测模型的训练方法及超声设备
CN109145752A (zh) 用于评估对象检测和跟踪算法的方法、装置、设备和介质
CN115994899A (zh) 一种螺栓松动检测方法、装置及检测设备
CN114154563A (zh) 基于混合监督训练的目标检测方法
CN116977257A (zh) 缺陷检测方法、装置、电子设备、存储介质及程序产品
CN114841255A (zh) 检测模型的训练方法、装置、设备、存储介质及程序产品

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20805309

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20805309

Country of ref document: EP

Kind code of ref document: A1