KR102198296B1 - Apparatus, method and computer program for automatically calculating the damage - Google Patents

Abstract

According to the present specification, the method comprising: detecting object regions corresponding to the damaged portion of the accident vehicle using one or more images photographing the accident vehicle by using an algorithm learned through the one or more image sets input by the device; Determining, by the device, labels of object regions detected in correspondence with each part of the accident vehicle; Each object region labeled by the device is filtered by the method adopted through a convolutional neural network (CNN) layer, and is arranged in the order of the level of damage inferred from the degree of damage, which is the result data, and the size of the matching probability based on the filtered images. Generating a failure level list; Disclosed is a method of automatically calculating the degree of damage of the accident vehicle comprising the step of determining the damage level having the highest priority in the damage level list.

Description

A device, method, and computer program that automatically calculates the degree of damage to an accident vehicle {APPARATUS, METHOD AND COMPUTER PROGRAM FOR AUTOMATICALLY CALCULATING THE DAMAGE}

An embodiment of the present invention relates to an apparatus, a method, and a computer program for automatically calculating the degree of damage of an accident vehicle.

In recent years, automobiles are increasingly used as means of transportation, transportation, and living spaces for people. In the past, it was in the form of a vehicle that functions as a simple means of movement and transportation, but in recent years, it is changing to a form of a vehicle that functions as a living space. Through this, the utilization of urban spaces can be increased, and representative examples include camping cars and food trucks.

In addition, it is becoming more common to have one or more vehicles per household. Accordingly, the amount of use of automobiles has increased significantly, and the importance of accident compensation has emerged as well as increased convenience.

In the event of a car accident, it is common to calculate the damage for the accident through confirmation of the automobile repairing company.

The damaged vehicle is moved to an industrial company, and the repair cost is claimed through the quotation system, and the damage assessment person in charge checks the claim details, damage photos, photos during repair, and repair completion photos, and proceeds with damage assessment, and finally, the damage assessment details. It was transferred to the computer system, paid for the repair, and then closed. The details of damages and image files that occurred during this process were stored separately in the server, so the correlation between the two was not known.

According to embodiments of the present invention, an apparatus, method, and computer program for automatically calculating the degree of damage of an accident vehicle that increase the accuracy of the degree of damage by updating an algorithm for calculating the damage level for each vehicle part and each vehicle part from an image Can be provided.

The automatic damage calculation method according to the embodiments of the present invention includes the steps of: receiving, by an automatic damage calculation device, an entire image of an accident vehicle and a partial image of the accident area from a user terminal; Determining a vehicle type and a model year of the accident vehicle using the entire image and pre-stored standard vehicle images; Detecting the entire image and the damaged portions included in the partial image using a partial detection algorithm; Determining a breakage level for each of the breakage portions using a breakage determination algorithm; And using a previously stored table to generate work information for the accident vehicle in consideration of the damaged part and the damage level for each damaged part, the vehicle name, the vehicle model, the year, and the painting coat, and the work information and It may include the steps of obtaining related detailed tasks, applying standardized working hours of the detailed tasks, and generating an automatic estimate by reflecting the maintenance cost of individual contracts for each company, and providing the automatic estimate to the user terminal. .

The automatic damage calculation method may further include transmitting the automatic estimate to the industrial company terminal to which the accident vehicle was moved.

The automatic damage calculation method includes receiving a suitability for the automatic estimate, a suitability for a detailed work included in the automatic estimate, and a suitability for the cost of the detailed work; And modifying the logic for generating an automatic quote in consideration of the received suitability.

The automatic damage calculation method obtains information on a vehicle owned by the user terminal based on the identification information of the user terminal, determines whether or not the vehicle type and year obtained through the vehicle information and the photographed image match, and matches. It can be characterized by controlling the creation of automatic quotes in only one case.

The automatic damage calculation method requests additional verification of the first damaged part when the first damaged part found in the image taken before the time of the accident is detected among one or more damaged parts determined through the captured image. It can be characterized.

The additional verification is primarily transmitted to the terminal of the vehicle owner or the terminal of the insurance company employee of the vehicle, and if it is determined that the first damaged part was caused by the owner's carelessness, the automatic estimate for the first damaged part It can be characterized by a point to be excluded from.

The automatic damage calculation apparatus may be characterized in that it provides a detailed list of work details to an industrial company's terminal so that repairs according to the automatically recognized damage level can be performed.

The partial detection algorithm or the damage determination algorithm may be updated by an external management server, and may be updated by a signal transmitted from the management server.

The automatic quotation may be determined by standard repair approval criteria, and the standard repair approval criteria are set to be updated through a predetermined correction procedure when a plurality of correction requests are received from at least one or more of the user terminals. You can do it.

The method of automatically calculating the degree of damage according to embodiments of the present invention uses an algorithm learned through one or more set of labeled and input images, and corresponds to one or more images of the accident vehicle with the damaged part of the accident vehicle. Detecting the object regions to be formed; Determining and labeling labels of object regions detected in correspondence with each part of the accident vehicle; Filtering each labeled object region by the method adopted through the CNN layer, and generating a list of damage levels arranged in the order of the size of the damage level and the matching probability inferred from the damage degree as the result data based on the filtered images; It may include; determining a failure level that is ranked highest in the failure level list.

The detecting of the object regions may include detecting object regions to include regions recognized as bumpers, fenders, and doors of the accident vehicle in the one or more image sets, but detection to match irregular boundaries of bumpers, fenders, and doors. It can be characterized by becoming.

The labeled object area may be filtered by at least one filter of a Sobel mask or a canny edge detector.

The labeled object area may be characterized in that after filtering, the contrast is uniformed by a uniformization algorithm.

The damage level of each object area is set to one of normal, sheet metal 0.5, sheet metal 1, sheet metal 1.5, sheet metal 2, and exchange, but is not limited thereto. The damage level list is arranged in the order of normal, sheet metal 0.5, sheet metal 1, sheet metal 1.5, sheet metal 2, and the size of the matching probability for exchange, and provides the highest level of damage with the highest matching probability. I can.

In the method of automatically calculating the degree of damage according to embodiments of the present invention, an error of a damaged part is calculated for one or more input images, and when the degree of error exceeds a preset first value, the image of the accident vehicle Labeling and re-learning; may further include.

In the method of automatically calculating the degree of damage according to embodiments of the present invention, when the error degree of the failure level is input and the degree of error exceeds a preset second value, the object area corresponding to the damaged part and the damage The step of re-learning the level; may further include.

A computer program according to an embodiment of the present invention may be stored in a medium to execute any one of the methods of automatically calculating the degree of damage of an accident vehicle according to the embodiment of the present invention using a computer.

In addition to this, another method for implementing the present invention, another system, and a computer-readable recording medium for recording a computer program for executing the method are further provided.

Other aspects, features, and advantages other than those described above will become apparent from the following drawings, claims, and detailed description of the invention.

According to embodiments of the present invention, by updating an algorithm for calculating a vehicle part and a damage level for each vehicle part from an image, the damage degree of the accident vehicle may be automatically calculated to increase the accuracy of the damage degree.

1 is a block diagram showing the structure of an automatic damage calculation system according to embodiments of the present invention.
2 is a block diagram showing the structure of an automatic damage calculation apparatus according to embodiments of the present invention.
3 is a block diagram showing the structure of a failure level determining unit.
4 is a diagram showing the structure of a management server according to embodiments of the present invention.
5 is a flowchart of an automatic loss calculation method according to embodiments of the present invention.
6 to 8 are flowcharts of a method for determining a failure level according to embodiments of the present invention.
9 is an exemplary diagram representing damaged parts recognized by the automatic damage calculation apparatus.
10 is an exemplary diagram of a damage level list provided by an automatic damage calculation apparatus according to embodiments of the present invention.
11 and 12 are diagrams for explaining an image pre-processing process performed by the damage level determination unit of the automatic damage calculation apparatus.
13 is an exemplary diagram of a screen comparing an estimate automatically generated by an automatic loss calculation device and an actual estimate for an industrial company.
14 is an exemplary diagram of result data obtained through an image.
15A, 15B, and 15C are exemplary diagrams of standard repair recognition criteria used in the automatic damage calculation apparatus.

Since the present invention can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and will be described in detail in the detailed description. Effects and features of the present invention, and a method of achieving them will be apparent with reference to the embodiments described later in detail together with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms.

According to embodiments of the present invention, the automatic damage calculation system may generate an automatic estimate by automatically calculating a repair cost by analyzing an image of an accident vehicle. The automatic damage calculation system can prevent excessive repair details from being added by sending an automatic estimate to the terminal of the industrial company that received the delivery of the accident vehicle according to image recognition technology and standard repair recognition standards. By providing standard repair details for accident vehicles, it is possible to limit the increase in auto insurance premiums. The automatic damage calculation system can accurately detect vehicle parts and damage levels for each vehicle part using only the image of the accident vehicle. The automatic loss calculation system helps to deal with accidents in connection with the auto insurance company. The automatic damage calculation system can automatically generate and provide an estimate based on the degree of damage and the damage determined by the naked eye through the image of the accident vehicle. Standardized accident vehicle repair estimates can be generated to avoid unpredictable charges.

1 is a block diagram showing the structure of an automatic damage calculation system according to embodiments of the present invention.

The automatic damage calculation system 10 may include user terminals 100 and 400, an automatic damage calculation device 200, and a management server 300.

When the automatic damage calculation apparatus 200 receives an automatic quotation request signal from the terminal 100 of the owner of the accident vehicle or the employee of the insurance company, the function of photographing the accident vehicle may be automatically executed. The automatic damage calculation system 10 may perform a process of scanning the accident vehicle on the preview screen of the camera, and may take and receive images during the scanning process.

The automatic damage calculation apparatus 200 may generate an automatic estimate of the accident vehicle according to the user's request signal for an automatic estimate. The automatic damage calculation apparatus 200 determines the vehicle type, model year, accident part, and vehicle part from the images of the accident vehicle and the accident vehicle, and finally calculates the damage level of the accident vehicle. The automatic damage assessment device 200 may generate an automatic estimate based on the damage level of the accident vehicle. The generated automatic estimate may be provided to the management server 300 for processing and history management of the accident vehicle, or may be provided to the industrial company's terminal 400 to handle the accident vehicle as an appropriate damage situation.

The automatic damage assessment apparatus 200 may learn the damage level determination unit or the automatic estimate generation unit using the ended details made by the damage assessment personnel.

User terminals 100 and 400 are electronic devices provided by a user related to an accident vehicle, such as a vehicle owner, an insurance company employee, and an industrial company's terminal, and the automatic damage calculation apparatus 200 and the management server 300 through a communication network. ) Can be connected. The user terminals 100 and 400 may receive information processed by the automatic damage calculation apparatus 200 and may input data to the management server 300.

When an accident is detected with an impact or the like by a black box or sensor of the accident vehicle 101, an accident detection signal is generated by the user terminal 100, and the user terminal 100 generates the accident detection signal by the automatic damage calculation device 200 ). The automatic damage calculation apparatus 200 receiving the accident detection signal through this process may acquire an image of the accident vehicle through the user terminal.

In FIG. 1, the user terminals 100 and 400 are respectively illustrated in a singular number, but a plurality of user terminals may be connected to the automatic loss calculation apparatus 200 and the management server 300.

The user terminals 100 and 400 refer to communication terminals capable of transmitting and receiving data in a wired or wireless communication environment. Here, the user terminals 100 and 400 may be a personal computer of the user or a portable terminal of the user. In FIG. 1, the portable terminal is shown as a smartphone, but the spirit of the present invention is not limited thereto, and as described above, a terminal equipped with a program capable of connecting to a communication network or connected to a communication module may be borrowed without limitation.

In more detail, the user terminals 100 and 400 include computers (eg, desktops, laptops, tablets, etc.), media computing platforms (eg, cables, satellite set-top boxes, digital video recorders), and handheld computing. Devices (eg, PDA, email client, etc.), mobile phones, or other types of computing or communication platforms may be included, but the present invention is not limited thereto.

The automatic damage calculation device 200 detects the vehicle part of each accident vehicle from the images of the accident vehicle, infers the damage level for each vehicle part, and applies the inferred contents to the standard repair recognition criteria to calculate an automatic estimate. I can.

In the conventional automatic damage assessment device 200, after extracting the image data based on the accreditation details of damage assessment criteria such as bumper replacement, bumper sales 2 hours, fender exchange, etc., damage assessment specialists individually perform normal, scratch, minor damage, heavy damage. The level of damage was determined by labeling it as, bad damage, etc., or by using an algorithm learned with data classified as the same damage. However, even though the algorithm generated by using the training data of about 50,000 cases per part was used, the recognition accuracy had a limitation that did not increase beyond a certain level. As a result of analyzing the reason for the low recognition accuracy, the necessity of learning using at least 100,000 data per part emerged.

The management server 300 performs a function of updating an algorithm for inferring more accurate vehicle parts and damage levels for each vehicle part. The updated algorithm may be periodically provided to the automatic damage calculation apparatus 200. The management server 300 receives an image of an actual accident vehicle, an actual estimate for the accident, and the like, and modifies the modeling data for automatic calculation by the person concerned (insurance company employee, industrial company employee, etc.) of the accident vehicle. The modeling data refers to commands constituting the partial detection algorithm, the damage determination algorithm, and the like, and can be used to readjust the published algorithms for vehicle part detection and the degree of damage determination.

The communication network 500 connects a plurality of user terminals 100 and 400, the automatic loss calculation apparatus 200, and the management server 300. That is, the communication network 500 refers to a communication network that provides a connection path so that the user terminals 100 and 400 can transmit and receive data after accessing the automatic damage calculation apparatus 200 and the management server 300. The communication network 400 is, for example, a wired network such as LANs (Local Area Networks), WANs (Wide Area Networks), MANs (Metropolitan Area Networks), ISDNs (Integrated Service Digital Networks), wireless LANs, CDMA, Bluetooth, satellite communication, etc. It may cover wireless networks of, but the scope of the present invention is not limited thereto.

2 is a block diagram showing the structure of an automatic damage calculation apparatus 200 according to embodiments of the present invention.

As shown in FIG. 2, the automatic loss calculation apparatus 200 may include a processor 210, a memory 220, and a communication unit 230 to calculate an automatic estimate.

The processor 210 typically controls the overall operation of the device 200. For example, the processor 210 may overall control the communication unit 230 and the like by executing programs stored in the memory 220.

The communication unit 230 may include one or more components that enable communication between the user terminals 100 and 400, the management servers 300, and the automatic damage calculation apparatus 200. The communication unit 230 includes a Bluetooth communication unit, a Bluetooth Low Energy (BLE) communication unit, a Near Field Communication unit, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, an infrared data association (IrDA) communication unit, a WFD ( It may include a Wi-Fi Direct) communication unit, an ultra wideband (UWB) communication unit, a short-range communication unit such as an Ant+ communication unit, and a mobile communication network.

The memory 220 includes a reception unit 221 that receives data received through the communication unit 230, a damage level determination unit 240 that determines a damage level by analyzing the received data, a damage level, and a previously stored standard repair accreditation standard. A job information generation unit 222 that generates job information in consideration of, and an automatic quotation generation unit 223 that generates an automatic quotation in consideration of job information and current wages, and receives feedback in addition to automatic quotation generation. A feedback receiving unit 224 and a logic optimization unit 225 for updating the breakage level algorithm may be included.

When a vehicle accident is received from the user terminal 100, the receiver 221 receives images of the accident vehicle. The receiver 221 may further receive specific information on the damaged part through a third party in order to prevent additional repair of the previous damage history caused by the owner's carelessness.

In order to prevent false reporting of an insurance accident, the receiving unit 221 may identify whether the accident vehicle matches the user's vehicle, and thus prevent damage to other accident vehicles from being calculated. The receiving unit 221 compares the vehicle (vehicle type, year, etc.) information registered using the identification information of the user terminal with the vehicle type and year obtained through the image to determine whether the accident vehicle is identical, and when it is determined that it matches If so, it will start generating an automatic quote. In this case, information such as a vehicle type and a model year may be obtained through a vehicle information request signal including a vehicle number obtained through an image.

The damage level determination unit 240 may detect a vehicle part from images photographed of the accident vehicle and determine a damage level for each detected vehicle part. The damage level determination unit 240 may detect images photographed of the accident vehicle according to an irregular boundary line of a vehicle portion using a partial detection algorithm. The partial detection algorithm uses an image recognition algorithm, and one of various methods such as standardized algorithms such as YOLO, SSD, R CNN, MASK R CNN, and TEST R CNN is used, but is not limited thereto, and is freely partially or entirely free. can be changed. The partial detection algorithm is modeled by the management server 300 and follows an algorithm selected based on a ratio of accurately detecting a vehicle portion in an image of an accident vehicle.

Vehicle parts can be classified and recognized for each vehicle type. For example, the appearance of SUV and sedan models and parts of each vehicle may be recognized differently. In this embodiment, the vehicle parts are front bumper, left fender, head lamp left, right fender, head lamp right, left door, right door, left rear door, right rear door, left rear fender, rear combination lamp, right rear fender, rear combination, It may include at least one of a trunk lid, a rear bumper, and a hood.

In order to accurately determine the damaged area, a standard vehicle image can be managed for each vehicle part for each vehicle type. Vehicle parts according to vehicle types may be determined by comparing vehicle parts included in the captured image with a standard vehicle image for each vehicle part of a corresponding vehicle type.

The damage level determination unit 240 may determine a damage level by analyzing each of the object regions detected for each vehicle part. The damage level determination unit 240 may use a damage determination algorithm modeled in the management server 300. The damage determination algorithm includes a correspondence relationship between the properties of the images obtained by repeatedly learning the image of the accident vehicle and the data set of the damage level and the damage level. The damage determination algorithm is an image recognition algorithm, and various methods such as standardized models such as ResNet, Inception, Inception-ReNnet, etc. may be used, but are not limited thereto.

The work information generation unit 222 may generate work information of the accident vehicle by applying the vehicle part and the damage levels for each vehicle part acquired from the image of the accident vehicle to the standard repair recognition criteria prescribed according to the vehicle type and year.

The work information of the accident vehicle includes incidental work, and the scope of the accredited work is determined by standard repair approval criteria among incidental work. Examples of work information provided by this embodiment are front bumper replacement, front bumper repair, left fender replacement, left fender sheet metal, headlamp left replacement, right fender replacement, right fender sheet metal, headlamp right replacement, left door replacement, Left door sheet metal, right door replacement, right door sheet metal, left and rear door replacement, left and rear door sheet metal, upper and rear door replacement, upper and rear door sheet metal, left and rear fender replacement, left and rear fender sheet metal, rear combination lamp (left), right and rear fender replacement, right and rear fender sheet metal , Rear combination lamp (right), trunk lid replacement, trunk lid sheet metal, rear bumper replacement, rear bumper sheet metal, hood replacement, hood sheet metal. At this time, when it is determined as sheet metal, the time of the sheet metal may be additionally displayed as 0.5, 1, 1.5, 2, etc.

Here, the standard repair recognition criteria may include distinguished repair costs according to the vehicle type, year, painting coat, factory M/H, and repair method. Models registered in the standard repair recognition criteria include nf Sonata, Sonata Transform, yf Sonata, if Sonata, New Morning, All New Morning, All New Morning, Santa Fe, Sportage, SM5, Starex, Carnival, Sorento, Tucson, K5, etc. There may be.

The automatic quotation generating unit 223 generates an automatic quotation based on the work information of the accident vehicle. It is possible to extract the repair cost of each of the certified work included in the work information by considering the vehicle type and year of the accident vehicle, and generate an automatic estimate including the certification work details and repair cost.

The feedback receiving unit 224 receives feedback on the generated automatic quote. At this time, the feedback receiving unit 224 receives the suitability of the certified work, the suitability of the damaged part, the suitability of the damage level, and the suitability of the repair cost of each of the certified work in the automatic estimate form from the terminal of the industrial company that actually repairs. The feedback received from the industry can be used to update the scope of the pre-stored standard acceptance criteria for accreditation work and the repair cost of each accreditation operation. In addition, the feedback may be transmitted to the management server 300 and used to update the partial detection algorithm and the damage determination algorithm.

The logic optimization unit 225 performs a function of optimizing the standard acceptance criteria used to generate an automatic estimate. The logic optimization unit 225 updates the standard repair recognition criteria based on the price of each vehicle part of the vehicle manufacturer when a new model year appears.

When a discontinued vehicle occurs, the logic optimization unit 225 may process an accident estimate of the discontinued vehicle so that it is not possible to automatically estimate the discontinued vehicle, and may request an actual quotation through the terminal of the industry.

When the entire damaged accident vehicle image is received, the logic optimization unit 225 may stop generating an automatic quotation and request an actual processing quotation through a terminal of an insurance company or an industrial company.

Through this, the automatic damage calculation apparatus 200 according to the embodiments of the present invention can automatically infer the damage level of the accident vehicle and generate an automatic estimate without the need for an industrial company. The automatic damage calculation apparatus 200 may suppress occurrence of a repair process exceeding the degree of damage and reduce errors in damage assessment. The automatic damage calculation apparatus 200 may automatically generate an estimate only by inputting an image that minimizes the user's input. The automatic damage calculation apparatus 200 may simply receive an accident and provide brief damage assessment through a terminal held by the user. The automatic damage calculation apparatus 200 may increase the accuracy of the damage degree by updating an algorithm for calculating the vehicle part and the damage level for each vehicle part from the image.

3 is a block diagram showing the structure of the failure level determining unit 240.

As shown in FIG. 3, the damage level determination unit 240 includes a damage level inference unit 241 that detects a damaged vehicle part, and a damage level inference unit 242 that probably infers a damage level for each damaged part. Can include.

The damaged part inference unit 241 may detect object regions corresponding to the damaged part of the accident vehicle from one or more images photographed of the accident vehicle using an algorithm learned through one or more labeled and input image sets. At this time, the learned algorithm is learned using a large amount of accident vehicle images and damage level information, and after the learning, reinforcement learning may be performed through high-quality data. The algorithm learning process will be described with reference to FIGS. 6 to 8.

The damaged part inference unit 241 may be expressed by including a damage label in each of the object regions corresponding to the damaged part.

The damaged part inferring unit 241 detects object areas including areas recognized as bumpers, fenders, and doors of the accident vehicle from the received one or more image sets, but is detected to match irregular boundaries of bumpers, fenders, and doors. I can.

The damage level inference unit 242 acquires damage information on the object area corresponding to the damaged part by using the damage level determination model, which is a damage level determination algorithm, and at least one damage level deduced from the damage information and a matching probability for each damage level You can create a list of failure levels arranged in order of size. The damage level inference unit 242 filters each specifically labeled object region by a method adopted through a convolutional neural network layer (CNN), and at least one damage level inferred from the damage information, which is result data, based on the filtered images. It is possible to create a list of damage levels arranged in the order of the size of the matching probability by damage level. The CNN layer may perform a function of selecting a filtering method for filtering attributes related to a failure level among attributes included in an image. The labeled object area may be filtered with at least one filter of a Sobel mask or a canny edge detector. The object area is characterized in that after filtering, the brightness and darkness are equalized by the equalization algorithm. The damage level of each object area can be set to one of Normal, Sheet Metal 0.5, Sheet Metal 1, Sheet Metal 1.5, Sheet Metal 2, and Exchange. The damage level list is arranged in the order of normal, sheet metal, 0.5, sheet metal 1, sheet metal 1.5, sheet metal 2, and the size of the matching probability for the exchange, and the failure level with the highest matching probability can be given the highest priority.

The failure level inference unit 242 determines the failure level with the highest priority as the failure level for the corresponding damaged part.

The damaged part inference unit 241 calculates whether there is an error in the damaged part for one or more input images, and when the error degree exceeds a preset first value, label the image of the accident vehicle and transmit it to the management server for relearning. Can perform the function If the accuracy of the damaged part value is less than 50%, the detected damaged part can be determined as an error. The accuracy of the damaged part value can be obtained through information provided by the part detection algorithm.

The damage level inference unit 242 receives the error level of the damage level and transmits the error level to the management server so that the object area corresponding to the damaged part and the damage level are re-learned when the error level exceeds a preset second value. Function can be performed. If the probability of matching the failure level is less than 50%, the inferred failure level may be determined as an error. The accuracy of the failure level value can be obtained by means of the failure level list provided by the failure determination algorithm.

4 is a diagram showing the structure of a management server according to embodiments of the present invention.

The management server 300 may include a processor 310, a memory 320, and a communication unit 330 in order to build a damage level determination model. The management server 300 learns a failure level determination model using the previously terminated data. The management server 300 may evaluate the damage level determination model to determine whether or not the accuracy is higher than a certain level.

The processor 310 typically controls the overall operation of the management server 300. For example, the processor 310 may generally control the communication unit 330 by executing programs stored in the memory 320.

The communication unit 330 may include one or more components that enable communication between the user terminals 100 and 400, the automatic damage calculation apparatus 200, and the management server 300. The communication unit 330 includes a Bluetooth communication unit, a Bluetooth Low Energy (BLE) communication unit, a Near Field Communication unit, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, an infrared data association (IrDA) communication unit, a WFD ( It may include a Wi-Fi Direct) communication unit, an ultra wideband (UWB) communication unit, a short-range communication unit such as an Ant+ communication unit, and a mobile communication network.

The memory 320 may include a data input unit 321, an attribute extraction unit 322, a function application unit 323, a collection unit 324, a removal unit 325, and a model construction unit 326.

The data input unit 321 performs a function of inputting data for a damage level determination model to be built. The data input unit 321 may acquire data necessary for the damage level determination model. The data input unit 321 may obtain images of the accident vehicle, a damage level value, and an estimated amount corresponding to the damage level.

The attribute extraction unit 322 may pre-process the acquired data. The attribute extraction unit 322 may process the preprocessed data into a preset format. The attribute extracting unit 322 may perform a preprocessing process such as resizing while increasing or decreasing the resolution of the input image by a predetermined ratio. The attribute extraction unit 322 may select data necessary for the failure level determination model from among the preprocessed data. For example, the attribute extracting unit 322 may select data necessary for a damage level determination model by extracting various attributes (color value, brightness value, transparency, sharpness, etc. for each pixel) included in the preprocessed image. The attribute extraction unit 322 extracts as many attributes as possible from the image. The attribute extracting unit 322 may additionally perform a sampling process that reduces the amount of attributes. The attribute extraction unit 322 may provide data necessary for learning to the function application unit 323.

The function application unit 323 may train a failure level determination model using the input data. In this case, the failure level determination model may be a model that is trained using an artificial neural network algorithm. For example, images of an accident vehicle and damage level information input from a person in charge of damage assessment may be used. The function application unit 323 may generate a failure level determination model by hierarchically generating a correspondence relationship between attributes of input data. The function application unit 323 hierarchically creates a correspondence relationship between the attributes according to the input images and the damage level, and infers the correspondence relationship so that the attributes of the image correspond to the damage level, and By repeating the process of calculating the accuracy through the input data, modeling between the properties of the images and the level of damage can be established.

The model building unit 324 builds a failure level determination model having the highest accuracy for input data by repeatedly performing an attribute extraction unit and a function application unit.

The failure level determination model may be constructed in consideration of the application field of the recognition model, the purpose of learning, or the computer performance of the device. The failure level determination model may be, for example, a model based on a neural network. For example, a model such as a deep neural network (DNN), a recurrent neural network (RNN), or a bidirectional recurrent deep neural network (BRDNN) may be used as a failure level determination model, but is not limited thereto.

The model building unit 324 may train a failure level determination model through supervised learning using inputted training data as an input value. In addition, the model building unit 324 can learn the types of data necessary for determining the level of damage without special guidance, for example, to learn the model for determining the level of damage through unsupervised learning to discover the criterion for determining the level of damage. I can. The model building unit 324 may train the failure level determination model through reinforcement learning using feedback on whether the result of determining the failure level according to the learning is correct.

When the damage level determination model is learned, the model building unit 324 may store the learned damage level determination model. In this case, the model building unit 324 may store the learned damage level determination model in the memory of the automatic damage assessment apparatus 200.

The model evaluation unit 325 may input the evaluation data to the damage level determination model and, if the recognition result output from the evaluation data does not satisfy a predetermined criterion, the model construction unit 324 may retrain. In this case, the evaluation data may be preset data for evaluating the failure level determination model.

For example, the model evaluation unit 325 is a predetermined criterion when the number or ratio of evaluation data in which the failure level determination result is not accurate among the recognition results of the learned damage level determination model for the evaluation data exceeds a preset threshold. Can be evaluated as not satisfied.

On the other hand, when there are a plurality of learned failure level determination models, the model evaluation unit 325 evaluates whether each of the learned failure level determination models satisfies a predetermined criterion, and determines the model that satisfies the predetermined criterion. Can be determined as In this case, when there are a plurality of models that satisfy a predetermined criterion, the model evaluation unit 325 may determine any one or a predetermined number of models previously set in the order of the highest evaluation score as the failure level determination model.

5 is a flowchart of an automatic loss calculation method according to embodiments of the present invention.

In S110, the automatic damage calculation apparatus 200 receives an image set photographing an accident vehicle. The set of images photographing the accident vehicle can be photographed while being controlled by an automatic damage calculation device. For example, when an accident is detected through a black box of a vehicle, the camera of the owner's terminal is automatically executed and the camera can be controlled to be captured at the moment when the accident vehicle is recognized.

In S120, the automatic damage calculation apparatus 200 may determine the vehicle type and year of the accident vehicle in consideration of the image set. In the automatic damage calculation apparatus, the vehicle type and the year of the accident vehicle may be determined based on an image, but may be determined based on the subscription information input when subscribing to automobile insurance.

In S130, the automatic damage calculation apparatus 200 may detect object regions corresponding to the vehicle part in the image set by instant segmentation.

In S140, the automatic damage calculation device 200 calculates the degree of damage using features extracted from object areas using a damage level determination model, which is a damage level algorithm, and matches one or more damage levels inferred from the damage level. You can create a list of failure levels arranged in order of probability size.

In S150, the automatic damage calculation apparatus 200 determines the damage level having the highest matching probability as the damage level of the object area.

In S160, the automatic damage calculation device 200 converts work information according to the damage levels of the object areas, and the automatic damage calculation device calculates costs incurred from detailed work included in the work information using the standard repair recognition criteria. I can.

In S170, the automatic loss calculation apparatus 200 may generate an automatic estimate by synthesizing the calculated costs.

The automatic damage calculation apparatus 200 may perform a third party verification procedure for a damaged vehicle part in order to prevent damage that has not occurred in a corresponding vehicle accident from being repaired.

6 to 8 are flowcharts of methods for learning a failure level determination model, which is an algorithm included in the failure level determination unit according to embodiments of the present invention.

As shown in FIG. 6, the management server 300 learns a damage level determination model by using input data such as images of the accident vehicle, damage level information including damage parts and damage degrees. In FIG. 1, the automatic damage assessment device 200 and the management server 300 are illustrated as separate electronic devices, but are not limited thereto and may be included in one electronic device and implemented. In the description of FIG. 6, a process of processing input data once is described, but steps S210 to S260 may be repeatedly performed depending on the number of input data. The management server 300 receives and learns a large amount of data to complete a model with high accuracy of the damage level.

In S210, the management server 300 includes images of the accident vehicle (vehicle damage picture, work picture, completion picture, vehicle picture not related to the accident, non-vehicle picture, etc.), damage label information including the damaged part and the degree of damage. Receive input data.

In S220, the management server 300 generates training data by pre-processing the input data. The training data is transferred to the model building unit 254 to train the model.

In S230, the management server 300 learns the damage level determination model by using the learning data.

In S240, the management server 300 may output the damaged parts of the accident vehicle, the degree of damage to the damaged parts, and label information according to the degree of damage through the learned damage level determination model.

In S250, the management server 300 evaluates the damage label determination model by comparing the damage label information according to the damage degree of the damaged parts output through the model and the damage label information that has been actually assessed. For example, if the damage label information calculated by the damage label determination model does not differ from the actual damage assessed damage label information by more than a preset value, the evaluation of the damage label determination model is increased.

In S260, the management server 300 stores a damage level determination model that satisfies a predetermined evaluation criterion in the memory.

The management server 300 learns a damage level determination model by using damage label information including images related to the accident vehicle, the actual damaged part, and the degree of damage.

As shown in FIG. 7, step S220 may include a plurality of steps in detail.

In S221, the management server 300 is based on input data (including images collected from accident vehicles and industrial companies, such as vehicle photos, damage photos, photos during repair, repair completion photos, non-vehicle photos, etc.) Extract the work judgment content. The content of the judgment of the main operation may be registered by the person in charge of the damage assessment.

In S222, the management server 300 may classify a vehicle picture and a non-vehicle picture from among images input in relation to the first main job judgment content among the main job judgment content. Algorithms such as Inception v4 can be used to classify vehicle and non-vehicle photos.

In S223, the management server 300 detects one or more vehicle parts included in the vehicle picture using one of Mask R-CNN and Yolo. Other algorithms can be used to detect vehicle parts, not limited to Mask R-CNN and Yolo.

In S224, the management server 300 selects a damaged picture including a damaged vehicle part from among the vehicle pictures.

In S225, the management server 300 stores the damage label information in correspondence with each vehicle part based on the contents recognized in the actual damage assessment within the damage photo, and connects and registers the repair completion photo as a normal photo for the damage photo. Let it.

In S226, the management server 300 loads the damage picture including the damage level as learning data.

In this way, the management server 300 extracts damage level information representing the damage area and the degree of damage to the damaged area, and uses it to learn the damage level determination model. Through such a process, the management server 300 accurately selects images related to vehicle and vehicle damage among input data, and improves accuracy of determining the level of damage from images related to the damaged area.

The damage level information may consist of information such as the damaged part and items necessary to repair the damage. For example, it will be set such as bumper replacement, bumper sheet metal 2 hours, fender replacement, etc.

The flowchart of FIG. 8 shows a method of reinforcement learning the failure level determination model.

As shown in FIG. 8, the management server 300 may continuously repeatedly learn the damage level determination model by using a quality image set that satisfies a certain criterion.

In S310, the management server 300 receives an image set photographing an accident vehicle.

In S320, the management server 300 applies the received image set to the damage level determination model to obtain the damaged part and first damage level information according to the damaged part.

In S330, the management server 300 may determine a similarity between the first damage level information and the second damage level information input through the person in charge of damage assessment. The similarity may be determined as a difference value between the first damage level information and the second damage level information. When a difference value between the first and second damage level information is less than or equal to a preset reference difference value, the similarity may be determined to be high, otherwise the similarity may be determined to be low.

In S340, the management server 300 may update the damage level determination unit of the automatic damage calculation apparatus using the damage level determination model.

Through this, the accuracy of the damage level determination model is increased, and the damage level information can be accurately determined without a person in charge of damage assessment.

The failure level determination model learned by the method of FIG. 6 can be reinforced learning by the method of FIG. 8. The reinforcement-learned failure level determination model is periodically transmitted to the automatic damage calculation device and used to determine the failure level.

In addition, the automatic damage calculation device transmits the accident vehicle data on the newly registered accident to the management server. The management server may receive accident vehicle data (accident vehicle images, actual damage assessed details) from an automatic damage calculation device or a user terminal to learn a damage level determination model.

9 is an exemplary diagram representing damaged parts recognized by the automatic damage calculation apparatus.

The damage level determination unit of the automatic damage calculation device detects object areas corresponding to the vehicle part included in the image of the accident vehicle. The damage level determination unit detects an object area corresponding to the front bumper, front fender L, front fender R, front door L, and front door R of the accident vehicle. In this case, the detected object regions may have an irregular boundary directly surrounding the corresponding part in order to obtain an accurate damaged part name without overlapping each other. As shown in FIG. 9, the damage level determining unit is a front bumper area (FBP), a front fender R area (FFR), a front door R area (FDR), and a rear door R area (RDR) regardless of the shooting angle and image resolution. , The rear fender R region (RFR) can be detected and labeled in the accident vehicle image and provided. As the damage level determination unit recognizes the vehicle part through images taken according to various shooting angles and image resolutions by modeling by the management server, the damaged part is recognized regardless of the image capturing method.

10 is an exemplary diagram of damage level information provided by an automatic damage calculation apparatus according to embodiments of the present invention.

The automatic damage calculation device may set the damage level according to the degree of damage of the recognized vehicle part to one of a bad loss level, a heavy loss level, a burnout level, and an exchange level. At this time, the automatic damage calculation apparatus may infer a damage level and a matching probability for each damaged part, and determine the damage level with the highest matching probability as the damage level of the corresponding damaged part. As illustrated in FIG. 10, a list having the highest probability of the FFR region having the middle loss level, the FDR region having the highest probability of the bad loss level, and the FBP region having the highest probability of the middle loss level may be provided as illustrated in FIG. 10. Accordingly, the automatic loss calculation apparatus determines a medium loss level for the FFR region, a bad loss level for the FDR region, and a medium loss level for the FBP region.

The determined damage level may be provided to the owner of the vehicle, an adjuster in charge of the accident vehicle, an industrial company server of the accident vehicle, and the like.

11 and 12 are diagrams for explaining an image preprocessing process performed by the damage level determination unit 240 of the automatic damage calculation apparatus.

11 is an example of images before pre-processing, and FIG. 12 is an example of images processed through a pre-processing process.

The damage determination algorithm that determines the damage level for each vehicle part used by the damage level determination unit 240 receives an image of the vehicle part in the accident vehicle and the damage level lvl for the vehicle part and is included in the image. The relationship with the failure level lvl can be randomly determined through a classification process according to correspondence between unspecified features. The breakage level determination algorithm intuitively learns the association between the properties of an unprogrammed image and the level of breakage by learning through good quality data, that is, the exact breakage level (lvl) according to the image, and It is possible to infer the level of failure (lvl) through learned intuition. In addition, the damage level determination algorithm itself performs a process of selecting a preprocessing method of an image that increases the accuracy of inferring the damage level lvl among the previously disclosed preprocessing methods. As shown in FIGS. 11 and 12, the failure level determination algorithm may pre-process an image using a Sobel mask, a canny edge detector filter, and a contrast equalization algorithm.

If Img11, img 21, img31, img41, img51, img61, img71, img81, and img91 are preprocessed in a method selected by the damage determination algorithm, img 12, img22, img32, img42, img52, img62 to reveal the desired information more clearly. , img72, img82, and img92 will be created.

13 is an exemplary diagram of a screen comparing an estimate automatically generated by an automatic loss calculation device and an actual estimate for an industrial company.

The automatic damage calculation device may calculate the damage of the accident vehicle at the request of the user and provide an automatic quotation accordingly, and may further perform a process of comparing the provided automatic quotation with the actual quotation generated by the industrial company. The automatic damage calculation device compares the automatic estimate and the actual estimate, and determines whether the difference between the damage level automatically calculated from the image of the accident vehicle and the damage level specified by the industry is large or small. The accuracy of the calculated failure level can be evaluated. If the difference between the automatically calculated damage level and the damage level specified by the industrial company is not large, the data of the accident vehicle is transmitted to the management server and used for reinforcement learning of the damage level determination model.

As shown in FIG. 13, when the vehicle part detected using the damage level determination model and the damage level for each vehicle part are determined, the automatic damage calculation device as shown in FIG. 13 determines the damage level for each vehicle type, vehicle name, year, paint coat, and vehicle part. Considering it, work details can be automatically generated. For example, if bumper sheet metal and painting are required, the time required for bumper sheet metal is calculated according to the level of damage, and labor costs are calculated by applying the hourly wages of current engineers to the required time, and painting work included in painting work , Drying work, etc. will be created in detail.

At this time, the work details are generated according to the standard repair accreditation criteria set according to the damage level of each vehicle part, and the detailed work details included in the work details are extracted.

The automatic estimate can be generated by taking into account not only the cost of restoring the damaged area, but also incidental work such as waiting time for vehicle repair, drying time after painting, and detachment time of other parts when replacing the bumper.

14 is an exemplary diagram of result data acquired through an image. It is possible to assign a management number for each vehicle type, year, painting, and repair part, and set and store the range of authorized work among the subsidiary work by matching the management number. The automatic damage calculation apparatus according to embodiments of the present invention may search for a management number corresponding to the detected vehicle part, and extract an authorization work range for each damage level included in the management number. You can create an automatic quote by listing the scope of the approved work and the repair cost for each work.

The scope of the accreditation work can be modified through an actual estimate of the industry being fed back. For example, the certified work range was 3 hours, but according to the actual estimate of the industry, if the painting time is increased to 4 hours, 30 minutes, 50% of the increased time, can be added as the certified work range. In addition, the database can be stored by standardizing the criteria for accreditation of repair costs and damages.

15A, 15B, and 15C are exemplary diagrams of standard repair recognition criteria used in the automatic damage calculation apparatus. The automatic damage calculation device may search for a code corresponding to a damage level for each damaged part acquired through an image, and obtain a work history for the code and a wage for each work history.

As shown in FIGS. 15A and 15B, a code list for subdividing and identifying a vehicle type and a code list for identifying each vehicle part of the vehicle type may be provided.

The code that identifies the vehicle model by subdividing it can be registered and managed as Avante HD, Avante MD, Avante Hybrid LPi, Avante AD, etc. in detail by option and year, even if it is the same vehicle type.

The codes that identify each vehicle part include front bumper replacement, front bumper repair, left fender replacement, left fender sheet metal, headlamp left replacement, right fender replacement, right fender sheet metal, headlamp right replacement, left door replacement, left door sheet metal, Right door replacement, right door sheet metal, left and rear door replacement, left and rear door sheet metal, right rear door replacement, right rear door sheet metal, left and rear fender replacement, left and rear fender sheet metal, rear combination lamp, right rear fender replacement, right and rear fender sheet metal, etc. .

As shown in FIG. 15C, an automatic estimate of the accident vehicle may be generated in a table including the name of the damaged object, the vehicle type, the details, the type of work, the parts amount, the wages, the main work code, and the auxiliary work code.

According to the identification code (rcp_nv_seqno) assigned for each registered vehicle accident, the work details that must be performed on the accident vehicle are listed on the front bumper side bracket (left), the front bumper, the heating drying cost, the front bumper side bracket (right), the radiator grill, and the front bumper. Exchange, pain, work hours may be included.

Each work history is classified as a main work or an auxiliary work, and the work time is determined according to the damage level of the 00001 incident and the standard work time obtained by the standard repair recognition criteria. For example, considering the degree of damage in the event, the replacement of the front bumper takes 1.91 hours. The painting of the front bumper replacement is set to take 1.98 hours, and an automatic quote can be generated based on it.

Through this, when the damaged part and the damage level are determined through image recognition technology, an estimate for repairing the accident vehicle can be automatically generated by applying the vehicle type, year, damage part, and damage level to the standard repair recognition criteria.

The apparatus described above may be implemented as a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the devices and components described in the embodiments are, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, such as one or more general purpose computers or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of software. For the convenience of understanding, although it is sometimes described that one processing device is used, one of ordinary skill in the art, the processing device is a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to behave as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodyed in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of the program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

As described above, although the embodiments have been described by the limited embodiments and drawings, various modifications and variations are possible from the above description by those of ordinary skill in the art. For example, the described techniques are performed in a different order from the described method, and/or components such as a system, structure, device, circuit, etc. described are combined or combined in a form different from the described method, Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and claims and equivalents fall within the scope of the claims to be described later.

10: Automatic damage calculation system
100, 400: user terminal
101: accident vehicle
200: automatic loss calculation device
300: management server

Claims (7)

Detecting object regions corresponding to a damaged portion of the accident vehicle by using one or more images photographed of the accident vehicle by using an algorithm learned through the one or more image sets input by the device;
Determining, by the device, labels of object regions detected in correspondence with each part of the accident vehicle;
Each object region labeled by the device is filtered by the method adopted through a convolutional neural network (CNN) layer, and is arranged in the order of the level of damage inferred from the degree of damage, which is the result data, and the size of the matching probability based on the filtered images. Generating a failure level list;
Including; and determining the damage level that is ranked highest in the damage level list
The learned algorithm is
Extracting the judgment of the main task from the input data of the details of payment of insurance payments,
Among the main task determination contents, the input data is classified into vehicle photos and non-vehicle photos in relation to the first main task determination contents,
Using one of Mask R-CNN, Yolo, the vehicle part included in the vehicle picture is detected,
Selecting a damaged photo including a damaged vehicle part from among the vehicle photos,
In the damage picture above, damage label information is stored in correspondence with each vehicle part based on the contents recognized at the time of actual damage assessment,
Among the input data, a repair completion picture is linked and registered as a normal picture for the damaged picture,
A method of automatically calculating the degree of damage of the accident vehicle, characterized in that the damage picture including the damage label information is learned as learning data. The method of claim 1,
The step of detecting the object regions
In the set of one or more images, object regions are detected to include regions recognized as bumpers, fenders, and doors of the accident vehicle, but are detected to coincide with irregular boundaries of bumpers, fenders, and doors. How to calculate the degree of damage to the accident vehicle.
delete Using an algorithm learned through a set of one or more labeled and input images, one or more images of the accident vehicle are taken to detect object regions corresponding to the damaged part of the accident vehicle, and are detected in correspondence with each part of the accident vehicle. A damaged portion extracting unit that determines labels of object regions and labels them; And
Filter each labeled object region by the method adopted through the CNN layer, and generate a list of damage levels arranged in the order of the size of the damage level and the matching probability inferred from the damage degree, which is the result data, based on the filtered images,
Including; a damage level inference unit for determining the damage level that ranks highest in the damage level list
The learned algorithm is
Extracting the judgment of the main task from the input data of the details of payment of insurance payments,
Among the main task determination contents, the input data is classified into vehicle photos and non-vehicle photos in relation to the first main task determination contents,
Using one of Mask R-CNN, Yolo, the vehicle part included in the vehicle picture is detected,
Selecting a damaged photo including a damaged vehicle part from among the vehicle photos,
In the damage picture above, damage label information is stored in correspondence with each vehicle part based on the contents recognized at the time of actual damage assessment,
Among the input data, a repair completion picture is linked and registered as a normal picture for the damaged picture,
A device for automatically calculating the degree of damage of the accident vehicle, characterized in that the damage picture including the damage label information is learned as learning data. The method of claim 4,
The damaged part extraction part
In the set of one or more images, object regions are detected to include regions recognized as bumpers, fenders, and doors of the accident vehicle, but are detected to coincide with irregular boundaries of bumpers, fenders, and doors. A device that calculates the degree of damage to the accident vehicle.
delete A computer program stored in a computer-readable storage medium for executing the method of any one of claims 1 to 2 using a computer.

Images