JP2021167991A - Vehicle damage estimation device, estimation program therefor and estimation method therefor - Google Patents

Abstract

To estimate damages to a vehicle flexibly and accurately.SOLUTION: A part determination unit 4 determines an external part of a vehicle in a captured image with reference to a part learning model 9. The part learning model 9 is constructed by supervised learning using data related to vehicle parts as supervised data. An external damage determination unit 5 refers to a state learning model 10 and determines a damaged state of each external part determined by the part determination unit 4 as external damage. The state learning model 10 is constructed by supervised learning using data on the damaged state of external parts as supervised data. Processing is separated into the part determination and the damage determination, and the damage determination of each part is performed after the part determination is performed, whereby the determination accuracy as a whole can be improved.SELECTED DRAWING: Figure 1

Description

The present invention relates to a vehicle damage estimation device for estimating a vehicle damage state from a captured image, an estimation program thereof, and an estimation method.

For example, Patent Document 1 discloses an accident vehicle repair cost estimation system that can accurately and quickly evaluate damage to an accident vehicle and estimate repairs. This estimation system includes a capture means, a storage means, an input means, a display means, a link means, and an estimation means. The capturing means captures the image data of the accident vehicle. The storage means stores the vehicle attribute data necessary for estimating the accident vehicle repair cost. The input means inputs the estimation data necessary for estimating the repair cost of the accident vehicle. The display means displays various data including accident vehicle image data. The linking means determines which part of the vehicle the image data clearly indicates damage. The estimation means simultaneously displays the image data and the vehicle attribute data of the portion corresponding to the image data on the display means. In addition, the estimation means performs estimation processing of the cost required for repairing the accident vehicle based on the estimation data and the vehicle attribute data.

Japanese Unexamined Patent Publication No. 10-197285

However, in the system of Patent Document 1, the degree of damage is determined from the closeness of the image data of the accident vehicle to the image data of the vehicle repaired in the past. Therefore, if there is no appropriate past image data, Judgment accuracy is reduced.

The invention according to the first aspect is a vehicle state determination device having a parts determination unit and a state determination unit. The parts determination unit determines the vehicle parts in the captured image with reference to the parts learning model constructed by the teacher data regarding the vehicle parts. The state determination unit determines the damage state of each vehicle part determined by the parts determination unit with reference to the state learning model constructed by the teacher data regarding the damage state of the vehicle parts. The parts learning model is either or both of the teacher data in which the part of the vehicle part in the vehicle is specified by the bounding box and the teacher data in which the pixel corresponding to the part of the vehicle part in the vehicle is specified for the image of the vehicle. It is learned from the teacher data of the above, and the area extraction and classification of the vehicle parts in the captured image are output in response to the input of the captured image.
The invention according to the second aspect is a vehicle damage estimation device that has a parts determination unit, an external damage determination unit, and an internal damage estimation unit, and estimates a vehicle damage state from an captured image. The parts determination unit determines the external parts of the vehicle in the captured image with reference to the parts learning model. The parts learning model is constructed by supervised learning using data related to vehicle parts as teacher data. The external damage determination unit refers to the state learning model and determines the damage state of each external component determined by the component determination unit as external damage. The state learning model is constructed by supervised learning using data on the damaged state of external parts as supervised data. The internal damage estimation unit estimates the internal damage of the vehicle by referring to the damage case database and extracting damage cases similar to the external damage determined by the external damage determination unit. The damage case database holds past damage cases, including vehicle damage details, including external and internal damage.

According to the present invention, by determining the damaged state of a vehicle based on machine learning, it is possible to flexibly deal with various vehicles including an unknown vehicle. At that time, the processing is separated into the component determination and the damage determination, and the damage determination of each component is performed after the component determination is performed, so that the determination accuracy as a whole can be improved. Further, according to the present invention, the damage state of the external component is determined based on machine learning, and the internal damage is estimated by referring to the damage case database using this as a key. As a result, it is possible to flexibly and accurately estimate damage including not only external damage but also internal damage for various vehicles including unknown vehicles.

Block diagram of vehicle damage estimation device Figure showing screen display example of image reception The figure which shows the screen display example of the surface judgment result Explanatory diagram of object detection algorithm YOLO network configuration diagram The figure which shows the screen display example of the judgment result of an external part The figure which shows the screen display example of the judgment result of external damage Schematic block diagram of the damage case database The figure which shows the screen display example of the estimation result of internal damage Schematic block diagram of the quotation table Figure showing screen display example of estimation result

FIG. 1 is a block diagram of the vehicle damage estimation device 1 according to the present embodiment. The vehicle damage estimation device 1 estimates the damage state of the vehicle including not only external damage but also internal damage from the captured image designated by the user. The vehicle to be estimated is assumed to be a private vehicle in this embodiment, but this is an example, and any vehicle including trucks, buses, motorcycles, etc. should be targeted depending on the design specifications. Can be done. The vehicle damage estimation device 1 can be equivalently realized by installing a computer program on the computer.

The vehicle damage estimation device 1 includes an image reception unit 2, a surface determination unit 3, a parts determination unit 4, an external damage determination unit 5, an internal damage estimation unit 6, an input / output interface 7, and a learning model 8 to 10. And the damage case database 11 are mainly configured. Each processing unit 2 to 6 is connected to a display device 12 via an input / output interface 7, and the input / output interface 7 controls input / output between these and the display device 12. Basically, each process in the processing units 2 to 6 is sequentially performed, but the transfer of the process between adjacent processing units, specifically, the transfer from the surface determination unit 3 to the component determination unit 4, and the component determination. The transition from the unit 4 to the external damage determination unit 5 and the transition from the external damage determination unit 5 to the internal damage estimation unit 6 were approved by the user after allowing the user to correct the determination result. It is done on the condition. This is to improve the estimation accuracy of vehicle damage by appropriately reflecting the user's intention in the processing process. However, some of the process transitions may be done automatically without the condition of user approval. When the display device 12 is connected to the Internet or the like via a network, the input / output interface 7 has a communication function necessary for performing network communication.

The image receiving unit 2 receives an captured image to be estimated from the display device 13, specifically, an image captured by a camera of the appearance of the damaged vehicle. The direction in which the vehicle is imaged may be any of front, rear, and sideways, and may be diagonally forward, diagonally rearward, and the like.

The captured image to be estimated is designated by the user operating the display device 13 while looking at the screen, and is output / uploaded to the vehicle damage estimation device 1. FIG. 2 is a diagram showing an example of screen display of an image reception in the display device 13. The display screen 30 for receiving an image has an image receiving area 31. In the image receiving area 31, thumbnails of captured images to be estimated are displayed. The user specifies an estimation target by designating a specific image file through the file reference button or by dropping a specific image file in the image receiving area 31. As can be understood from the existence of a plurality of blank frames in the image receiving area 31, a plurality of estimation targets can be specified, and even an already designated estimation target can be canceled through the cancel button. When the specification of the estimation target is completed, the user presses the determination start button. By this action, the estimation target is output to the image receiving unit 2. The captured image received by the image receiving unit 2 is output to the surface determination unit 3.

The captured image to be estimated may be a color image, but a grayscale image may be accepted in order to reduce the memory usage. The size of the captured image is appropriately set in consideration of the memory usage of the system and the like.

The surface determination unit 3 determines the constituent surface of the vehicle projected on the captured image to be estimated. Here, the "construction surface" refers to an individual surface that constitutes a three-dimensional vehicle, and includes the front surface, the back surface, and the side surface of the vehicle. For example, in the case of a captured image of the vehicle taken from the front, the determination result is the front side, and in the case of the captured image of the vehicle taken from the rear, the determination result is the back side. However, the number of constituent surfaces obtained as a determination result is not limited to one, and may be multiple. For example, in the case of an captured image of a vehicle taken diagonally from the front, the determination results are front and side surfaces. The reason for determining the configuration surface in this way is to improve the determination accuracy in the post-processing.

The surface learning model 8 is referred to in the determination of the constituent surface. This surface learning model 8 is mainly composed of a neural network system that imitates the human cranial nerves. The neural network system is formed on the work area of a computer and has an input layer, a hidden layer, and an output layer. The input layer is weighted by an activation function when transmitting an input signal to the hidden layer. Then, the transmission with weighting according to the number of layers of the hidden layer is repeated, and the signal transmitted to the output layer is finally output. The number of ports in the input layer, the number of ports in the output layer, the number of layers in the hidden layer, etc. are arbitrary. The output layer also outputs the classification probabilities of the outputs (front, back, sides, etc.).

The surface learning model 8 is constructed by supervised learning using data related to the constituent surfaces of the vehicle as teacher data. The teacher data has a captured image of the vehicle and a classification of the constituent surfaces of the vehicle, and various and large amounts of data are used including various vehicle types, various body colors, various shooting directions, and the like. In supervised learning, the classification probability of the output with respect to the input of supervised data is verified, and the surface learning model 8 is set to a desired state by repeating the adjustment of the activation function (weighting) based on this.

As the surface learning model 8, in addition to the neural network, a machine learning method such as a support vector machine, a decision tree, a Bayesian network, a linear regression, a multivariate analysis, a logistic regression analysis, or a judgment analysis may be used. Further, a convolutional neural network and R-CNN (Regions with CNN features) using the convolutional neural network may be used. This point is the same for the state learning model 10 described later.

Further, in the determination of the constituent surface, in addition to the reference of the surface learning model 8, the constituent surface may be estimated by paying attention to the presence or absence of the unique external component existing in the specific constituent surface. As used herein, the term "external parts" refers to vehicle parts that are exposed in appearance. In addition, among vehicle parts, those that are not exposed in appearance are called "internal parts".
For example, the front light and the front grille are unique to the front surface of the vehicle and have a characteristic shape that can be distinguished from other external parts. Therefore, due to the presence of the front light and the like, at least the front surface is included in the captured image. It can be considered to be included. For this reason, when an external component unique to a specific constituent surface is present in the captured image, the constituent surface corresponding to the unique external component is included in the determination result.

Further, the surface learning model 8 may be reduced in weight in order to suppress the memory usage of the GPU (Graphics Processing Unit) or to improve the learning efficiency. As an example, the number of VGG blocks (1 block by convolution-> convolution-> pooling) in a general-purpose model such as VGG-16 can be reduced, and the size of the captured image can be reduced. VGG-16 is a method of using an already trained model for a CNN (convolutional neural network), that is, one of the transfer learning models, which includes a total of 16 layers including a convolutional layer and a fully connected layer, and has a large convolutional filter. All of them are 3x3, and the fully connected layer consists of 2 layers of 4096 units and 1 layer of 1000 units for classification.

The determination result of the surface determination unit 3 is output to the display device 12. FIG. 3 is a diagram showing a screen display example of the surface determination result displayed on the display device 12. The determination result display screen 40 has an image display area 41, a plurality of determination result display areas 42, and a plurality of check boxes 43. In the image display area 41, a captured image of the vehicle related to the estimation target is displayed. In each determination result display area 42, candidates (front surface, side surface, back surface) of the constituent surface of the vehicle are displayed with a classification probability. The check boxes 43 are provided corresponding to the respective determination result display areas 42, and those satisfying a predetermined condition are marked with a check mark. The predetermined conditions are that the classification probability of the constituent surface candidates is equal to or higher than the predetermined threshold value (in this case, a plurality of check marks may be added), and the constituent surface classification probability is the highest. And so on. When the user determines that the determination result is valid, the user presses the determination continuation button. If the user determines that this is not appropriate, he / she corrects the determination result including the change of the tick mark of the check box 43, and then presses the determination continuation button. By this action, the determination result of the surface determination unit 3 (including the determination result corrected by the user) is output to the component determination unit 4 together with the captured image.

The determination result modified by the user may be reflected in the surface learning model 8. Thereby, the learning depth in the determination of the constituent surface can be deepened.

The component determination unit 4 individually extracts the external component extraction process projected on the captured image, specifically, the component area on which the external component is projected and the attributes of the external component. In this component determination, the component learning model 9 is referred to. This component learning model 9 is constructed based on an object detection algorithm by deep learning such as YOLO and SSD in consideration of multi-scale property and operation speed.

FIG. 4 is an explanatory diagram of the object detection algorithm. As shown in FIG. 6A, the conventional detection method used for face detection and the like is divided into three stages of input processing: area search, feature extraction, and machine learning. That is, first, the area search is performed, then the feature extraction is performed according to the object to be detected, and finally the appropriate machine learning method is selected. In this detection method, object detection is realized by dividing it into three algorithms. As for the feature quantity, basically, only a specific target can be detected because it is designed exclusively for the detection target. Therefore, in order to eliminate such restrictions, an object detection algorithm by deep learning as shown in FIGS. (B) and (c) has been proposed. As shown in FIG. 3B, in R-CNN and the like, feature extraction is automatically realized by using deep learning. This allows flexible classification of various objects simply by designing the network. However, the area search still remains as a separate process. Therefore, the method shown in FIG. 6C, represented by YOLO (You Only Look Once) and SSD (Single Shot MultiBox Detector), includes the area search in the deep learning. In this method, by inputting an input (captured image) to a single neural network, extraction of an item area and classification of its attributes are performed collectively. The first feature of this method is that it is a regression problem approach. Regression is an approach that directly predicts numerical values from the tendency of data, and instead of deciding a region and then classifying what it is, the coordinates and size of an object are directly predicted. Second, the process is completed in a single network. It can also be called "End-to-End" processing in the sense that after data is input, it goes to the end (output result) only by deep learning.

For example, the processing of YOLO is as follows. First, the input image is divided into S × S regions. Next, the classification probabilities of the objects in each region are derived. Then, the parameters (x, y, height, width) and the confidence (confidence) of B bounding boxes (hyperparameters) are calculated. The bounding box is the circumscribed quadrangle of the object area, and the reliability is the degree of agreement between the predicted and the correct bounding box. For object detection, the product of the classification probability of the object and the reliability of each bounding box is used. FIG. 5 is a network configuration diagram of YOLO. In YOLO, the captured image is input to the CNN (Convolutional Neural Network) layer, and the result is output through a plurality of fully connected layers. The output includes an image area divided into S * S pieces, five parameters of a bounding box (BB) including reliability (classification accuracy), and the number of classes (item attributes).

The parts learning model 9 is constructed by supervised learning using data related to external parts for each constituent surface of the vehicle as teacher data. Specifically, the parts learning model 9 is performed by learning the parts of the external parts for each constituent surface of the vehicle from the image of the vehicle by the teacher data specified by the bounding box. Then, by such supervised learning, the parts learning model 9 outputs the region extraction and classification of the vehicle parts in the captured image in response to the input of the captured image.
The teacher data has a partial image of the external part and an attribute of the external part, and various and large amounts of data are used including various vehicle models, various external parts, various shooting directions, and the like. In order to secure a large amount of data, a source image that has undergone image processing is also used. However, in order to distinguish between the right headlamp and the left headlamp, the image is not flipped horizontally, which is one of the image processing. By using such a parts learning model 9, it is possible to determine and extract external parts included in the constituent surface of the vehicle regardless of the presence or absence of damage to the vehicle and its degree.

The parts determination unit 4 filters the external parts specified based on the parts learning model 9 based on the constituent surface of the vehicle determined by the surface determination unit 3, and outputs the filtered external parts as the determination result. For example, when the determination result of the constituent surface is the front surface and the side surface, it is clear that this is an erroneous determination even if the tail lamp is extracted as a result of the component determination. This is because the tail lamps are on the back and cannot be on the front and sides. Such correlation is also recognized for side doors unique to the sides, headlights specific to the front, front grilles, front windows, and the like. Therefore, among the external parts obtained as the result of the part judgment, only the parts related to the constituent surface obtained as the judgment result of the constituent surface are included in the judgment result, and the others are excluded to improve the parts judgment accuracy. Can be planned.

The determination result of the component determination unit 4 is output to the display device 13. FIG. 6 is a diagram showing a screen display example of the component determination result. The determination result display screen 50 has an image display area 51, a plurality of determination result display areas 52, and a plurality of check boxes 53. In the image display area 51, a captured image of the vehicle related to the estimation target is displayed, and a rectangular frame showing individual external parts extracted by the part determination unit 4 is displayed. In the determination result display area 52, candidates for each external component (right headlight, left headlight, front window, etc.) are displayed with a classification probability. The check boxes 53 are provided corresponding to the respective determination result display areas 52, and those satisfying a predetermined condition are marked with a check mark. As a predetermined condition, typically, the classification probability of the candidate of the constituent surface is equal to or higher than the predetermined threshold value. When the user determines that the determination result is valid, the user presses the determination continuation button. If the user determines that this is not appropriate, he / she corrects the determination result including the change of the tick mark of the check box 53, and then presses the determination continuation button. By this action, the determination result of the component determination unit 4 (including the determination result corrected by the user) is output to the external damage determination unit 5 together with the captured image.

The determination result corrected by the user may be reflected in the component learning model 9. As a result, the learning depth of component determination can be deepened.

The external damage determination unit 5 determines the damage state of each external component extracted by the component determination unit 4. Then, the determination result summarizing the damage states of the plurality of external parts is output as external damage. The determination of external damage is performed by referring to the state learning model 10 constructed by supervised learning using supervised data on the damaged state of the external component. The configuration of the state learning model 10 basically conforms to that of the surface learning model 8.

Further, the state learning model 10 is constructed by supervised learning using data on the damaged state of the external component as teacher data. This teacher data contains a partial image of the damaged external part and the attributes that classify the damaged state (for example, "replacement", "detachment", damage degree "large", "medium", "small"). It has a large amount of diverse data, including various vehicle models, various external parts, and various shooting directions. Here, "replacement" means damage to the extent that the external parts themselves must be replaced rather than being repaired. "Detachment" means that the external parts themselves are not damaged, but must be removed from the vehicle body in order to replace or repair other damaged vehicle parts. In addition, in order to secure a large amount of data, a certain source image that has undergone image processing may be used. By determining the damaged state for each external component using such a state learning model 10, it is possible to improve the accuracy of determining the damaged state.

In the present embodiment, the external damage determination unit 5 uses a network in which only the fully connected layer is replaced based on the general-purpose model such as the VGG-16 described above. Further, since the amount of data varies widely for each external component, fine tuning may be performed by the weights learned in advance by ImageN et in order to perform learning even with a small amount of data. Fine tuning is a method of constructing a new model by reusing a part of an existing model. Furthermore, in order to deal with the case where an image that is slightly pulled for each data and an image of a different type from the up image are mixed, the up image may be selected and cleansing may be performed by trimming the target portion (trimming). Some of the images that have been cropped are also used).

The determination result of the external damage determination unit 5 is output to the display device 12. FIG. 7 is a diagram showing a screen display example of the determination result of external damage displayed on the display device 12. The determination result display screen 60 has an image display area 61, a plurality of determination result display areas 62, and a plurality of check boxes 63. In the image display area 61, a captured image of the vehicle related to the estimation target is displayed, and a round frame indicating the damaged portion determined by the external damage determining unit 5 is displayed. In the determination result display area 62, candidates for the damaged state (replacement, desorption, large, medium, small) are displayed with a classification probability. The check boxes 63 are provided corresponding to the respective determination result display areas 62, and those satisfying a predetermined condition are marked with a check mark. A predetermined condition typically includes a probability of classification of damage degree candidates greater than or equal to a predetermined threshold. When the user determines that the determination result is appropriate, the user presses the estimate start continuation button. If the user determines that this is not appropriate, he / she corrects the determination result including the change of the tick mark of the check box 63, and then presses the estimation start button. By this action, the determination result of the external damage determination unit 5 (including the determination result corrected by the user) is output to the internal damage estimation unit 6 together with the captured image.

The determination result corrected by the user may be reflected in the state learning model 10. As a result, the learning depth for determining the damage state can be deepened.

The internal damage estimation unit 6 estimates the internal damage of the vehicle by referring to the damage case database 11 and extracting damage cases similar to the external damage determined by the external damage determination unit 5. FIG. 8 is a schematic configuration diagram of the damage case database 11. The damage case database 11 holds a large amount of actual damage cases that have occurred in the past (big data). Each damage case includes as a log the damage content of the vehicle, including external and internal damage. Specifically, for each vehicle part (including both external parts and internal parts) that composes the vehicle, the degree of damage is displayed as a percentage, such as 0% for no damage and 100% for total loss. There is. For example, in damage case 1, the damage rate of the front bumper is 81% (large), the damage rate of the front window is 53% (medium), the damage rate of the radiator is 17% (small), and the damage rate of the core support is 12% ( Small). Then, when the determination result of the external damage determination unit 5 is, for example, the damage state of the front bumper is "large" and the damage state of the front window is "medium", the damage case 1 is extracted as similar to this. .. As a result, the damage state of the radiator is "small" and the damage state of the core support is "small" as the estimation result of the internal parts.

When a plurality of damage cases similar to the external damage determined by the external damage determination unit 5 are extracted, the internal damage of the vehicle is estimated by statistical processing using these as samples. For example, for a certain internal component, a histogram of the damaged state divided into predetermined ranges is taken, and the damaged state having the highest frequency of appearance is adopted.

The estimation result of the internal damage estimation unit 6 is output to the display device 12 as the estimation result of the vehicle damage including the external damage. FIG. 9 is a diagram showing a screen display example of the vehicle damage estimation result displayed on the display device 12. The estimation result display screen 70 has an image display area 71 and a result display area 72. In the image display area 71, a captured image of the vehicle related to the estimation target is displayed. In the result display area 72, the damaged vehicle parts (including external parts and internal parts) and the degree of damage are displayed. When the user presses the end button, a series of processes in the vehicle damage estimation device 1 are completed.

Further, the internal damage estimation unit 6 may refer to the estimation table to estimate the cost required for repairing the vehicle related to the estimation target and then present it to the user. In this estimation table, the vehicle part name including the external part and the internal part, the degree of damage, and the cost including the wage are kept in association with each other. For the external parts, the cost is specified based on the external parts specified by the part determination unit 4 and the damage state determined by the external damage determination unit 5. For internal parts, the cost is calculated based on the internal parts specified by the internal damage estimation unit 6 and the damaged state thereof. The cost required to repair the vehicle is the total of the costs for each vehicle part. The cost required for repairing the vehicle is displayed in the result display area 72 on the display screen shown in FIG.

If the design specifications are such that the vehicle type is specified by the analysis of the captured image, or the vehicle type is specified by the user, and the cost is calculated on the assumption that the vehicle type is known, the estimation table is used. The vehicle type (for example, "ABC") may be included as an item of. In this way, by setting the cost in detail for each vehicle type, it is possible to accurately calculate the estimated amount according to the vehicle type.

As described above, according to the present embodiment, the damage state of the external component is determined based on machine learning, and the internal damage is estimated by referring to the damage case database 11 using this as a key. As a result, it is possible to flexibly and accurately estimate damage including not only external damage but also internal damage for various vehicles including unknown vehicles.

Further, according to the present embodiment, for component determination in which many external components can be detected as objects in the captured image, the captured image is input to a single neural network system as represented by YOLO and SSD. , Adopt an object detection algorithm that collectively extracts the area of vehicle parts with attribute classification by a regression problematic approach. This makes it possible to speed up the processing in the component determination unit 4.

Further, according to the present embodiment, prior to the processing of the component determination unit 4, the surface determination unit 3 identifies the constituent surface of the vehicle in the captured image, and then executes the component determination for the captured image. As a result, among the vehicle parts obtained as the determination result of the component determination unit 4, those that cannot exist on the constituent surface specified by the surface determination unit 3 can be excluded as erroneous determination, so that the accuracy of component determination is improved. Can be planned.

Further, according to the present embodiment, the internal damage estimation unit 6 estimates the repair cost of the vehicle and then presents the cost required for the repair of the vehicle to the user. As a result, the convenience of the user can be further enhanced.

(Modification example 1)
The vehicle damage estimation device 1 according to the present embodiment may have an estimation calculation unit 16 in place of or in addition to the internal damage estimation unit 6. Here, when the user presses the estimation start button, the determination result (including the determination result corrected by the user) of the external damage determination unit (state determination unit) 5 is output to the estimation calculation unit 16 together with the captured image. .. The determination result corrected by the user may be reflected in the state learning model 10. As a result, the learning depth for determining the damage state can be deepened.

The estimation calculation unit 16 estimates the cost required for repairing the vehicle related to the determination target by referring to the estimation table 21. FIG. 10 is a schematic configuration diagram of the estimation table 21. The estimation table 21 holds the vehicle part name, the degree of damage, and the cost including wages in association with each other. The cost is specified for each vehicle part by searching the estimation table 21 using the vehicle parts specified by the parts determination unit 4 and the damage state of the vehicle parts determined by the external damage determination unit 5 as keys. The cost required to repair the vehicle is the total of the costs for each vehicle part. For example, when the parts determination unit 4 extracts the "front bumper" and the "headlight", and the external damage determination unit 5 determines that the degree of damage to the front vehicle is "medium" and the degree of damage to the latter is "small", the former The wage of is "200,000 yen", the wage of the latter is "30,000 yen", and the total amount is "230,000 yen".

If the design specifications are such that the vehicle type is specified by the analysis of the captured image, or the vehicle type is specified by the user, and the cost is calculated on the assumption that the vehicle type is known, the estimation table is used. The vehicle type (for example, "ABC") may be included as the item of 21. In this way, by setting the cost in detail for each vehicle type, it is possible to accurately calculate the estimated amount according to the vehicle type.

The estimation result of the estimation calculation unit 16 is output to the display device 12. FIG. 11 is a diagram showing a screen display example of the estimation result displayed on the display device 12. The estimation result display screen 70 has an image display area 71 and a result display area 72. In the image display area 71, a captured image of the vehicle related to the determination target is displayed. In the result display area 72, the damaged vehicle parts (parts), the degree of damage, and the cost are displayed. If there are multiple damaged vehicle parts, the total cost will be displayed along with the individual costs. When the user presses the end button, a series of processes in the vehicle damage estimation device 1 are completed.

As described above, by providing the estimation calculation unit 16 that estimates the vehicle repair cost from the damage state of each vehicle part determined by the external damage determination unit 5, the cost required for vehicle repair is presented to the user. As a result, the convenience of the user can be further enhanced.

(Modification 2)
In the vehicle damage estimation device 1 according to the present embodiment, the area to be determined is limited by constructing the surface determination unit 3 and the component determination unit 4 as separate models. Thereby, the determination accuracy can be improved. However, the present invention is not limited to such a form, and the surface determination unit 3 and the component determination unit 4 can be constructed as the same model.

(Modification example 3)
Further, in the present embodiment, the parts learning model 9 is constructed by learning the vehicle parts with the teacher data specified by the bounding box with respect to the image of the vehicle, but the parts learning model 9 is constructed. Is not limited to this. For example, the parts learning model 9 may be constructed by learning the image of the vehicle with the teacher data in which the pixels corresponding to the parts of the vehicle parts are specified. Specifically, it may be constructed by a network structure such as U-Net. Further, the component learning model 9 may be a model in which these are integrated. That is, the parts learning model 9 is teacher data in which the part of the vehicle parts is designated by the bounding box for the image of the vehicle, and the teacher data (Mask R-CNN) in which the pixels corresponding to the parts of the vehicle are designated. ) May be constructed by learning. In short, the object detection algorithm according to the present embodiment is not limited to a model such as YOLO or SSD, and may be a model such as Semantic segmentation or Instance segmentation. In addition, any model capable of extracting and classifying the area of the object can be adopted.

(Modification example 4)
Further, in the present embodiment, regarding the external damage determination unit 5, the configuration of the state learning model 10 is similar to that of the surface learning model 8, but the present embodiment is not limited to such a form. The state learning model 10 may adopt the same object detection algorithm as the component learning model 9. Specifically, the state learning model 10 can improve the determination accuracy by using the above-mentioned object detection algorithm in determining the state of dents and / or scratches.

1 Vehicle damage estimation device 2 Image reception unit 3 Surface judgment unit 4 Parts judgment unit 5 External damage judgment unit 6 Internal damage estimation unit 7 Input / output interface 8 Surface learning model 9 Parts learning model 10 State learning model 11 Damage case database 12 Display device 16 Estimate calculation unit 21 Estimate table

Claims (15)

In the vehicle damage estimation device that estimates the damage state of the vehicle from the captured image,
A parts determination unit that determines external parts of the vehicle in the captured image by referring to the parts learning model constructed by supervised learning using data related to vehicle parts as teacher data.
External damage determination that determines the damage state of each external part determined by the component determination unit as external damage by referring to the state learning model constructed by supervised learning using data related to the damage state of external parts as supervised learning. With a department,
The parts learning model is either the teacher data in which the part of the external part in the vehicle is specified by the bounding box or the teacher data in which the pixel corresponding to the part of the external part in the vehicle is specified with respect to the image of the vehicle. It is constructed by learning with both teacher data, and outputs the region extraction and classification of the external component in the captured image in response to the input of the captured image.
Vehicle damage estimation device. The component learning model is constructed by YOLO or SSD.
The vehicle damage estimation device according to claim 1. The component learning model is constructed by Mask R-CNN.
The vehicle damage estimation device according to claim 1. It also has a surface determination unit that determines the constituent surface of the vehicle in the captured image by referring to the surface learning model constructed by supervised learning using the data related to the constituent surface of the vehicle as supervised learning.
The component determination unit outputs an external component as a determination result based on the constituent surface determined by the surface determination unit.
The vehicle damage estimation device according to any one of claims 1 to 3. When an external component unique to a specific constituent surface is present in the captured image, the surface determination unit includes the constituent surface corresponding to the unique external component in the determination result.
The vehicle damage estimation device according to claim 4. By referring to the estimation table in which the repair cost associated with the damaged state of the external part is held for each external part, the repair cost of the vehicle is estimated from the damaged state of each external part determined by the state determination unit. It also has an estimate calculation unit,
The vehicle damage estimation device according to any one of claims 1 to 5. At least one of the transfer of the process from the surface determination unit to the component determination unit, the transfer of the process from the component determination unit to the state determination unit, and the transfer of the process from the state determination unit to the estimate calculation unit. One is performed on the condition that the user's approval is obtained after allowing the user to modify the judgment result.
The vehicle damage estimation device according to any one of claims 1 to 6. By referring to the damage case database that holds past damage cases including the damage contents of the vehicle including external damage and internal damage, and extracting damage cases similar to the external damage determined by the external damage determination unit, the vehicle Internal damage estimation unit, which estimates the internal damage of
The vehicle damage estimation device according to any one of claims 1 to 7, further comprising. In the vehicle damage estimation device that estimates the damage state of the vehicle from the captured image,
A parts determination unit that determines external parts of the vehicle in the captured image by referring to the parts learning model constructed by supervised learning using data related to vehicle parts as teacher data.
External damage determination that determines the damage state of each external part determined by the component determination unit as external damage by referring to the state learning model constructed by supervised learning using data related to the damage state of external parts as supervised learning. Department and
By referring to the damage case database that holds past damage cases including the damage contents of the vehicle including external damage and internal damage, and extracting damage cases similar to the external damage determined by the external damage determination unit, the vehicle Internal damage estimation unit that estimates the internal damage of
A vehicle damage estimation device equipped with. The internal damage estimation unit estimates the internal damage of the vehicle based on a plurality of damage cases similar to the external damage determined by the external damage determination unit.
The vehicle damage estimation device according to claim 9. The internal damage estimation unit
The vehicle repair cost is estimated based on the external damage determined by the external damage determination unit and the internal damage estimated by the internal damage estimation unit.
The vehicle damage estimation device according to claim 9 or 10. In a vehicle damage estimation program that estimates the damage state of a vehicle from captured images,
With reference to the parts learning model constructed by supervised learning using data related to vehicle parts as teacher data, a part determination step for determining external parts of the vehicle in the captured image, and
External damage determination that determines the damage state of each external part determined by the component determination unit as external damage by referring to the state learning model constructed by supervised learning using data related to the damage state of external parts as supervised learning. Steps and
Is to make the computer execute the process having
The parts learning model is either the teacher data in which the part of the external part in the vehicle is specified by the bounding box or the teacher data in which the pixel corresponding to the part of the external part in the vehicle is specified with respect to the image of the vehicle. It is constructed by learning with both teacher data, and outputs the region extraction and classification of the external component in the captured image in response to the input of the captured image.
Vehicle damage estimation program. In a vehicle damage estimation program that estimates the damage state of a vehicle from captured images,
With reference to the parts learning model constructed by supervised learning using data related to vehicle parts as teacher data, a part determination step for determining external parts of the vehicle in the captured image, and
External damage determination that determines the damage state of each external part determined by the component determination unit as external damage by referring to the state learning model constructed by supervised learning using data related to the damage state of external parts as supervised learning. Steps and
By referring to the damage case database that holds past damage cases including the damage contents of the vehicle including external damage and internal damage, and extracting damage cases similar to the external damage determined by the external damage determination unit, the vehicle Internal damage estimation step to estimate internal damage and
Vehicle damage estimation program with. In the vehicle damage estimation method for estimating the damage state of the vehicle from the captured image,
With reference to the parts learning model constructed by supervised learning using data related to vehicle parts as teacher data, a part determination step for determining external parts of the vehicle in the captured image, and
External damage determination that determines the damage state of each external part determined by the component determination unit as external damage by referring to the state learning model constructed by supervised learning using data related to the damage state of external parts as supervised learning. Steps and
Including
The parts learning model is either the teacher data in which the part of the external part in the vehicle is specified by the bounding box or the teacher data in which the pixel corresponding to the part of the external part in the vehicle is specified with respect to the image of the vehicle. It is constructed by learning with both teacher data, and outputs the region extraction and classification of the external component in the captured image in response to the input of the captured image.
Vehicle damage estimation method. In the vehicle damage estimation method for estimating the damage state of the vehicle from the captured image,
With reference to the parts learning model constructed by supervised learning using data related to vehicle parts as teacher data, a part determination step for determining external parts of the vehicle in the captured image, and
External damage determination that determines the damage state of each external part determined by the component determination unit as external damage by referring to the state learning model constructed by supervised learning using data related to the damage state of external parts as supervised learning. Steps and
By referring to the damage case database that holds past damage cases including the damage contents of the vehicle including external damage and internal damage, and extracting damage cases similar to the external damage determined by the external damage determination unit, the vehicle Internal damage estimation step to estimate internal damage and
Vehicle damage estimation method including.

Images